Image processing device, image processing system, and image processing method

Abstract

The user can easily and efficiently review the past and / or current circumstances of the person or vehicle captured in the video data. [Solution] The image processing device acquires multiple images that meet predetermined conditions from multiple images captured by at least one imaging device as multiple target images, and displays the target images captured in a first time period in a first area of ​​the screen, and the target images captured in a second time period in a second area of ​​the screen, the second time period being a time period that is updated as the current time progresses, and the first time period being a past time period that is not updated as the current time progresses.

Description

Technical Field

[0001] The present disclosure relates to a video processing apparatus, a video processing system, and a video processing method.

Background Art

[0002] In recent years, with the improvement of security awareness, a large number of surveillance cameras have been installed in buildings and / or on streets. Video data captured by surveillance cameras is stored in a storage device and viewed as needed.

[0003] In Patent Document 1, a storage device holds a plurality of images captured by a plurality of imaging devices. When an image of a moving object is specified as a search query, a processor executes a similarity search targeting the plurality of images held in the storage device with the specified image as the search query. A display device displays the plurality of images obtained as a result of the similarity search. When an image of a moving object is additionally specified as a search query, the processor executes a similarity search with all the specified images as the search query, and the display device updates the displayed images so as to display the plurality of images obtained as a result of the similarity search with all the specified images as the search query. An information processing system is disclosed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, when a user checks the past situation of a target person or vehicle from the stored video data, or checks the current situation of a target person or vehicle from the video data being captured by a surveillance camera, the conventional operations are complicated and thus time-consuming for confirmation, and it is difficult to use in, for example, an urgent situation.

[0006] This disclosure aims to provide a technology that allows users to easily and efficiently confirm the past and / or present circumstances of a person or vehicle, etc., captured in video data. [Means for solving the problem]

[0007] One aspect of the present disclosure provides an image processing device comprising a processor and a memory, wherein the processor cooperates with the memory to acquire a plurality of images that meet predetermined conditions from a plurality of images captured by at least one imaging device as a plurality of target images, and displays the target images captured in a first time period in a first area of ​​the screen and the target images captured in a second time period in a second area of ​​the screen, wherein the second time period is a time period that is updated as the current time progresses, and the first time period is a past time period that is not updated as the current time progresses.

[0008] One aspect of the present disclosure provides an image processing system comprising: at least one imaging device; and a display control device that acquires a plurality of images conforming to predetermined conditions as a plurality of target images from a plurality of images captured by the at least one imaging device, displays the target images captured in a first time period in a first area of ​​the screen, and displays the target images captured in a second time period in a second area of ​​the screen, wherein the second time period is a time period that is updated as the current time progresses, and the first time period is a past time period that is not updated as the current time progresses.

[0009] One aspect of the present disclosure provides an image processing method comprising: acquiring a plurality of images that meet predetermined conditions from a plurality of images captured by at least one imaging device as a plurality of target images; displaying the target images captured in a first time period in a first area of ​​a screen, and displaying the target images captured in a second time period in a second area of ​​the screen, wherein the second time period is a time period that is updated as the current time progresses, and the first time period is a past time period that is not updated as the current time progresses.

[0010] These comprehensive or specific embodiments may be implemented as systems, devices, methods, integrated circuits, computer programs, or recording media, or as any combination of systems, devices, methods, integrated circuits, computer programs, and recording media. [Effects of the Invention]

[0011] According to this disclosure, users can easily and efficiently confirm the past and / or present circumstances of a person or vehicle, etc., captured in the video data. [Brief explanation of the drawing]

[0012] [Figure 1] Block diagram showing an example configuration of the video management system according to Embodiment 1. [Figure 2] A diagram showing an example of the setting screen for the search or tracking target according to Embodiment 1. [Figure 3] This figure shows an example of a search results screen with time zones set in two columns according to Embodiment 1. [Figure 4] This figure shows an example of a search results screen in which the time zones according to Embodiment 1 are set to three columns. [Figure 5] Block diagram showing an example configuration of the video management system according to Embodiment 2. [Figure 6] A diagram showing an example of the search results screen according to Embodiment 2. [Figure 7] Block diagram showing an example of the hardware configuration of the information processing device related to this disclosure. [Modes for carrying out the invention]

[0013] Embodiments of the present disclosure will be described in detail below, with appropriate reference to the drawings. However, descriptions that are unnecessarily detailed may be omitted. For example, detailed descriptions of already well-known matters and redundant descriptions of substantially identical configurations may be omitted. This is to avoid the following description becoming unnecessarily verbose and to facilitate understanding for those skilled in the art. The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure and are not intended to limit the subject matter of the claims. The functions of one configuration shown in this embodiment may be realized by two or more physical configurations, or the functions of two or more configurations may be realized by, for example, one physical configuration.

[0014] (Embodiment 1) <System Configuration> Figure 1 is a block diagram showing an example configuration of the video management system 10 according to Embodiment 1.

[0015] The video management system 10 is a system for acquiring, storing, and analyzing video data captured by numerous surveillance cameras 12 installed inside buildings and / or in urban areas. Users may search and confirm specific people or vehicles, etc., from the video data through the video management system 10. This allows users to search and track where a specific person or vehicle, etc., was located at what time, where it is currently located, and / or what route it traveled. The object of search and tracking is not limited to people or vehicles, but may be any object.

[0016] The video management system 10 includes a plurality of surveillance cameras 12, a video distribution device 13, an image analysis device 14, a video storage device 15, an image matching device 16, and an analysis result display device 17. These devices can transmit and receive data and information to and from each other through a predetermined communication network 11. Examples of the communication network 11 include a wired Local Area Network (LAN), a wireless LAN, the Internet, various dedicated line networks, and a mobile communication network, etc.

[0017] Note that the video management system 10 may be read as a video processing system. Also, the surveillance camera 12 may be read as a photographing device. Further, the analysis result display device 17 may be read as a video processing device or an information processing device 1000 (see FIG. 7). Also, the video distribution device 13, the image analysis device 14, the video storage device 15, the image matching device 16, and the analysis result display device 17 do not necessarily have to be separate devices, and may be realized as at least one device.

[0018] The surveillance cameras 12 are installed at various locations for surveillance (such as inside a building and / or in the street, etc.). The surveillance cameras 12 transmit the captured video data to the video distribution device 13 through the communication network 11. Note that the surveillance cameras 12 may directly transmit the captured video data to other devices without going through the video distribution device 13. The video data may be video data or a plurality of still image data captured at regular intervals.

[0019] The video distribution device 13 distributes the video data received from the surveillance cameras 12 to the image analysis device 14. Such distribution may be multicast distribution. Also, in the case of a configuration where the surveillance cameras 12 directly transmit the captured video data to other devices, the video distribution device 13 may not be provided.

[0020] The image analysis device 14 performs image analysis of the video data received from the video distribution device 13. Through image analysis, it detects people or vehicles that may be targets of surveillance from the video data and identifies the time period in which they were detected. For example, the image analysis device 14 generates detection information that associates the identification information of the surveillance camera 12 that captured the video data in which the person or vehicle was detected, the time period in the video data in which the person or vehicle was detected, and the range within the image frame in which the person or vehicle was detected during that time period, and transmits this information to the video storage device 15 along with the video data.

[0021] The video storage device 15 stores video data and detection information received from the image analysis device 14. The image analysis device 14 may also acquire video data from the video storage device 15 and perform image analysis.

[0022] The image matching device 16 performs matching on people or vehicles detected by image analysis. For example, the image matching device 16 searches for other images in which the same person is photographed as the person photographed in a given image. For example, the image matching device 16 searches for other images in which the same vehicle is photographed as the vehicle photographed in a given image.

[0023] The analysis result display device 17 searches for people or vehicles that match the search criteria from the video data stored in the video storage device 15, and displays the search results on the search results screen. The analysis result display device 17 may also perform the search in cooperation with the image analysis device 14 and / or the image matching device 16.

[0024] For example, a user inputs information about a person to be monitored or tracked (hereinafter referred to as the "target person") as search criteria into the analysis result display device 17. The analysis result display device 17 works in conjunction with the image analysis device 14 to detect image frames from the video data in which the target person matching the search criteria is captured, extracts the area within the detected image frame in which the target person is captured, and displays it as an image of the target person.

[0025] This allows users to check the past and present circumstances of a target person and track them through the analysis result display device 17. However, since the images of the target person detected from the numerous video data provided by numerous surveillance cameras 12 are numerous and are detected at various times, it is not easy for users to grasp the past and present circumstances of a target person or to track them using the conventional search results screen display.

[0026] Therefore, this embodiment provides a search results screen that allows users to more easily understand the past and present circumstances of a target person and to track that person. While the following describes an example where a person is the target of monitoring or tracking, the target of monitoring or tracking is not limited to a person; for example, a vehicle or an object may also be used.

[0027] <Search or tracking settings screen> Figure 2 shows an example of the screen for setting the search or tracking target according to Embodiment 1.

[0028] The analysis result display device 17 displays a screen for setting the search or tracking target (hereinafter referred to as the target setting screen 100) as shown in Figure 2. The target setting screen 100 consists of a camera area 110, a captured image area 120, a search or tracking target selection area 130, and a period input area 140.

[0029] The camera area 110 may display camera identification information 111, a camera image 112, and a checkbox 113. The camera identification information 111 is information for identifying the surveillance camera 12, such as the ID, number, or name of the surveillance camera 12. The camera image 112 is an image taken by the surveillance camera 12 corresponding to the camera area 110. The checkbox 113 can be turned ON / OFF by the user. As will be described later, one or more camera areas 110 may be managed as a camera group.

[0030] The captured image area 120 displays the image captured by the surveillance camera 12 indicated by the camera area 110 as a thumbnail image 101. As shown in Figure 2, in the target setting screen 100, the camera area 110 and the captured image area 120 are displayed in association (i.e., as a set).

[0031] As shown in Figure 2, in the target setting screen 100, multiple sets of camera area 110 and captured image area 120 may be arranged vertically. Multiple sets of camera area 110 and captured image area 120 arranged vertically can be scrolled vertically by the user. Furthermore, each unit of camera area 110 can also be scrolled vertically.

[0032] If the checkbox 113 for the camera area 110 is turned OFF, the captured image area 120 associated with the camera area 110 may hide the thumbnail image 101. This allows the user to adjust the number of thumbnail images 101 displayed, making it easier to find the desired thumbnail image 101.

[0033] The period input area 140 is where the search period is entered as one of the search conditions.

[0034] The user may operate the target setting screen 100 as in the following steps S11 to S15 to set the person to be searched or monitored (target person).

[0035] (S11) The user sets the search criteria for image analysis. Details of the search criteria will be described later.

[0036] (S12) The analysis result display device 17 works in conjunction with the image analysis device 14 to perform image analysis on the video data stored in the video storage device 15, and extracts from the video data image frames in which people matching the search conditions set in step S11 are captured. Furthermore, the analysis result display device 17 extracts from the image frames the range in which people matching the search conditions are captured.

[0037] (S13) The analysis result display device 17 displays the range extracted from the image frame in step S12 as a thumbnail image 101, and arranges the thumbnail image 101 in the captured image area 120 corresponding to the camera area 110 of the surveillance camera 12 that took the picture. At this time, the information can be rearranged in units of the camera area 110.

[0038] (S14) The user selects a thumbnail image 101 of the person to be searched or tracked (i.e., the target person) from among the displayed thumbnail images 101. For example, the user drags and drops the target person's thumbnail image 101 into the selection area 130 for searching or tracking (see the dotted arrow in Figure 2). The analysis result display device 17 displays the thumbnail image 101 of the target person selected by the user in the selection area 130 for searching or tracking.

[0039] (S15) The analysis result display device 17 sets the person included in the selected thumbnail image 101 as the target person. For example, the analysis result display device 17 works in conjunction with the image analysis device 14 to analyze the characteristics of the person included in the selected thumbnail image 101 and sets the information indicating the characteristics of that person as the target person. Note that there may be multiple target people set.

[0040] Although not shown in Figure 2, the target settings screen 100 may display a UI that provides at least one of the following functions (A1) to (A7).

[0041] (A1) A function to select the surveillance camera 12 to be set in the camera area 210 on the search results screen 200 (see Figures 3 and 4) and 300 (see Figure 6), which will be described later. (A2) Function to select camera groups. A camera group is a collection of multiple surveillance cameras 12 grouped together. For example, multiple surveillance cameras 12 installed on the same floor or in the same room of a building may be grouped together. For example, multiple surveillance cameras 12 placed in a designated area in a city may be grouped together. When selecting surveillance cameras 12 as described in (A1) above, the user can select multiple surveillance cameras 12 together by selecting a camera group. (A3) As one of the search conditions for the image analysis described above, there is a function to set the shooting period of the captured images in the period input area 140. (A4) A function to change the display size of thumbnail image 101. (A5) A function to search for thumbnail images 101 in which a person is presumed to be the same as or similar to a given person. This function may be implemented in cooperation with an image matching device 16. (A6) A function to detect individuals who are suspected of exhibiting abnormal behavior. This function may be implemented in cooperation with the image analysis device 14. (A7) A function to detect a person or vehicle that matches the characteristics specified by the user. For a person, examples of characteristics that can be specified include height, clothing, accessories (e.g., jewelry, watch, etc.), and facial features. For a vehicle, examples of characteristics that can be specified include vehicle type (e.g., truck, bus, car, motorcycle, bicycle, etc.), vehicle color, and license plate number. This function may be implemented in cooperation with the image analysis device 14.

[0042] The user can use the above function to search for and set a target person from the video data stored in the video storage device 15.

[0043] <Search results screen with 2 columns> Figure 3 shows an example of a search results screen 200 according to Embodiment 1, in which the time zone area 220 is set to two columns.

[0044] The analysis result display device 17 displays multiple images of the target person that have been detected as a target for search or tracking as thumbnail images 201 on the search results screen 200. As shown in Figure 3, when the time zone area 220 is set to two columns, the search results screen 200 is composed of a camera area 210 and time zone areas 220A and 220B. Furthermore, the search results screen 200 is composed of a period input area 230A corresponding to time zone area 220A and a period input area 230B corresponding to time zone area 220B.

[0045] The camera area 210 displays information about the surveillance camera 12 selected on the target settings screen 100. The camera area 210 may display camera identification information 211, a camera image 212, and a checkbox 213. The camera identification information 211 is information for identifying the surveillance camera 12, such as the ID, number, or name of the surveillance camera 12. The camera image 212 is an image taken by the surveillance camera 12 corresponding to the camera area 210. The checkbox 213 can be turned ON / OFF by the user.

[0046] The time zone area 220A displays images of the subject person taken during the first time zone set in the period input area 230A, arranged as thumbnail images 201. The time zone area 220A can display multiple thumbnail images 201.

[0047] The time zone area 220B displays images of the subject taken during the second time zone set in the period input area 230B, arranged as thumbnail images 201. The time zone area 220B can display multiple thumbnail images 201.

[0048] Time zone area 220A and time zone area 220B are displayed side by side on the screen. This allows the user to view thumbnail images 201 taken in the first time zone and thumbnail images 201 taken in the second time zone at the same time.

[0049] On the search results screen 200, sets of camera area 210 and time zone areas 220A and 220B are arranged vertically in multiple rows. Multiple vertically arranged sets of camera area 210 and time zone areas 220A and 220B can be scrolled up and down by the user.

[0050] If the checkbox 213 for camera region 210 is turned OFF, the time regions 220A and 220B associated with that camera region 210 may hide the thumbnail images 201. This allows the user to adjust the number of thumbnail images 201 displayed, making it easier to find the desired thumbnail image 201.

[0051] For each of the period input areas 230A and 230B, you can set either "specified period" or "real-time". Figure 3 illustrates the case where the user sets "specified period" for period input area 230A and "real-time" for period input area 230B.

[0052] The user sets a "specified period" in the period input area 230A corresponding to the time zone area 220A, and sets the specified period to the first time (start time) and the second time (end time) in the past. In this case, the first time zone in the time zone area 220A will be from the "first time" to the "second time". For example, the user sets the time period in which a specific event occurred for the target person as the first time zone. In the example in Figure 3, the first time is set to "August 20, 2024, 19:00", and the second time is set to "August 20, 2024, 19:45".

[0053] The user sets the time period input area 230B, which corresponds to the time zone area 220B, to "Realtime". In this case, the second time zone in time zone area 220B changes from "Second Time" to "Current Time". Figure 3 shows an example where the current time is "August 20, 2024, 8:00 PM". The current time is automatically updated as actual time progresses.

[0054] The analysis result display device 17 displays, in the time zone area 220A for which a "specified time" has been set, images of the target person taken by the surveillance camera 12 indicated by the camera area 210 between the first time and the second time (first time zone) as thumbnail images 201. For example, the analysis result display device 17 displays, in the time zone area 220A corresponding to the camera area 210-1 (hereinafter referred to as time zone area 220A-1), images of the target person taken by the surveillance camera 12 indicated by the camera area 210-1 between the first time and the second time (first time zone) as thumbnail images 201. For example, the analysis result display device 17 displays, in the time zone area 220A corresponding to the camera area 210-2 (hereinafter referred to as time zone area 220A-2), images of the target person taken by the surveillance camera 12 indicated by the camera area 210-2 between the first time and the second time (first time zone) as thumbnail images 201.

[0055] The analysis result display device 17 displays, in the time zone area 220B where "real-time" is set, thumbnail images 201 of the target person captured by the surveillance camera 12 indicated by the camera area 210 between the second time zone and the present time. For example, in the time zone area 220B corresponding to the camera area 210-1 (hereinafter referred to as time zone area 220B-1), the analysis result display device 17 displays, as thumbnail images 201, the target person captured by the surveillance camera 12 indicated by the camera area 210-1 between the second time zone and the present time (second time zone). For example, in the time zone area 220B corresponding to the camera area 210-2 (hereinafter referred to as time zone area 220B-2), the analysis result display device 17 displays, as thumbnail images 201, the target person captured by the surveillance camera 12 indicated by the camera area 210-2 between the second time zone and the present time (second time zone).

[0056] In other words, the analysis result display device 17 displays the target person images captured by each surveillance camera 12 in the time zones 220A and 220B, for each surveillance camera 12.

[0057] In the time zone 220B where "real-time" is set, the analysis result display device 17 updates the current time and displays newly captured images of the target person taken after the update. For example, the analysis result display device 17 displays newly captured images of the target person taken by the surveillance camera 12 indicated in camera zone 210-1 up to the current time after the update in time zone 220B-1, and newly displays newly captured images of the target person taken by the surveillance camera 12 indicated in camera zone 210-2 up to the current time after the update in time zone 220B-2.

[0058] In other words, the second time zone, set to "real-time," is a time zone that is updated as time progresses, while the first time zone, set to "specified period," is a past time zone that is not updated as time progresses.

[0059] This allows users to view the subject in time zone 220A to see images of the subject taken at a specified time, while simultaneously viewing the subject in time zone 220B to see images of the subject taken in real time. Furthermore, users can simultaneously view images of the subject taken by different surveillance cameras 12 during a specified period and images of the subject taken in real time. Additionally, time zones 220A and 220B can display multiple images (thumbnail images 201) of the subject at once. Therefore, users can efficiently check the subject's situation during a specified period in the past and their current situation from the search results screen 200.

[0060] Furthermore, the time zone area 220A with a "specified period" setting and the time zone area 220B with a "real-time" setting may be displayed in different ways. For example, the time zone area 220A with a "specified period" setting and the time zone area 220B with a "real-time" setting may have different frame colors or background colors. This allows users to recognize at a glance whether each time zone area is for a "specified period" or "real-time".

[0061] Note that the time period setting method shown in Figure 3 is just one example, and users can also set a "specified period" in both the period input areas 230A and 230B.

[0062] <Search results screen with 3 columns> Figure 4 shows an example of a search results screen 200 according to Embodiment 1, in which the time zone area 220 is set to three columns.

[0063] As shown in Figure 4, when the time zone area 220 is set to three columns, the search results screen 200 consists of the camera area 210 and the time zone areas 220A, 220B, and 220C. Furthermore, the search results screen 200 consists of a period input area 230A corresponding to time zone area 220A, a period input area 230B corresponding to time zone area 220B, and a period input area 230C corresponding to time zone area 220C. The configuration of the camera area 210 is as described above.

[0064] The time zone area 220A displays images of the subject person taken during the first time zone set in the period input area 230A, arranged as thumbnail images 201. The time zone area 220A can display multiple thumbnail images 201.

[0065] The time zone area 220B displays images of the subject person taken during the second time zone set for the period input area 230B, arranged as thumbnail images 201. The time zone area 220B can display multiple thumbnail images 201.

[0066] The time zone area 220C displays images of the subject person taken during the third time zone set for the period input area 230C, arranged as thumbnail images 201. The time zone area 220C can display multiple thumbnail images 201.

[0067] Time zone areas 220A, 220B, and 220C are displayed side-by-side on the screen. This allows the user to view thumbnail images 201 taken in the first time zone, thumbnail images 201 taken in the second time zone, and thumbnail images 201 taken in the third time zone all at once.

[0068] On the search results screen 200, sets of camera area 210 and time zone areas 220A, 220B, and 220C are arranged vertically in multiple rows. Multiple vertically arranged sets of camera area 210 and time zone areas 220A, 220B, and 220C can be scrolled vertically by the user. Independent scrolling is also possible for each time zone area 220A, 220B, and 220C. Independent scrolling is also possible for each camera area 210.

[0069] If the checkbox 213 for camera region 210 is turned OFF, the time regions 220A, 220B, and 220C associated with that camera region 210 may hide the thumbnail images 201. This allows the user to adjust the number of thumbnail images 201 displayed, making it easier to find the desired thumbnail image 201.

[0070] For each of the time zones 220A, 220B, and 220C, you can set either "Specified Period" or "Real-time". Figure 4 illustrates the case where the user sets "Specified Period" for time zone 220A and "Real-time" for time zones 220B and 220C.

[0071] The user sets a "specified period" in the period input area 230A corresponding to the time zone area 220A, and sets the past first time (start time) and second time (end time) as the specified period. In this case, the first time zone in the time zone area 220A will be from the "first time" to the "second time". In the example in Figure 4, the first time is set to "August 20, 2024, 19:30", and the second time is set to "August 20, 2024, 19:35".

[0072] The user sets the period input area 230B, which corresponds to the time zone area 220B, to "Real-time" and sets the time zone from "N minutes ago" to "Current time". In this case, the second time zone of time zone area 220B will be from "the time N minutes ago from the current time" to "the current time". Figure 4 shows an example where "5 minutes ago" is set as N minutes ago and the current time is "August 20, 2024, 19:45". The user can set N minutes ago arbitrarily. The current time is automatically updated as time passes, and accordingly, "the time N minutes ago from the current time" is also automatically updated.

[0073] The user sets the period input area 230C corresponding to the time zone area 220C to "Real-time". In this case, the analysis result display device 17 may automatically set the third time zone of time zone area 220C from "Second time" to "N minutes before the current time". In this case, the third time zone of time zone area 220C will be from "Second time" to "N minutes before the current time". In the example in Figure 4, "August 20, 2024, 19:35" is set as the second time, and "5 minutes before" is set as N minutes before. In this case, as the current time is updated, "the time N minutes before the current time" is also automatically updated. Therefore, as the current time is updated, the third time zone of time zone area 220B becomes longer.

[0074] In other words, the analysis result display device 17 displays the time zones 220A, 220C, and 220B in order on the search results screen 200. Here, the third time zone corresponding to time zone 220C is a time zone that is later than the first time zone corresponding to time zone 220A and earlier than the second time zone corresponding to time zone 220B. In addition, there may be multiple time zones 220C, in which case there will also be multiple third time zones.

[0075] The analysis result display device 17 displays, in the time zone area 220A for which a "specified period" is set, thumbnail images 201 of the target person images taken by the surveillance camera 12 indicated by the camera area 210 between the first time and the second time (first time zone). For example, in the time zone area 220A-1 associated with camera area 210-1 in the time zone area 220A, the analysis result display device 17 displays, as thumbnail images 201, multiple target person images taken by the surveillance camera 12 indicated by camera area 210-1 between the first time and the second time (first time zone). For example, in the time zone area 220A-2 associated with camera area 210-2 in the time zone area 220A, the analysis result display device 17 displays, as thumbnail images 201, the target person images taken by the surveillance camera 12 indicated by camera area 210-2 between the first time and the second time (first time zone).

[0076] The analysis result display device 17 displays, in the time zone 220C where "real-time" is set, thumbnail images 201 of the target person captured by the surveillance camera 12 indicated by the camera zone 210 between the "second time" and "N minutes before the current time" (third time zone). For example, in the time zone 220C-1 associated with camera zone 210-1 in the time zone 220C, the analysis result display device 17 displays, as thumbnail images 201, the target person captured by the surveillance camera 12 indicated by camera zone 210-1 between the "second time" and "N minutes before the current time" (third time zone). For example, in the time zone 220C-2 associated with camera zone 210-2 in the time zone 220C, the analysis result display device 17 displays, as thumbnail images 201, the target person captured by the surveillance camera 12 indicated by camera zone 210-2 between the "second time" and "N minutes before the current time" (third time zone).

[0077] The analysis result display device 17 displays, in the time zone area 220B where "real-time" is set, thumbnail images 201 of the target person captured by the surveillance camera 12 indicated by the camera area 210 between "N minutes before the current time" and "the current time" (second time zone). For example, in the time zone area 220B-1 associated with camera area 210-1 in the time zone area 220B, the analysis result display device 17 displays, in the time zone area 220B-2 associated with camera area 210-2 in the time zone area 220B, thumbnail images 201 of the target person captured by the surveillance camera 12 indicated by camera area 210-2 between "N minutes before the current time" and "the current time" (second time zone). For example, in the time zone area 220B-2 associated with camera area 210-2 in the time zone area 220B, the analysis result display device 17 displays, in the time zone area 220B-2, thumbnail images 201 of the target person captured by the surveillance camera 12 indicated by camera area 210-2 between "N minutes before the current time" and "the current time" (second time zone).

[0078] In the time zones 220B and 220C where "real-time" is set, the analysis result display device 17 updates the current time and displays newly captured images of the target person taken after the update. For example, the analysis result display device 17 displays newly captured images of the target person taken by the surveillance camera 12 indicated by camera zone 210-1 up to the current time after the update in time zone 220B-1, and newly displays newly captured images of the target person taken by the surveillance camera 12 indicated by camera zone 210-2 up to the current time after the update in time zone 220B-2.

[0079] Here, along with the update of the current time, the "N minutes prior to the current time" is also updated. Therefore, the analysis result display device 17 moves the subject person images that were displayed in time zone area 220B-1 and have fallen outside the time zone from "N minutes prior to the updated current time" to "the updated current time" (the second time zone) to time zone area 220C-1 (i.e., the third time zone from "the second time zone" to "N minutes prior to the updated current time"). Similarly, the analysis result display device 17 moves the subject person images that were displayed in time zone area 220B-2 and have fallen outside the time zone from "N minutes prior to the updated current time" to "the updated current time" (the second time zone) to time zone area 220C-2 (i.e., the third time zone from "the second time zone" to "N minutes prior to the updated current time").

[0080] In other words, the second and third time periods set to "real-time" are time periods that are updated as the current time progresses, while the first time period set to "specified period" is a past time period that is not updated as the current time progresses.

[0081] This allows the user to view the subject in time zone 220A to see the subject photographed during the "specified period," view the subject in time zone 220B to see the subject photographed most recently, and view the subject in time zone 220C to see the subject photographed a little earlier than the most recent. Furthermore, as in Figure 3, the user can simultaneously view the subject photographed during the "specified period" and the subject photographed in "real time" by different surveillance cameras 12. Also, as in Figure 3, time zones 220A, 220B, and 220C can display multiple subject images at once. Therefore, the user can efficiently check the subject's situation during the past specified period, their most recent situation, and their situation a little earlier than the most recent situation from the search results screen 200.

[0082] Note that the time setting method shown in Figure 3 is just one example. Users can also set "Specified Period" for period input areas 230A and 230C, and "Real-time" for period input area 230B. Alternatively, users can set "Specified Period" for all of period input areas 230A, 230B, and 230C.

[0083] Furthermore, although the above describes examples where the time zone domain 220 has two or three columns, the number of columns in the time zone domain 220 can be any number.

[0084] (Embodiment 2) In Embodiment 2, components already described in Embodiment 1 are given common reference numerals, and their descriptions may be omitted.

[0085] Figure 5 is a block diagram showing an example configuration of the video management system 10 according to Embodiment 2. The video management system 10 shown in Figure 5 further includes a video data acquisition device 18 compared to Figure 1.

[0086] The video data acquisition device 18 acquires video data provided by other systems. The video data may be any type of video data, such as video data captured by surveillance cameras of other systems, or video data captured by smartphones or digital cameras. The video data acquisition device 18 may acquire video data through the communication network 11. Alternatively, the video data acquisition device 18 may acquire video data by reading video data stored in a storage device (e.g., a USB memory stick or DVD-RAM). The video data acquired by the video data acquisition device 18 may be interpreted as externally captured images.

[0087] The video data acquisition device 18 stores the acquired video data in the video storage device 15. As a result, the video storage device 15 stores not only video data captured by the surveillance camera 12, but also video data acquired from other systems.

[0088] The video distribution device 13 may acquire video data from the video storage device 15 and transfer it to the image analysis device 14. The image analysis device 14 may analyze the video data received from the video distribution device 13 in the same manner as the video data described in Embodiment 1 and generate detection information. The image analysis device 14 may then store the generated detection information in association with the video data in the video storage device 15.

[0089] The analysis result display device 17 may treat the video data acquired by the video data acquisition device 18 in the same way as the video data captured by the surveillance camera 12 in the target setting screen 100 shown in Figure 2. Furthermore, the target setting screen 100 shown in Figure 2 provides a function that allows the selection of cameras that captured video data acquired from other systems, in addition to the surveillance camera 12 of the system itself, as cameras to be displayed in the camera areas 310A and 310B of the search results screen 300 shown in Figure 6.

[0090] Figure 6 shows an example of the search results screen 300 according to Embodiment 2.

[0091] The analysis result display device 17 displays multiple images of the target person that was set as the target for search or tracking as thumbnail images 301 on the search results screen 300.

[0092] As shown in Figure 6, the search results screen 300 is composed of a camera area 310 and time zone areas 320A and 320B. Furthermore, the search results screen 300 is composed of a period input area 330A corresponding to time zone area 320A and a period input area 330B corresponding to time zone area 320B.

[0093] Camera areas 310A and 310B display information about the surveillance camera 12 or the camera of another system that captured video data, which was selected on the target setting screen 100. As shown in Figure 6, multiple camera areas 310A and 310B may be arranged vertically. Camera areas 310A and 310B may display camera identification information 311, a camera image 312, and a checkbox 313. These are the same as the camera identification information 211, camera image 212, and checkbox 213 described above.

[0094] The time zone area 320A displays images of the subject person taken during the fourth time zone set in the period input area 330A, arranged as thumbnail images 201. The time zone area 320A can display multiple thumbnail images 201.

[0095] The time zone area 320B displays images of the subject person taken during the fifth time zone set for the period input area 330B, arranged as thumbnail images 201. The time zone area 320B can display multiple thumbnail images 201.

[0096] The user sets the "specified period" in the period input area 330A, and sets the specified period to the first and second past time points. In this case, the fourth time point in the time zone area 320A will be the "first time point" and the "second time point". In the example in Figure 6, the first time point is set to "August 20, 2024, 19:00", and the second time point is set to "August 20, 2024, 19:45".

[0097] The user can configure a camera in the camera area 310A to correspond to the time zone area 320A. This camera may be the surveillance camera 12, or it may be a camera from another system that has captured video data.

[0098] The user sets the "specified period" in the period input area 330B, and sets the specified period to the third and fourth time points in the past. In this case, the fifth time point in the time zone area 320B will be the "third time point" to the "fourth time point". Note that the second and third time points may be set to the same time or to different times. In the example in Figure 6, the third time point is set to "August 20, 2024, 19:45", and the fourth time point is set to "August 20, 2024, 20:00".

[0099] The user can configure a camera in the camera area 310B to correspond to the time zone area 320B. This camera may be the surveillance camera 12, or it may be a camera from another system that captures video data.

[0100] The analysis result display device 17 displays, as thumbnail images 201, images of the target person taken by the camera indicated in the camera area 310A-1 (for example, the surveillance camera 12) between the first time period and the second time period (the fourth time period) in the time period area 320A that corresponds to the camera area 310A-1 (hereinafter referred to as time period area 320A-1).

[0101] The analysis result display device 17 displays, as thumbnail images 201, images of the target person taken by the camera indicated in the camera area 310A-2 (for example, the surveillance camera 12) between the first time period and the second time period (the fourth time period) in the time period area 320A that corresponds to the camera area 310A-2 (hereinafter referred to as the time period area 320A-2).

[0102] The analysis result display device 17 displays, as thumbnail images 201, images of the target person taken by the camera indicated in the camera area 310B-1 (for example, the surveillance camera 12) between the third and fourth time periods (the fifth time period) in the time period area 320B that corresponds to the camera area 310B-1 (hereinafter referred to as time period area 320B-1).

[0103] The analysis result display device 17 displays, as thumbnail images 201, images of the target person taken by the camera indicated in the camera area 310B-2 (for example, the camera that captured video data) between the third and fourth time periods (the fifth time period) in the time period area 320B that corresponds to the camera area 310B-2 (hereinafter referred to as the time period area 320B-2).

[0104] In the search results screen 300, sets of camera area 310A and time zone area 320A are arranged vertically in multiple rows, and sets of camera area 310B and time zone area 320B are also arranged vertically in multiple rows. In addition, in the search results screen 300, sets of camera area 310A and time zone area 320A, and sets of camera area 310B and time zone area 320B may be arranged horizontally. Multiple sets of camera area 310A and time zone area 320A arranged vertically can be scrolled up and down by the user. Similarly, multiple sets of camera area 310B and time zone area 320B arranged vertically can be scrolled up and down by the user. Multiple sets of camera area 310A and time zone area 320A arranged vertically, and multiple sets of camera area 310B and time zone area 320B arranged vertically may be scrolled up and down independently.

[0105] For example, if the checkbox 313 for camera region 310A is turned OFF, the time region 320A-1 associated with camera region 310A may hide the thumbnail image 301. This allows the user to adjust the number of thumbnail images 301 displayed, making it easier to find the desired thumbnail image 301.

[0106] Thus, according to the video management system 10 of this embodiment, images of target individuals, searched from both video data captured by the system's surveillance camera 12 and video data acquired from other systems, can be displayed together on the search results screen 300.

[0107] This allows users to view not only video data captured by their own system, but also video data captured by other systems, enabling them to efficiently search for or track individuals.

[0108] (Hardware configuration) Figure 7 is a block diagram showing an example of the hardware configuration of the information processing device 1000 according to this disclosure.

[0109] The video distribution device 13, image analysis device 14, video storage device 15, image matching device 16, and analysis result display device 17 described above may be configured as the information processing device 1000 shown in Figure 7.

[0110] The information processing device 1000 comprises a processor 1001, a memory 1002, a storage 1003, an input device 1004, a display device 1005, a communication device 1006, and a reading device 1007.

[0111] The processor 1001 is a device that executes computer programs stored in memory 1002 (i.e., works in cooperation with memory 1002) to realize the functions described above. Therefore, the processing described above, mainly involving the video distribution device 13, image analysis device 14, video storage device 15, image matching device 16, or analysis result display device 17, can be reinterpreted as processing mainly involving the processor 1001. The processor 1001 may also be reinterpreted as a Central Processing Unit (CPU), controller, or control unit, etc. Furthermore, the processor 1001 may include a Graphics Processing Unit (GPU) and / or a Neural Processing Unit (NPU).

[0112] Memory 1002 is composed of a volatile storage medium and / or a non-volatile storage medium and is a device for storing computer programs and data handled by the information processing device 1000. Memory 1002 may also include Random Access Memory (RAM) and / or Read-Only Memory (ROM).

[0113] The storage device 1003 is composed of a non-volatile storage medium and stores computer programs and data handled by the information processing device 1000. Examples of the storage device 1003 include a Hard Disk Drive (HDD), a Solid State Drive (SSD), or flash memory.

[0114] The input device 1004 is a device that receives data to be input to the processor 1001. Examples of input devices 1004 include a keyboard, mouse, touchpad, touch panel, and microphone.

[0115] The display device 1005 is a device that outputs data generated by the processor 1001. Examples of the display device 1005 include liquid crystal displays or organic EL displays. The target setting screen 100 shown in Figure 2, the search result screens 200 shown in Figures 3 and 4, and the search result screen 300 shown in Figure 6 may be displayed on the display device 1005 provided in the analysis result display device 17.

[0116] The communication device 1006 is a device that sends and receives data with other devices via the communication network 11. Examples of the communication network 11 include wired LANs, wireless LANs, the Internet, and mobile communication networks.

[0117] The reading device 1007 is a device that reads data from recording media such as Universal Serial Bus (USB) memory or Digital Versatile Disk Read Only Memory (DVD-ROM).

[0118] (Summary of this disclosure) The following technologies are disclosed in accordance with the above description in this disclosure.

[0119] <Technology 1> An image processing device (e.g., analysis result display device 17) according to one embodiment comprises a processor (1001) and a memory (1002). The processor cooperates with the memory to acquire multiple images that meet predetermined conditions from multiple images captured by at least one imaging device (e.g., surveillance camera 12) as multiple target images (e.g., target person images). Of the acquired multiple target images, the target images captured in a first time period are displayed in a first area (e.g., time period area 220A) of the screen (e.g., search result screen 200), and the target images captured in a second time period are displayed in a second area (e.g., time period area 220B) of the screen. The second time period is a time period that is updated as the current time progresses, and the first time period is a past time period that is not updated as the current time progresses. This allows users to easily and efficiently check the situation in the past (first time period) and the current situation in the (second time period).

[0120] <Technology 2> In the video processing apparatus described in Technical 1, the processor updates the display of the target image in the second region in accordance with the update of the second time period. This allows users to efficiently check changes in the target's current situation (e.g., the target's actions and movements).

[0121] <Technology 3> In the image processing apparatus described in Technical 1, the processor displays, among the acquired plurality of target images, the target image captured in the third time period in the third area of ​​the screen (for example, time period area 220C) together with the first area and the second area, wherein the third time period is later than the first time period and earlier than the second time period. This allows users to easily and efficiently check the situation in the first time period in the past, the situation in the second time period currently, and the situation in a third time period slightly earlier than the present.

[0122] <Technology 4> In the image processing apparatus described in Technical 3, the processor updates the third time period in response to the update of the second time period, and updates the display of the target image in the third region in response to the update of the third time period. This allows users to easily and efficiently check the changes (e.g., the subject's actions and movements) between the subject's current situation and a situation slightly prior to the present.

[0123] <Technology 5> In the video processing apparatus described in Technology 3 or 4, the processor displays the first area, the third area, and the second area on the screen in order. This allows users to easily compare the situation in the first time period in the past, the situation in the second time period currently, and the situation in a third time period slightly earlier than the present.

[0124] <Technology 6> In the image processing apparatus described in Technology 1 or 2, the processor displays the target images captured by each imaging device in the first and second regions, respectively. This allows users to easily identify the camera that captured the target image in the first time period and the camera that captured the target image in the second time period. Therefore, users can easily determine, from the location where each camera is installed, the location where the person or vehicle captured in the target image was in the past (first time period) and the current location in the second time period.

[0125] <Technology 7> In the video processing device described in any one of the technologies 1 to 6, the predetermined conditions are detection conditions relating to a person or vehicle to be searched or tracked. This allows users to search for target images in which the person or vehicle they wish to search for or track has been photographed.

[0126] <Technology 8> In the image processing device described in any one of technologies 1 to 7, the processor acquires a plurality of externally captured images (e.g., video data) that meet the predetermined conditions from a plurality of externally captured images (e.g., video data) captured or stored by another device, as the plurality of target images. This allows the image processing device to handle not only images captured by the imaging device but also externally captured images taken by other devices as a single target image.

[0127] <Technology 9> An image processing system (for example, an image management system 10) according to one embodiment includes at least one shooting device, and a display control device that acquires a plurality of images that meet predetermined conditions as a plurality of target images from a plurality of images captured by the at least one shooting device, displays the target images captured in a first time period in a first area of ​​the screen, and displays the target images captured in a second time period in a second area of ​​the screen, wherein the second time period is a time period that is updated as the current time progresses, and the first time period is a past time period that is not updated as the current time progresses. This allows users to easily and efficiently check the situation in the past (first time period) and the current situation in the (second time period).

[0128] <Technology 10> An image processing method according to one embodiment involves acquiring multiple images that meet predetermined conditions from a plurality of images captured by at least one shooting device as a plurality of target images, displaying the target images captured in a first time period in a first area of ​​the screen, and displaying the target images captured in a second time period in a second area of ​​the screen, wherein the second time period is a time period that is updated as the current time progresses, and the first time period is a past time period that is not updated as the current time progresses. This allows users to easily and efficiently check the situation in the past (first time period) and the current situation in the (second time period).

[0129] While embodiments have been described above with reference to the attached drawings, this disclosure is not limited to such examples. It is clear to those skilled in the art that various modifications, alterations, substitutions, additions, deletions, and equivalents can be conceived within the scope of the claims, and these are also understood to fall within the technical scope of this disclosure. Furthermore, the components of the embodiments described above can be combined in any way without departing from the spirit of the invention. [Industrial applicability]

[0130] The technology disclosed herein is useful for searching or reviewing video footage captured by surveillance cameras, etc. [Explanation of symbols]

[0131] 10 Video Management Systems 11. Communication Networks 12 surveillance cameras 13 Video distribution device 14 Image analysis device 15. Video storage device 16 Image matching device 17 Analysis result display device 18. Video data acquisition device 100 Target Settings Screen 101,201,301 Thumbnail images Camera areas 110, 210, 310, 310A, 310B 111,211,311 Camera identification information 112,212,312 camera images 113,213,313 checkboxes 120 Captured image area 130 Selected area for search or tracking 140, 230A, 230B, 230C, 330A, 330B Period input area Time zones: 220A, 220A-1, 220A-2, 220B, 220B-1, 220B-2, 220C, 220C-1, 220C-2, 320A, 320A-1, 320A-2, 320B, 320B-1, 320B-2 1000 Information Processing Devices 1001 Processor 1002 memory 1003 Storage 1004 Input device 1005 Display device 1006 Communication device 1007 Reader

Claims

1. A video processing device comprising a processor and memory, The aforementioned processor cooperates with the memory, From multiple images captured by at least one imaging device, multiple images that meet predetermined conditions are acquired as multiple target images. Of the acquired multiple target images, the target images taken during the first time period are displayed in the first area of ​​the screen, and the target images taken during the second time period are displayed in the second area of ​​the screen. The second time period is a time period that is updated as time progresses. The first time period is a past time period that is not updated as the current time progresses. Image processing device.

2. The processor updates the display of the target image in the second region in accordance with the update of the second time period. The image processing apparatus according to claim 1.

3. The processor displays, among the acquired plurality of target images, the target image captured during the third time period in the third area of ​​the screen, together with the first and second areas. The third time period is a time period that is later than the first time period and earlier than the second time period. The image processing apparatus according to claim 1.

4. The aforementioned processor, In response to the update of the second time period, the third time period is updated. In accordance with the update of the third time period, the display of the target image in the third region is updated. The image processing apparatus according to claim 3.

5. The processor displays the first area, the third area, and the second area in order on the screen. The image processing apparatus according to claim 3 or 4.

6. The processor displays the target images captured by each imaging device in the first and second regions, for each imaging device. The image processing apparatus according to claim 1.

7. The aforementioned predetermined conditions are detection conditions relating to the person or vehicle to be searched or tracked. The image processing apparatus according to claim 1.

8. The processor acquires a plurality of externally captured images that meet the predetermined conditions from a plurality of externally captured images captured or stored by another device, as the plurality of target images. The image processing apparatus according to claim 1.

9. It is an image processing system, At least one imaging device, The system includes a display control device that acquires multiple images that meet predetermined conditions from a plurality of images captured by the at least one imaging device, and displays the target images captured in a first time period in a first area of ​​the screen, and the target images captured in a second time period in a second area of ​​the screen, The second time period is a time period that is updated as time progresses. The first time period is a past time period that is not updated as the current time progresses. Video processing system.

10. A video processing method, From multiple images captured by at least one imaging device, multiple images that meet predetermined conditions are acquired as multiple target images. Of the acquired multiple target images, the target images taken during the first time period are displayed in the first area of ​​the screen, and the target images taken during the second time period are displayed in the second area of ​​the screen. The second time period is a time period that is updated as time progresses. The first time period is a past time period that is not updated as the current time progresses. Image processing methods.

Images