Video creation system and video creation method

Abstract

The task involves tracking unpredictable objects moving between multiple camera units. [Solution] The surveillance and filming system 1 comprises a filming device 201... and a monitoring device 10. The filming device 201... comprises a plurality of filming units, an analysis unit, and an identification unit. The monitoring device 10 comprises a control unit. The plurality of filming units film a filming range and generate video data. The analysis unit analyzes the filmed video data and extracts feature quantities of the object to be tracked. The identification unit identifies the object from the plurality of filmed video data based on the feature quantities. The control unit arranges the plurality of video data in which the object has been identified in chronological order and makes them playable.

Description

Technical Field

[0001] The present invention relates to a video creation system and a video creation method.

Background Art

[0002] Conventionally, there is an image analysis technique for tracking a specific object or person across multiple cameras. For example, for identifying a person, feature amounts such as face authentication, clothing, ornaments, and gait are used, and when analyzing the video of each camera, the same person can be identified using those feature amounts. For example, in tracking an object, the manufactured product on a factory line is determined by the speed of the production line. Therefore, when it is desired to track the video of a specific lot number, tracking can be performed across multiple camera videos by specifying the date and time of passing on the production line.

[0003] In addition, a surveillance camera system that synthesizes a combined video data of one moving image from a plurality of recorded individual video data is known (see Patent Document 1). The plurality of individual video data are cut, frame-adjusted, and combined in the display order to be combined video data.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The above surveillance camera system can create combined video data in which objects such as a person moving unpredictably across multiple cameras or a fixed viewing angle are combined for each time zone according to a rule such as moving at a certain speed. However, in the combined video data, an object such as a person moving unpredictably across multiple cameras cannot be tracked.

[0006] The objective of this invention is to track unpredictable objects, such as people, that move between multiple imaging units. [Means for solving the problem]

[0007] To solve the above problem, the video creation system of the invention described in claim 1 is: Multiple shooting units that capture the shooting range and generate video data, An analysis unit analyzes the captured video data and extracts characteristic features of the object to be tracked, Based on the aforementioned feature quantities, an identification unit identifies the object from the multiple captured video data, The system includes a control unit that enables playback of multiple video data in which the aforementioned object has been identified, arranged in chronological order.

[0008] The invention described in claim 2 is, in the video creation system described in claim 1, The control unit creates video data by arranging multiple video data in which the target object is identified in chronological order.

[0009] The invention described in claim 3 relates to the video creation system described in claim 1 or 2, The analysis unit analyzes the captured video data and extracts the characteristic quantities of the objects shown.

[0010] The invention described in claim 4 relates to the video creation system described in claim 1 or 2, The analysis unit analyzes the captured video data and extracts the feature quantities of the specified input object.

[0011] The invention described in claim 5 relates to the video creation system described in claim 1 or 2, The identification unit adds additional information to the video data in which the object has been identified.

[0012] The invention described in claim 6 is, in the video creation system described in claim 4, The additional information includes at least one of the following: a display element indicating the position of the object, and the time of photography.

[0013] The invention described in claim 7 relates to the video creation system described in claim 1 or 2, The control unit enables playback of multiple frames in parallel when there are multiple frames taken at the same time in multiple video data in which the object has been identified.

[0014] The invention described in claim 8 is an image creation system according to claim 1 or 2, The control unit, when there are multiple video data in which the object has been identified that contain multiple frames taken at the same time, selects the frame with the largest display area of ​​the object and plays it back.

[0015] The invention described in claim 9 relates to the video creation system described in claim 1 or 2, The control unit generates and supplements video data for the time period in which the object was identified if there is a time period in the multiple video data in which the object was identified that was not captured.

[0016] The invention described in claim 10 relates to the video creation system described in claim 1 or 2, The control unit plays back the multiple video data arranged in chronological order, along with map information and trajectory information of the area where the object included in the multiple video data moved.

[0017] The video creation method of the invention described in claim 11 is: The process involves analyzing video data captured by multiple shooting units that capture the shooting range and generate video data, and extracting the characteristic features of the object to be tracked. A process of identifying the object from multiple video data captured by the multiple imaging units based on the aforementioned feature quantities, The process includes a control step that arranges the plurality of video data in which the object has been identified in chronological order and makes them playable. [Effects of the Invention]

[0018] According to the present invention, it is possible to track an unpredictable object moving between a plurality of imaging units.

Brief Description of the Drawings

[0019] [Figure 1] It is a block diagram showing a surveillance imaging system according to a first embodiment of the present invention. [Figure 2] It is a block diagram showing the functional configuration of a surveillance device. [Figure 3] It is a block diagram showing the functional configuration of an imaging device. [Figure 4] It is a flowchart showing a first video creation process. [Figure 5] It is a flowchart showing a feature amount sharing process. [Figure 6] It is a flowchart showing a video sharing process. [Figure 7] It is a diagram showing combined video data. [Figure 8] It is a flowchart showing a second video creation process. [Figure 9] It is a plan view showing a camera unit provided in a commercial facility and the path of an object.

Modes for Carrying Out the Invention

[0020] The advantages and features provided by one or more embodiments of the present invention will be more fully understood from the following detailed description and the accompanying drawings. However, these drawings are for illustrative purposes only. Therefore, they are not intended to define the limits of the present invention. Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the scope of the present invention is not limited to the disclosed embodiments.

[0021] (First Embodiment) A first embodiment of the present invention will be described with reference to Figures 1 to 7. First, the device configuration of the surveillance and photography system 1, which is the video creation system of this embodiment, will be described with reference to Figures 1 to 3. Figure 1 is a block diagram of the surveillance and photography system 1 of this embodiment. Figure 2 is a block diagram of the functional configuration of the surveillance device 10. Figure 3 is a block diagram of the functional configuration of the photography device 201.

[0022] As shown in Figure 1, the surveillance camera system 1 is a system that captures the space to be monitored and creates combined video data in which frames showing the object to be tracked are arranged in chronological order. The purpose of surveillance is, for example, crime prevention, but is not limited to this. The space to be monitored is, for example, an indoor commercial facility, but is not limited to this. The space to be monitored may be other spaces such as indoor and outdoor commercial facilities, public facilities, educational facilities, medical facilities, or outdoor street corners. The object to be tracked is, for example, a person who is a suspect, but is not limited to this. The object may be other objects such as a person such as a VIP, an animal, or a car.

[0023] The surveillance and imaging system 1 comprises a surveillance device 10 and imaging devices 201, 202, etc. The number of imaging devices shall be at least one. The surveillance device 10 and imaging devices 201, etc. are each connected to each other via a communication network 30 so as to be able to communicate with one another. The communication network 30 is, for example, a LAN (Local Area Network), but is not limited to this. The communication network 30 may also be a wired communication network, wireless communication network, or other communication network such as a WLAN (Wireless LAN).

[0024] The monitoring device 10 is installed, for example, in a monitoring room in a commercial facility. The monitoring device 10 is operated by a monitor and displays real-time video data captured by the imaging devices 201... and combined video data showing the target object. Each imaging device 201... has at least one camera unit 25 (Figure 3), and generates video data by capturing the space to be monitored with the camera unit 25. In addition, each imaging device 201... extracts the target object from the video data using the feature quantities of the target object. Furthermore, the shooting range of each camera unit 25 of the imaging device 201... includes at least mutually exclusive ranges.

[0025] Next, with reference to Figure 3, the internal functional configuration of the monitoring device 10 will be explained. As shown in Figure 3, the monitoring device 10 has a control unit 11, an operation unit 12, a storage unit 13, a display unit 14, and a communication unit 15. The control unit 11 functions as a creation unit and a display control unit. Each part of the monitoring device 10 is connected to each other via a bus.

[0026] The control unit 11 controls each part of the monitoring device 10. The control unit 11 has a CPU (Central Processing Unit) and RAM (Random Access Memory). The control unit 11 reads various programs stored in the memory unit 13, loads them into RAM, and executes various processes in cooperation with the CPU and the loaded programs.

[0027] The operation unit 12 includes a keyboard and a pointing device such as a mouse. The operation unit 12 receives key input and position input from the user acting as a monitor and outputs the operation information to the control unit 11.

[0028] The storage unit 13 is an HDD (Hard Disk Drive), SSD (Solid State Drive), etc. The storage unit 13 stores information such as video data in a read-and-write manner. In particular, the storage unit 13 stores the video sharing program. The video sharing program is a program for executing the video sharing process described later.

[0029] The display unit 14 has at least one display panel such as an LCD (liquid crystal display) or an ELD (electro-luminescent display). The display unit 14 displays display information on the display panel in accordance with the control of the control unit 11. For example, multiple real-time video data captured by each camera unit 25 in the imaging device 201... are displayed on one display panel by dividing or switching between them. However, the configuration is not limited to this. For example, the display unit 14 may have multiple display panels, and multiple real-time video data captured by each camera unit 25 may be displayed separately on multiple display panels.

[0030] The communication unit 15 is a communication module such as a network card, and performs wired or wireless communication with external devices such as imaging devices 201, etc. on the communication network 30. The control unit 11 transmits and receives information with the imaging devices 201, etc. via the communication unit 15.

[0031] Next, with reference to Figure 3, the internal functional configuration of the imaging device 201 will be explained. Since the imaging devices 202, etc., have the same configuration as imaging device 201, imaging device 201 will be described as representative. As shown in Figure 3, imaging device 201 has a control unit 21, an operation unit 22, a storage unit 23, a communication unit 24, a camera unit 25 as an imaging unit, and a timing unit 26. The control unit 21 functions as an analysis unit and a identification unit. Each part of imaging device 201 is connected to each other via a bus.

[0032] The control unit 21 controls various parts of the imaging device 201. The control unit 21 has a CPU and RAM. The control unit 21 reads various programs stored in the memory unit 23, loads them into RAM, and executes various processes in cooperation with the CPU and the loaded programs.

[0033] The operation unit 22 has, for example, various switches for various settings. The operation unit 22 receives input from the user and outputs the operation information to the control unit 21.

[0034] The memory unit 23 is a flash memory or the like. The memory unit 23 stores information such as video data in a read-and-write manner. In particular, the memory unit 23 stores the first video creation program and the feature data sharing program. The first video creation program is a program for executing the first video creation process, which will be described later. The feature data sharing program is a program for executing the feature data sharing process, which will be described later.

[0035] The communication unit 24 is a communication module such as a network card, and performs wired or wireless communication with external devices such as the monitoring device 10, the imaging device 202, etc. on the communication network 30. The control unit 21 transmits and receives information with the monitoring device 10, the imaging device 202, etc. via the communication unit 24.

[0036] The camera unit 25 is a digital camera that has an optical system such as a lens and an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). The camera unit 25 captures the space to be monitored, generates video data as real time video, and outputs it to the control unit 21.

[0037] The timing unit 26 is a real-time clock that measures the current date and time and outputs it to the control unit 21. Each frame of the video data is associated with the shooting date and time based on the current date and time of the timing unit 26.

[0038] Next, the operation of the surveillance and filming system 1 will be explained with reference to Figures 4 to 7. Figure 4 is a flowchart of the first video creation process. Figure 5 is a flowchart of the feature sharing process. Figure 6 is a flowchart of the video sharing process. Figure 7 is a diagram showing the frames of the combined video data 40.

[0039] First, with reference to Figure 4, the first video creation process performed by the shooting device 201... will be explained. The first video creation process is a process in which the feature quantities of an object detected from the captured video data are shared with other shooting devices, and video data with additional information about the object is added. In advance, the control unit 21 of the shooting device 201 captures the space to be monitored with the camera unit 25 and generates real-time video data of the space to be monitored. At this time, the control unit 21 associates the shooting date and time with each frame of the real-time video data based on the current date and time from the timing unit 26. The control unit 21 transmits the real-time video data to the monitoring device 10 via the communication unit 24. The control unit 11 of the monitoring device 10 receives the real-time video data from the shooting device 201 via the communication unit 15 and displays it on the display unit 14.

[0040] The memory unit 23 of the camera 201... stores real-time video data captured by the camera for a predetermined period of time. The memory unit 23 of the camera 201... also stores characteristic quantities of pre-set anomalies (such as violence by the subject, high speed of movement, or entry into a dangerous area). These anomaly characteristic quantities are information that indicates various image features used to detect the occurrence of the anomaly during image analysis of the video data.

[0041] Similar to the imaging device 201, the imaging devices 202… also generate real-time video data captured by the camera unit 25 and display it on the monitoring device 10. Here, the operation of imaging device 201 will be described as representative, but the operation of the other imaging devices 202… is similar. In imaging device 201, for example, triggered by the power being turned on, the control unit 21 executes the first video creation process according to the first video creation program in the storage unit 23.

[0042] As shown in Figure 4, the control unit 21 reads the anomaly feature quantity from the storage unit 23 (step S11). In step S11, the control unit 21 uses the read anomaly feature quantity to perform image analysis on real-time video data captured by each camera unit 25 of the aircraft. The control unit 21 determines whether or not an anomaly was detected from at least one frame of video data in the image analysis in step S12 (step S12). If no anomaly was detected (step S12; NO), the process returns to step S11.

[0043] If an anomaly is detected (step S12; YES), the control unit 21 performs image analysis on the video data of the anomaly scene and extracts feature quantities of the object captured in the scene (step S13). The feature quantities of the object are numerical information used to identify, for example, a suspicious person, such as their face, clothing, belongings, skeleton, gait, etc. The control unit 21 transmits the feature quantities acquired in step S13 to all other imaging devices 202... via the communication unit 24 (step S14).

[0044] The control unit 21 performs image analysis on the video data stored in the memory unit 23 based on the feature quantities acquired in step S14 (step S15). In step S15, the control unit 21 extracts video data from the stored video data that contains frames showing the target object. In step S15, for example, video data from the start of the abnormality in step S13 is performed image analysis.

[0045] The control unit 21 adds additional information to the video data extracted in step S15 (step S16). The additional information is added to each frame image of the video data, and may include, for example, the date and time of shooting, a frame surrounding the object, a marker indicating the position of the object, or a mark. The control unit 21 transmits the video data to which the additional information was added in step S16 to the monitoring device 10 (step S17). The transmitted video data is associated with feature quantities. The first video creation process is completed.

[0046] Next, referring to Figure 5, the feature sharing process performed by the imaging devices 201... is explained. The feature sharing process is performed by the imaging device that receives the feature quantities of the target object in the first video creation process, and extracts the target object that appears in the video data captured by the device based on those feature quantities. Here, we will explain an example in which imaging device 202, which receives the feature quantities from imaging device 201, performs the feature sharing process, but the same applies to other imaging devices.

[0047] In the imaging device 202, the feature quantities transmitted from the imaging device 201 in step S14 of the first video creation process in Figure 4 are received via the communication unit 24. The control unit 21 uses this reception start as a trigger to execute the feature quantity sharing process according to the feature quantity sharing program stored in the storage unit 23.

[0048] As shown in Figure 5, the control unit 21 of the imaging device 202 receives the feature quantities transmitted from the imaging device 201 in step S14 of Figure 4 via the communication unit 24 (step S21). Based on the feature quantities received in step S21, the control unit 21 performs image analysis on past video data stored in the storage unit 23 (step S22). In step S22, the control unit 21 extracts video data from the stored video data that contains frames showing the target object.

[0049] The control unit 21 adds additional information to the video data extracted in step S22 (step S23). The control unit 21 transmits the video data with the added information from step S23 to the monitoring device 10 (step S24). The transmitted video data is associated with feature quantities. The feature quantity sharing process is completed. Note that if no frame containing the target object is extracted from the video data in step S22, the video data is not transmitted in step S24.

[0050] Next, referring to Figure 6, the video sharing process performed by the monitoring device 10 will be explained. The video sharing process is the process of combining video data in which the target object is shown in the first video creation process and the feature data sharing process.

[0051] In the monitoring device 10, video data transmitted from the imaging device is started to be received via the communication unit 15 in step S17 of the first video creation process in Figure 4 and in step S24 of the feature quantity sharing process in Figure 5. The control unit 21 uses this start of reception as a trigger to execute the video sharing process according to the video sharing program stored in the storage unit 23.

[0052] As shown in Figure 6, the control unit 11 receives video data transmitted from the imaging device via the communication unit 15 in step S17 of Figure 4 and step S24 of Figure 5 (step S31). In step S31, the control unit 11 stores the received video data in the storage unit 13. The video data received in step S31 may be one or multiple data associated with the same feature quantities. The received video data is stored in the storage unit 13, associated with the feature quantities.

[0053] The control unit 11 combines the video data received in step S31 with video data associated with the same feature quantities as the video data in step S32, arranging them in chronological order. In other words, combined video data is created from multiple video data, having frames arranged in chronological order according to the date and time of shooting. The combined video data is a series of multiple video data in which frames showing the object are arranged in chronological order, or a single video data (video file) in which frames showing the object are arranged in chronological order. The combined video data is in a format that can be played on other devices, such as mobile devices such as smartphones. In step S32, if the control unit 11 has captured the object in frames with the same shooting date and time using multiple camera units 25, it may, for example, use the one with the larger display area of ​​the object for combination. Alternatively, in step S32, if the control unit 11 has captured the object in frames with the same shooting date and time using multiple camera units 25, it may use all of those frames in parallel. Furthermore, in step S32, if there is a time period during the shooting date and time when not all camera units 25 have captured images, the control unit 11 may display a message on the display unit 14 indicating that there is a time period during which images were not captured. In addition, the control unit 11 may supplement the combined video data by adding video frames from the time period during which images were not captured. The video frames from the time period during which images were not captured may be image data generated by image generation AI (Artificial Intelligence) or supplementary image data. Image generation AI is a service or software that automatically generates completed image data simply by providing the desired image or atmosphere in text or image data.

[0054] For example, consider a facility with two adjacent camera units 25A and 25B. If the object being tracked moves from camera unit 25A to a space that cannot be photographed from the field of view of either camera unit, and then to camera unit 25B, the unphotographed space is filled in with image data from the image generation AI or supplementary image data. Supplementary image data is, for example, a photograph of the unphotographed space taken in the past, used as background image data. Image data from the image generation AI is, for example, image data of the facility generated by the image generation AI based on image data taken by camera units 25A and 25B. Alternatively, the image data from the image generation AI or supplementary image data is image data plotted on the background image, using a video generation AI or image data of the object before and after the object, to show how the object being tracked moves.

[0055] The control unit 11 stores the combined video data combined in step S32 in the storage unit 13 and sets the combined video data to be playable (displayed) on the display unit 14 (step S33). The video sharing process ends. In step S33, the control unit 11 may also notify the user that the creation of the combined video data is complete by displaying this information on the display unit 14. Furthermore, if a playback instruction for the combined video data is input via the operation unit 12, the control unit 11 reads the combined video data from the storage unit 13 and plays (displays) it on the display unit 14.

[0056] As shown in Figure 7, in the video sharing process, for example, combined video data 40 is generated. Combined video data 40 includes frames 41 and 42. Frame 41 is, for example, one frame of a video in a first shooting range captured by the camera unit 25 of the shooting device 201. Frame 42 is a frame taken after the shooting time of frame 41. Frame 42 is, for example, one frame of a video in a second shooting range different from the first shooting range, captured by the camera unit 25 of the shooting device 202. Both frames 41 and 42 show the object 51. Frame 41 has, as additional information, a frame 411 surrounding the object 51 and the shooting time 412. Frame 42 has, as additional information, a frame 421 surrounding the object 51 and the shooting time 422.

[0057] In addition, during the first video creation process or the feature sharing process, the accuracy of extracting video data containing the target object using the features (the degree of match corresponding to the features) may be low. The control unit 21 of the shooting device 201... may transmit a message to the monitoring device 10, associated with the video data, indicating that the accuracy of extracting the video data containing the target object is low, if the accuracy of extracting the video data containing the target object is low. When the monitoring device 10 combines and displays the received video data, it will display a message indicating that the accuracy of extracting the video data containing the target object is low.

[0058] Furthermore, in the first video creation process, the extracted feature quantities may be stored for a long period of time in the memory unit 13 of each shooting device. The control unit 21 of each shooting device may use the stored feature quantities to execute steps S14 to S17 when a previously tracked object re-enters the field of view of the real-time video data.

[0059] Furthermore, in the first video creation process, the control unit 21 of the shooting device 201 may share the location information and shooting date and time of the camera unit 25 that captured the image, in addition to the feature quantities, with other shooting devices 202... In the feature quantity sharing process, the control unit 21 of the shooting devices 202... identifies video data that is highly likely to contain the target object based on the shared location information and shooting date and time of the camera unit 25. The video data that is highly likely to contain the target object is the video data of frames captured at a shooting date and time that is inferred to be highly likely to contain the target object based on the location information and shooting date and time of the camera unit 25 that captured the image. The control unit 21 of the shooting devices 202... extracts the video data containing the target object from the identified video data. This reduces the processing time for creating video data.

[0060] As described above, according to this embodiment, the surveillance and photography system 1 comprises a photography device 201... and a monitoring device 10. The photography device 201... comprises a plurality of camera units 25 and a control unit 21. The monitoring device 10 comprises a control unit 11. Each of the plurality of camera units 25 captures a shooting range and generates video data. The control unit 21 analyzes the captured video data and extracts feature quantities of the object to be tracked. Based on the feature quantities, the control unit 21 identifies the object from the plurality of captured video data. The control unit 11 arranges the plurality of video data in which the object has been identified in chronological order and makes them playable as combined video data. Therefore, by playing (displaying) the combined video data, it is possible to track an unpredictable object moving between the plurality of camera units 25.

[0061] Furthermore, by playing back the combined video data, users of the surveillance camera system 1 installed in commercial facilities, etc., can visually confirm the trajectory, appearance, and actions of specific objects. Users can view the data by playing back only a series or a single combined video data, without having to switch access to multiple camera units 25 or switch between multiple saved video data. Because it is a series or a single combined video data, it can be viewed on other devices with small displays, such as mobile devices. The main use cases are shown below. (1) Risk behavior analysis When dangerous behavior by a user occurs within the facility, it may be necessary to share past actions related to the object in question with staff. In an environment where staff and security guards with mobile devices can access the monitoring device 10, the combined video data of this embodiment is shared. Each person can play back the combined video data and check necessary information such as details of the dangerous behavior, the scope of impact, and the date and time of occurrence. (2) Behavioral analysis of the subjects When analyzing the behavior of facility users, a series of actions by the subject can be quickly viewed.

[0062] Furthermore, the control unit 11 arranges multiple video data of identified objects in chronological order to create video data as combined image data. This makes it possible to create a single video data that tracks an unpredictable object moving between multiple camera units 25.

[0063] The control unit 21 analyzes the captured video data and extracts the characteristic features of the objects shown. Therefore, when an anomaly occurs, the characteristic features of the objects shown in the video data can be automatically extracted, and appropriate characteristic features of the objects related to the anomaly can be easily obtained.

[0064] The control unit 21 adds additional information to the video data in which the object has been identified. This additional information includes display elements such as frames, markers, and symbols indicating the object's position, as well as the date and time of shooting. As a result, combined video data containing this additional information can be created from multiple video data sets. When playing back the combined video data, the user can easily recognize the position of the object within the field of view and the date and time of shooting by referring to the additional information.

[0065] The control unit 11 enables playback of multiple frames in parallel when there are multiple frames with the same shooting date and time in multiple video data where the target object has been identified. Therefore, when playing back combined video data, it is easy to check all of the multiple frames in which the target object has been identified and the same shooting time has been taken.

[0066] The control unit 11, when multiple video data sets containing an identified object have multiple frames with the same shooting date and time, selects the frame with the largest display area of ​​the object for playback. Therefore, when playing back combined video data, the object can be easily and reliably identified by using the object with the largest display area.

[0067] The control unit 11, when there are time periods in multiple video data where the target object has been identified that are not captured, uses image generation AI and supplementary image data to generate and supplement video data for those time periods. As a result, when the combined video data is played back, the target object can be recognized without any interruption in time.

[0068] (Second Embodiment) A second embodiment of the present invention will be described with reference to Figures 8 and 9. Figure 8 is a flowchart of the second video creation process. Figure 9 is a plan view showing the camera units 71-88 and the path 100 of the target object installed in the commercial facility 60.

[0069] In the first embodiment, the camera generated video data showing the target object when it detected an anomaly. In this embodiment, the camera generates video data showing the target object when the user specifies and inputs the target object to be tracked.

[0070] The apparatus configuration of this embodiment uses a surveillance and imaging system 1. However, the storage unit 13 of the surveillance device 10 stores a second video creation program. The second video creation program is a program for executing the second video creation process, which will be described later. The control unit 11 functions as an analysis unit.

[0071] Next, the operation of the surveillance and imaging system 1 will be explained with reference to Figures 8 and 9. Referring to Figure 8, the second video creation process performed by the surveillance device 10 will be explained. The second video creation process is a process in which the feature quantities of a specified object from the captured video data are shared with other imaging devices, and video data showing the object created in the feature quantity sharing process is combined. In advance, the control unit 21 of the imaging device 201 captures the space to be monitored with the camera unit 25 and generates real-time video data of the space to be monitored. The control unit 21 transmits the real-time video data to the surveillance device 10 via the communication unit 24. The control unit 11 of the surveillance device 10 receives the real-time video data from the imaging device 201 via the communication unit 15 and displays it on the display unit 14. Similarly to the imaging device 201, imaging devices 202... also generate real-time video data captured with the camera unit 25 and display it on the surveillance device 10.

[0072] In the monitoring device 10, the start of the display processing of the real-time video data is used as a trigger to execute the second video creation process according to the second video creation program stored in the storage unit 23. The user visually checks whether any suspicious objects, such as an unusual person, appear within the frame (viewing angle) of the displayed real-time video data. If an object appears, the user inputs the object within the viewing angle via the operation unit 12.

[0073] As shown in Figure 6, the control unit 11 receives input from the user via the operation unit 12 to specify the object to be tracked within the frame in the displayed real-time video data (step S41). In step S41, the control unit 11 determines whether or not the object to be tracked has been specified. If the object has not been specified (step S41; NO), the process proceeds to step S41.

[0074] If an object is specified (step S41; YES), the control unit 11 performs image analysis on the video data scene in which the specified object is shown (step S42). In step S42, the control unit 11 extracts the feature quantities of the object shown in the scene. The control unit 11 transmits the feature quantities acquired in step S42 to all the imaging devices 201... via the communication unit 15 (step S43).

[0075] Each of the imaging devices 201… that have received the feature data performs the feature data sharing process shown in Figure 5. Steps S44 to S46 are the same as steps S31 to S33 of the video sharing process shown in Figure 6. The second video creation process is then completed.

[0076] Next, with reference to Figure 9, an embodiment of the surveillance camera system 1 and the imaging of an object will be described. As shown in Figure 9, in the commercial facility 60, camera units 71 to 88 are installed as camera units 25 of multiple imaging devices. Each of the camera units 71 to 88 has an imaging range 90. The imaging ranges 90 do not basically overlap, but there are parts where adjacent imaging ranges 90 overlap. The object is a user of the commercial facility 60, moving along the path 100 in the direction indicated by the arrow.

[0077] The user, who is a security officer at the commercial facility 60, is checking real-time video data being captured by cameras 71-88 on the display unit 14 of the monitoring device 10. While monitoring with the cameras, the user notices a user making suspicious movements as they leave the commercial facility 60 and wants to check the user's movements and behavior within the commercial facility 60 up to that point. In step S41 of the second video creation process, the user designates the user as a suspicious object to be tracked.

[0078] In step S42, the monitoring device 10 extracts the feature quantities of the specified input object. In step S43, the monitoring device 10 communicates with all the camera units 71-88 of the monitoring and shooting system 1, and transmits the extracted feature quantities of the object. In step S44, the monitoring device 10 collects all video data of the object from the shooting devices. In step S45, the monitoring device 10 creates combined video data by linking all the video data of the object in chronological order and makes it playable. In chronological order, the combined video data is created by linking the video in the following order along the path 100: (camera unit) 72→71→78→82→83→79→85→80→81→77→76→80→79→72.

[0079] In response to step S46, the control unit 11 may display the combined video data image, map information of the commercial facility 60 site, and location information of the camera units 71-88 on the display unit 14. Furthermore, the control unit 11 may plot how the object moved along the path 100 in chronological order on the map information as trajectory information and display it on the display unit 14 along with the playback of the combined video data.

[0080] As described above, according to this embodiment, the control unit 11 acquires the feature quantities of an object specified and input by the user via the operation unit 12. Therefore, by playing back the combined video data, the user can specify and input an unpredictable object moving between multiple camera units 25 and track it.

[0081] The control unit 11 displays the combined video data on the display unit 14 along with map information and trajectory information of the area where the object included in the combined video data moved. As a result, the user can check how the object moved within the site and how its appearance changed, including the map information, and can understand and confirm the degree of impact on the surrounding area.

[0082] The above description discloses an example in which a storage unit (HDD, SSD) is used as a computer-readable medium for the program according to the present invention, but the invention is not limited to this example. Other computer-readable mediums that can be used include non-volatile memory such as flash memory and portable recording media such as CD-ROMs. Furthermore, a carrier wave can also be used as a medium for providing program data according to the present invention via a communication line.

[0083] The above-described embodiments are merely examples of the video creation system and video creation method according to the present invention, and are not limited thereto. For example, the first and second embodiments may be combined as appropriate.

[0084] Furthermore, in the above embodiment, the control unit 21 of the imaging device 201... analyzes the captured video data and extracts the feature quantities of the object to be tracked, but this is not limited to this. The control unit 21 of each imaging device 201... may transmit the captured video data as is to the monitoring device 10. The control unit 11 of the monitoring device 10 analyzes the received video data and extracts the feature quantities of the object to be tracked. Based on these feature quantities, the control unit 11 identifies the object from the multiple captured video data (extracts video data in which the object is shown). The control unit 11 adds additional information to the multiple video data in which the object has been identified. Alternatively, the control unit 11 may receive the multiple video data in which the object has been identified by the imaging device 201... and add additional information to the multiple video data in which the object has been identified.

[0085] While embodiments of the present invention have been described and illustrated in detail, the disclosed embodiments are for illustrative purposes only and are not limiting. The scope of the present invention should be interpreted by the terms of the appended claims. [Explanation of Symbols]

[0086] 1. Surveillance and filming system 10 Monitoring equipment 11 Control Unit 12 Control section 13 Storage section 14 Display section 15 Communications Department 201, 202… Imaging device 21 Control Unit 22 Control section 23 Memory section 24 Communications Department 25 Camera Section 30 Communication Networks

Claims

1. Multiple shooting units that capture the shooting range and generate video data, An analysis unit analyzes the captured video data and extracts characteristic features of the object to be tracked, Based on the aforementioned feature quantities, an identification unit identifies the object from the multiple captured video data, A video creation system comprising: a control unit that enables playback of multiple video data in which the aforementioned object has been identified, arranged in chronological order.

2. The video creation system according to claim 1, wherein the control unit arranges a plurality of video data in which the target object is identified in chronological order to create video data.

3. The video creation system according to claim 1 or 2, wherein the analysis unit analyzes the captured video data and extracts the feature quantities of the objects shown.

4. The video creation system according to claim 1 or 2, wherein the analysis unit analyzes the captured video data and extracts the feature quantities of the specified input object.

5. The video creation system according to claim 1 or 2, wherein the identifying unit adds additional information to the video data in which the object has been identified.

6. The video creation system according to claim 5, wherein the additional information is at least one of a display element indicating the position of the object and the time of shooting.

7. The video creation system according to claim 1 or 2, wherein the control unit can play back multiple frames in parallel when there are multiple frames with the same shooting time in multiple video data in which the object has been identified.

8. The video creation system according to claim 1 or 2, wherein the control unit, when there are multiple frames with the same shooting time in multiple video data in which the object has been identified, selects the frame with the largest display area of ​​the object and plays it back.

9. The video creation system according to claim 1 or 2, wherein the control unit generates and supplements video data for a time period in which the object has been identified if there is a time period in the multiple video data in which the object has been identified.

10. The video creation system according to claim 1 or 2, wherein the control unit plays back a plurality of video data arranged in chronological order together with map information and trajectory information of the area in which an object included in the plurality of video data has moved.

11. The process involves analyzing video data captured by multiple shooting units that capture the shooting range and generate video data, and extracting the characteristic features of the object to be tracked. A process of identifying the object from multiple video data captured by the multiple imaging units based on the aforementioned feature quantities, A video creation method comprising a control step that arranges the plurality of video data in which the aforementioned object has been identified in chronological order and makes them playable.

Images