Apparatus, method and program

Abstract

To provide a device, a method, and a program for performing image analysis on image data acquired by a surveillance camera.SOLUTION: In a system 1, a device 4 for performing image analysis on image data acquired by at least one surveillance camera includes: a communication unit that receives an image analysis engine included in a plurality of image analysis engines from a distribution server that stores the plurality of image analysis engines, and additionally receives another image analysis engine included in the plurality of image analysis engines; and an analysis unit that receives the setting of operating conditions for each image analysis engine in the device from the distribution server, operates each image analysis engine under the set operating conditions, and performs image analysis on the image data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an apparatus, a method, and a program. [Background technology]

[0002] BACKGROUND ART Conventionally, in a system for monitoring using a monitoring camera, an image analysis unit is provided in a device that acquires image data from the monitoring camera (see, for example, Patent Document 1). Patent Document 1: JP 2015-162232 A Summary of the Invention [Problem to be solved by the invention]

[0003] However, conventional systems do not have a mechanism for changing the image analysis engine used depending on the shooting conditions of each device or the target subject, so image analysis cannot be performed using an appropriate image analysis engine depending on the device. [Means for solving the problem]

[0004] In order to solve the above problem, a first aspect of the present invention provides an apparatus for performing image analysis on image data acquired from at least one surveillance camera, the apparatus comprising: a communication unit that receives an image analysis engine included in a plurality of image analysis engines from a distribution server that stores the image analysis engines, and receives additional image analysis engines included in the plurality of image analysis engines; and an analysis unit that accepts settings of operating conditions for each image analysis engine in the apparatus from the distribution server, and operates each image analysis engine under the set operating conditions to perform image analysis on the image data.

[0005] The above device may further include an image storage unit that stores the image data acquired from the at least one surveillance camera, and the analysis unit may perform image analysis on the image data stored in the image storage unit.

[0006] In the above device, the image storage section may store the image data for at least a reference period, and successively overwrite and store new image data.

[0007] In any of the above devices, the image analysis engine may be a model obtained by machine learning.

[0008] In any of the above devices, the analysis section may output an alert signal in response to detecting a target subject that should be detected from among subjects that may be included in the image data through image analysis of the image data.

[0009] In the above device, the communication unit may transmit the content of the alert signal to an external server.

[0010] In a second aspect of the present invention, there is provided a method executed by a device that performs image analysis on image data acquired from at least one surveillance camera, the method comprising: a communication step of receiving an image analysis engine included in a plurality of image analysis engines from a distribution server that stores the plurality of image analysis engines, and additionally receiving another image analysis engine included in the plurality of image analysis engines; and an analysis step of accepting settings of operating conditions for each image analysis engine in the device from the distribution server, and operating each image analysis engine under the set operating conditions to perform image analysis on the image data.

[0011] In a third aspect of the present invention, a program is provided that causes a computer to function as an apparatus that performs image analysis on image data acquired from at least one surveillance camera, the program including a communication unit that receives an image analysis engine included in a plurality of image analysis engines from a distribution server that stores the image analysis engines and receives additional image analysis engines included in the plurality of image analysis engines, and an analysis unit that accepts settings of operating conditions for each image analysis engine in the apparatus from the distribution server and operates each image analysis engine under the set operating conditions to perform image analysis on the image data.

[0012] In another aspect of the present invention, there is provided a distribution server connected to a device that performs image analysis using an image analysis engine on image data acquired by a surveillance camera. The distribution server may include an analysis engine storage unit that stores multiple image analysis engines that differ in at least one of the shooting conditions of the image data suitable for image analysis and the target subject to be detected from among the subjects that may be included in the image data. The distribution server may include a supply unit that supplies the device with an image analysis engine from the multiple image analysis engines that corresponds to at least one of the shooting conditions of the image data acquired by the device and the target subject set in the device.

[0013] The supplying unit may update the image analysis engine in the device with the supplied image analysis engine.

[0014] The supplying unit may add the supplied image analysis engine to the device.

[0015] The distribution server may further include a setting unit that sets the operating conditions of each image analysis engine within the device.

[0016] The distribution server may further include an instruction unit that instructs each image analysis engine in the device to perform image analysis.

[0017] The distribution server may be connected to a plurality of devices. The distribution server may further include a photographing condition storage unit that stores, for each of the plurality of devices, photographing conditions of image data acquired by the device. The supply unit may supply the same image analysis engine to each of two or more devices among the plurality of devices that acquire image data having at least partially the same photographing conditions.

[0018] The distribution server may be connected to a plurality of devices. The distribution server may further include a detection target storage unit that stores, for each of the plurality of devices, a target subject set in the device. The supply unit may supply the same image analysis engine to each of two or more devices among the plurality of devices that have the same target subject.

[0019] In another aspect of the present invention, there is provided a method. The method may include an analysis engine storage step of storing a plurality of image analysis engines that differ in at least one of shooting conditions of image data suitable for image analysis and target subjects to be detected among subjects that may be included in the image data. The method may include a supply step of supplying, to a device that performs image analysis on image data acquired by a surveillance camera using the image analysis engines, an image analysis engine from the plurality of image analysis engines that corresponds to at least one of the shooting conditions of the image data acquired by the device and the target subjects set in the device.

[0020] In another aspect of the present invention, a program is provided. The program may cause a computer of a distribution server connected to an apparatus that performs image analysis on image data acquired by a surveillance camera using an image analysis engine to function as an analysis engine storage unit that stores multiple image analysis engines that differ in at least one of the shooting conditions of the image data suitable for image analysis and the target subjects to be detected among those that may be included in the image data. The program may cause the computer to function as a supply unit that supplies the apparatus with an image analysis engine from the multiple image analysis engines that corresponds to at least one of the shooting conditions of the image data acquired by the apparatus and the target subjects set in the apparatus.

[0021] The above summary of the invention does not list all of the necessary features of the present invention, and subcombinations of these features may also constitute inventions. [Brief explanation of the drawings]

[0022] [Figure 1] 1 shows a system 1 according to a first embodiment. [Figure 2] 1 shows the contents stored in the position storage unit 43. [Figure 3] 1 shows the contents stored in the imaging condition storage unit 62. [Figure 4] 1 shows the contents stored in the detection target storage unit 63. [Figure 5]This shows the operation related to the distribution of the image analysis engine. [Figure 6] This shows the monitoring operations. [Figure 7] 1 shows a system 1A according to a second embodiment. [Figure 8] This shows the monitoring operations. [Figure 9] 22 illustrates an example computer 2200 in which aspects of the present invention may be embodied, in whole or in part. DETAILED DESCRIPTION OF THE INVENTION

[0023] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0024] [1. First embodiment] [1-1. System 1] FIG. 1 shows a system 1 according to a first embodiment.

[0025] The system 1 includes one or more security sensors 2, one or more surveillance cameras 3, one or more devices 4, a file server 5, a distribution server 6, a surveillance terminal 7, and one or more mobile terminals 8, and performs remote monitoring of the surveillance area.

[0026] The monitoring area may be an area monitored by the monitoring terminal 7, and may include an area where an abnormality is detected by the security sensor 2 (also referred to as a security area) and an area photographed by the monitoring camera 3 (also referred to as a photographing area). In this embodiment, as an example, the monitoring area is the premises of the facility, but it may also be a roadway or a sidewalk. The facility may be a plant, a school, a house, a station, a museum, etc. Examples of plants include industrial plants such as chemical and bio plants, plants that manage and control wellheads and surrounding areas of gas fields and oil fields, plants that manage and control power generation such as hydroelectric power, thermal power, and nuclear power, plants that manage and control environmental power generation such as solar and wind power, and plants that manage and control water supply and sewage systems, dams, etc.

[0027] Here, among the components of system 1, device 4, file server 5, distribution server 6, monitoring terminal 7, and mobile terminal 8 may be connected to one another via a communication network 10. In the figure, device 4, file server 5, distribution server 6, monitoring terminal 7, and mobile terminal 8 are connected via a single communication network 10, but they may also be connected via separate communication networks. Furthermore, each security sensor 2 and monitoring terminal 7 may be further connected via a relay device or communication network (not shown).

[0028] The communication network 10 may be configured to include various networks, such as the Internet, a wide area network (WAN), a local area network, or a combination thereof. The communication network 10 may include at least one of wired and wireless connection points. The communication network 10 may be realized by a dedicated line that is separate from a public line such as the Internet.

[0029] [1-1.1. Security Sensor 2] Each security sensor 2 detects abnormalities in the security area, thereby detecting abnormalities in the monitored area. Each security sensor 2 may be an intrusion detection sensor that detects intrusion into the monitored area, a human presence sensor that detects a human body within the monitored area, or an opening / closing sensor that detects the opening or closing of at least one of a door and a window within the monitored area. Each sensor may be an ultrasonic sensor, a microwave sensor, an infrared sensor, a wire sensor, a trap sensor, an electric field sensor, a magnetic sensor, etc.

[0030] Each security sensor 2 may output an alert signal in response to detecting an abnormality. The alert signal may include the time at which the alert signal was generated and identification information (also referred to as a sensor ID) of the security sensor 2 that is the source of the alert signal. The time at which the alert signal was generated may be the time at which the security sensor 2 detected the abnormality.

[0031] Each security sensor 2 may supply an alert signal to the device 4. Each security sensor 2 may also supply an alert signal to other devices (for example, the monitoring terminal 7).

[0032] [1-1.2. Surveillance Camera 3] Each surveillance camera 3 captures an image of the area to be photographed. Each surveillance camera 3 may supply image data acquired by photographing to the device 4. The surveillance camera 3 may be a visible light camera, or may be an infrared or ultraviolet (X-ray, for example) camera.

[0033] The number of surveillance cameras 3 and the number of security sensors 2 provided in the system 1 may be equal to or different from each other. For example, the system 1 may be provided with the same number of surveillance cameras 3 as the number of security sensors 2, with one surveillance camera 3 installed for each security sensor 2. Alternatively, the system 1 may be provided with fewer surveillance cameras 3 than the number of security sensors 2, with one surveillance camera 3 installed for each group of multiple security sensors 2. The areas photographed by the surveillance cameras 3 and the security areas covered by the security sensors 2 may be the same, or may differ at least in part.

[0034] [1-1.3.Device 4] Each device 4 is connected to at least one security sensor 2 and at least one surveillance camera 3. Each device 4 has an image storage unit 40, an analysis unit 41, an acquisition unit 42, a position storage unit 43, an identification unit 44, and a communication unit 45.

[0035] [1-1.3.1. Image storage unit 40] The image storage unit 40 stores image data acquired by the surveillance camera 3. The image storage unit 40 may store each of the image data acquired by a plurality of surveillance cameras 3.

[0036] The image storage unit 40 may store image data captured by the surveillance cameras 3 in association with the shooting date and time and the identification information (also referred to as camera ID) of each surveillance camera 3. The image storage unit 40 may store image data for at least a reference period (for example, five days) and may successively overwrite and store new image data. The capacity of the image storage unit 40 may be, for example, 1 TB.

[0037] [1-1.3.2.Analysis section 41] The analysis unit 41 performs image analysis on the image data and outputs an alert signal in response to detecting a target subject to be detected from among subjects that may be included in the image data. The target subject may be set in the device 4 in advance by an operator, or may be changed as desired. The analysis unit 41 may perform image analysis on the image data stored in the image storage unit 40, and may supply the alert signal to the acquisition unit 42.

[0038] The alert signal output from the analysis unit 41 includes at least the time the alert signal was generated and the camera ID of the surveillance camera 3 that captured the image data. The alert signal may further include information about the target subject (for example, the type, characteristics, and position of the target subject within the image data) and information indicating that the source of the alert signal is the analysis unit 41. The time the alert signal was generated may be the time the image data in which the target subject was detected was captured.

[0039] The target subject may be a subject that should be noted for security purposes among subjects that can be detected by image analysis. For example, the target subject may be at least one of a human face, the entire body of a person (for example, an adult or a child), clothing, a vehicle (a moving vehicle or a parked vehicle), a vehicle model, a vehicle color, a license plate, and a specific object (a bag, a weapon). The target subject may also be the face of a specific person or a specific vehicle. The target subject may be a person or vehicle moving between images (between frames), or an action that occurs across multiple images.

[0040] The analysis unit 41 is an example of a first analysis unit, and may perform image analysis on image data using one or more image analysis engines (not shown). As an example in the present embodiment, the analysis unit 41 may perform image analysis by operating multiple image analysis engines in parallel or at different times.

[0041] Operating multiple image analysis engines at different times may mean, for example, operating one image analysis engine during a period when image analysis is performed on image data captured during the day, and operating another image analysis engine during a period when image analysis is performed on image data captured at night. Operating multiple image analysis engines at different times may mean operating one image analysis engine during a period when image analysis is performed on image data captured in direct sunlight, and operating another image analysis engine during a period when image analysis is performed on image data captured in backlight. Operating multiple image analysis engines at different times may mean operating one image analysis engine during a period when image analysis is performed on image data captured in a sunny environment, and operating another image analysis engine during a period when image analysis is performed on image data captured in a cloudy or rainy environment. Operating multiple image analysis engines at different times may mean operating different image analysis engines at each reference interval (for example, one minute). Furthermore, operating multiple image analysis engines at different times may mean operating separate image analysis engines so that the target subject changes during a period in which the same (or similar) subject is included in the image data. Changing the target subject during a period in which the same (or similar) subject is included in the image data may mean, for example, that the target subject changes between a face and a movement during a period in which a person is included as the subject in the image data, or that the target subject changes between a vehicle model and a license plate during a period in which a car is included as the subject in the image data.

[0042] At least one of the target subject and the shooting conditions of the image data suitable for image analysis may differ among the multiple image analysis engines. The shooting conditions of the image data may be conditions determined by the surveillance camera 3 that captures the image, or may be conditions determined by the installation status and shooting timing of the surveillance camera 3. For example, the shooting conditions of the image data may be at least one of the performance (number of pixels, frame rate, etc.) of the surveillance camera 3 that captures the image, its purpose (indoor, outdoor, etc.), the shooting direction (direction, orientation relative to the light source, orientation relative to target subjects that may be included in the image data, etc.), the installation location (address, GPS data, relative position relative to target subjects that may be included in the image data, indoor or outdoor, etc.), the number of target subjects that may be included in the image data (single or multiple, etc.), the status of target subjects that may be included in the image data (moving, stationary, etc.), the time of day when the image is captured, and the status of the shooting area at the time of capture (color, brightness, weather of the lighting).

[0043] The image analysis engine may be, for example, a model obtained by machine learning such as deep learning, but is not limited to this.

[0044] [1-1.3.3. Acquisition part 42] The acquisition unit 42 acquires an alert signal that is generated in accordance with the situation within the monitored area. The acquisition unit 42 may acquire the alert signal from at least one security sensor 2, or may acquire the alert signal from the analysis unit 41. The acquisition unit 42 may supply the acquired alert signal to the identification unit 44.

[0045] [1-1.3.4.Position storage section 43] The position storage unit 43 stores the position of the shooting area of ​​each of the multiple surveillance cameras 3. The position storage unit 43 may further store the position of the security area of ​​each of the multiple security sensors 2.

[0046] [1-1.3.5. Specific part 44] The identification unit 44 identifies each of the multiple surveillance cameras 3 having a shooting area corresponding to the location where the cause of the alert signal occurred. The identification unit 44 may identify the surveillance camera 3 by referring to the position memory unit 43. The identification unit 44 may supply the camera ID of the identified surveillance camera 3 and the alert signal to the communication unit 45.

[0047] Here, when an alert signal is output from a security sensor 2, the cause of the alert signal may be an abnormality in the object detected by the security sensor 2, and the location where the cause of the alert signal occurred may be the security area of ​​the security sensor 2 that output the alert signal. In this case, the identification unit 44 may refer to the position memory unit 43 to identify each surveillance camera 3 having a shooting area corresponding to the security area of ​​the security sensor 2 that output the alert signal. The shooting area corresponding to the position of the security area of ​​the security sensor 2 may be a shooting area within a reference distance from the security area, a shooting area on the same floor (story) or building as the security area, or a shooting area on a route that includes the security area.

[0048] When an alert signal is output from the analysis unit 41, the cause of the alert signal may be the target subject, and the location where the cause of the alert signal occurred may be the shooting area of ​​the surveillance camera 3 that captured the image data in which the target subject was detected. In this case, the identification unit 44 may identify the surveillance camera 3 that corresponds to the camera ID included in the alert signal.

[0049] [1-1.3.6.Communications Section 45] The communication unit 45 communicates with the file server 5 and the like via the communication network 10.

[0050] The communication unit 45 is an example of a transmission unit, and transmits to the file server 5 image data within a reference time span including the time when an alert signal was generated, from among the image data stored in the image storage unit 40, and alert response information including the content of the alert signal. In response to acquisition of an alert signal by the acquisition unit 42, the communication unit 45 may transmit image data within a reference time span including the time when the alert signal was generated, acquired by each of the multiple surveillance cameras 3 identified by the identification unit 44, and alert response information including the content of the alert signal. In this embodiment, the reference time span is, for example, two minutes, but may be another time span, such as ten minutes or one day. Transmitting image data within a period may mean transmitting a copy of the image data within the period.

[0051] In addition, the communication unit 45 may further include in the alert response information and transmit the identification information of the device 4 (also referred to as the device ID) and the camera ID of the surveillance camera 3 identified by the identification unit 44, i.e., the surveillance camera 3 that captured the image data within the period.

[0052] Furthermore, when no alert signal is supplied from the identification unit 44, the communication unit 45 may periodically transmit thumbnail images of image data acquired by each monitoring camera 3 to the file server 5 or the monitoring terminal 7. Furthermore, the communication unit 45 may transmit image data stored in the image storage unit 40 to the file server 5 before it is overwritten. In this case, the communication unit 45 may compress and transmit the image data. However, if the image data to be transmitted before overwriting includes image data that has already been transmitted as image data within the period, the communication unit 45 may transmit image data within a time span (e.g., 30 minutes) including the image data within the period without compressing it. This allows image data captured at a time close to the image data within the period to be stored in the file server 5.

[0053] [1-1.4. File Server 5] The file server 5 can be accessed by devices other than the device 4 (in this embodiment, as an example, the monitoring terminal 7 and the mobile terminal 8), and stores files that can be shared with the other devices. The file server 5 may be simultaneously accessible from multiple devices without increasing communication costs. The file server 5 includes an analysis unit 51 and a storage unit 52.

[0054] [1-1.4.1.Analysis section 51] The analysis unit 51 is an example of a second analysis unit, and performs image analysis on image data within a period corresponding to the alert response information (image data within a period included in the alert response information in this embodiment) using an image analysis engine. The analysis unit 51 may perform image analysis when alert response information including an alert signal from the analysis unit 41 of the device 4 is received by the file server 5. The analysis unit 51 may perform image analysis to attempt to detect the same target subject as the analysis unit 41 of the device 4 that sent the alert response information. For example, the analysis unit 51 may attempt to detect the same type of target subject based on the type of target subject included in the alert signal.

[0055] The image engine of analysis unit 51 may require greater processing power for image analysis and may provide higher accuracy in image analysis than the image analysis engine of analysis unit 41. As an example, if the image analysis engines of analysis unit 41 and analysis unit 51 are each neural networks, the image analysis engine of analysis unit 51 may be a neural network with a greater number of layers than the image analysis engine of analysis unit 41. Analysis unit 51 may supply the analysis results to storage unit 52.

[0056] [1-1.4.2. Storage section 52] The storage unit 52 stores the alert response information transmitted from the device 4. Of the alert response information transmitted from the device 4, the storage unit 52 may store only the alert response information corresponding to image data within a period in which the target subject was detected by the analysis unit 51. With regard to the alert response information including the alert signal from the security sensor 2, the storage unit 52 may store each piece of alert response information regardless of the analysis result by the analysis unit 51.

[0057] The storage unit 52 may further store image data of thumbnail images transmitted from the device 4 when no alert signal has been generated, and image data transmitted from the device 4 before being erased by overwriting, in association with the device ID of the transmission source. The storage unit 52 may have a larger capacity than the image storage unit 40.

[0058] [1-1.5. Distribution Server 6] The distribution server 6 distributes the image analysis engine to each device 4. The distribution server 6 includes an analysis engine storage unit 61, a shooting condition storage unit 62, a detection target storage unit 63, a setting unit 64, and a supply unit 65.

[0059] [1-1.5.1. Analysis engine memory unit 61] The analysis engine storage unit 61 stores multiple image analysis engines. At least one of the target subject and the shooting conditions of the image data suitable for image analysis may differ between the multiple image analysis engines. If the image analysis engines are generated by machine learning, each image analysis engine stored in the analysis engine storage unit 61 may be updated by learning using the accuracy of the analysis results.

[0060] [1-1.5.2. Shooting condition storage unit 62] The imaging condition storage unit 62 stores, for each of the multiple devices 4, the imaging conditions of the image data acquired by the device 4. If the imaging conditions change, the imaging condition storage unit 62 may store multiple imaging conditions. The imaging conditions to be stored may be set by an operator.

[0061] Alternatively, the stored shooting conditions may be set based on the results of image analysis performed by each image analysis engine in the analysis engine storage unit 61 on image data output from the device 4. For example, image analysis may be performed on image data from one device 4 using two or more image analysis engines from the multiple image analysis engines in the analysis engine storage unit 61, each having the same target subject but different shooting conditions for image data suitable for image analysis. Then, as a result of the image analysis, the shooting conditions corresponding to the image analysis engine that detected the most target subjects may be set as the shooting conditions for the image data acquired by the one device 4. The image data to be subjected to image analysis may be, for example, image data such as thumbnail images stored in the file server 5.

[0062] [1-1.5.3. Detection target storage unit 63] The detection target storage unit 63 stores, for each of the multiple devices 4, the target subject set in that device 4. The detection target storage unit 63 may store two or more target subjects for one device 4. The target subjects to be stored may be set by an operator, or may be changed as desired.

[0063] [1-1.5.4.Settings section 64] The setting unit 64 sets the operating conditions of each image analysis engine in each device 4. The operating conditions of the image analysis engine may be conditions that determine the timing at which the image analysis engine performs image analysis, or may be shooting conditions of the image data to be analyzed. As an example, the operating condition may be that the image data is captured during the day or at night. In this case, the image analysis engine performs image analysis on image data captured during the day or at night.

[0064] The setting unit 64 may set the operating conditions of the image analysis engine used by the analysis unit 41 to the analysis unit 41 of each device 4 via the supply unit 65. As a result, the analysis unit 41 of each device 4 operates each image analysis engine under the set operating conditions.

[0065] [1-1.5.5. Supply section 65] The supply unit 65 supplies one of the multiple image analysis engines stored in the analysis engine storage unit 61 to at least one corresponding device 4. The supply unit 65 may supply the device 4 with an image analysis engine that corresponds to at least one of the shooting conditions of the image data acquired by the device 4 and the target subject set in the device 4.

[0066] The supply unit 65 may refer to the shooting condition storage unit 62 to detect the shooting conditions of the image data acquired by the device 4. The supply unit 65 may refer to the detection target storage unit 63 to detect the target subject set in the device 4.

[0067] The supply unit 65 may supply the same image analysis engine to two or more devices 4 that acquire image data under at least partially equal shooting conditions, among the multiple devices 4 included in the system 1. As a result, for example, when two or more devices 4 acquire image data captured in the same time period, an image analysis engine suitable for the image data captured in that time period is supplied to each of the two or more devices 4.

[0068] The supply unit 65 may supply the same image analysis engine to each of two or more devices 4 having the same target subject among the multiple devices 4 included in the system 1. The supplied image analysis engine may be received by the communication unit 45 of the device 4 and used by the analysis unit 41.

[0069] [1-1.6.Monitoring Terminal 7] The monitoring terminal 7 monitors the monitoring area. The monitoring terminal 7 includes a display control unit 71 and a reporting unit 72.

[0070] [1-1.6.1. Display control unit 71] When alert response information is output from the device 4, the display control unit 71 displays the image data within the period included in the alert response information and the content of the alert signal. When the source of the alert signal is the security sensor 2, displaying the content of the alert signal may mean displaying the time the alert signal was generated, the sensor ID of the security sensor 2, and the content of the abnormality detected by the security sensor 2 with that sensor ID. When the source of the alert signal is the analysis unit 41, displaying the content of the alert signal may mean displaying the time the alert signal was generated, information about the target subject, and the camera ID of the surveillance camera 3.

[0071] When no alert response information is output from a device 4, the display control unit 71 may display a list of thumbnail images of image data acquired from each monitoring camera 3 via each device 4. When one or more pieces of alert response information are output in this state, the display control unit 71 may distinguishably display thumbnail images of image data acquired by the monitoring cameras 3 of each camera ID included in the alert response information. Then, in response to the operator selecting a thumbnail image of one of the monitoring cameras 3, the display control unit 71 may display the image data within the period acquired from that monitoring camera 3 and an alert signal corresponding to the image data within the period.

[0072] Alternatively, when one or more pieces of alert response information are output while thumbnail images are displayed in a list, the display control unit 71 may display a list of the contents of the alert signals included in each piece of alert response information, and when an alert signal is selected by the operator, the content of the alert signal and the image data within the period corresponding to that alert signal are displayed.

[0073] In addition, when a list of thumbnail images is displayed and no alert response information is output from the device 4, the display control unit 71 may read and display image data obtained from the surveillance camera 3 from the image memory unit 40 of the device 4 in response to the operator selecting a thumbnail image of the surveillance camera 3.

[0074] [1-1.6.2. Reporting Section 72] In response to alert response information being transmitted from one of the multiple devices 4, the reporting unit 72 sends a report to a report destination corresponding to the one device 4. The reporting unit 72 may have a table (not shown) that associates a report destination with each device 4, and may determine the report destination by referring to this table. The report may be sent by data communication, and the communicated data may include the alert response information.

[0075] In this embodiment, the destination of the report is, for example, a security company in the area that includes the monitoring area of ​​the device 4, but it may also be the police, fire station, or emergency hospital.

[0076] [1-1.7.Mobile Devices8] Each mobile terminal 8 is carried by a security guard of a local security company. The mobile terminal 8 may be able to access the file server 5.

[0077] [1-2. Benefits obtained from System 1] According to the distribution server 6 of the above-described system 1, an image analysis engine corresponding to at least one of the shooting conditions of the image data acquired by the device 4 and the target subject set in the device 4 is supplied to the device 4 from among the multiple image analysis engines stored in the analysis engine storage unit 61. This allows the device 4 to perform image analysis using an appropriate analysis engine. For example, when the shooting conditions of the image data differ depending on the surveillance camera 3, the device 4 can be made to perform image analysis using an image analysis engine corresponding to the shooting conditions of the surveillance camera 3. Furthermore, when the target subject is changed, the device 4 can be made to perform image analysis using an image analysis engine corresponding to the changed target subject.

[0078] In addition, since the same image analysis engine is supplied to each of two or more devices 4 that acquire image data with at least partially equal shooting conditions, image analysis can be performed using the same image analysis engine on the image data acquired by each device 4 with at least partially equal shooting conditions.

[0079] Furthermore, since the same image analysis engine is supplied to each of two or more devices 4 that have the same target subject, the same image analysis engine can be used to perform image analysis in each device 4 that is to detect the same target subject.

[0080] Furthermore, the setting unit 64 sets the operating conditions of each image analysis engine in the device 4, so that by setting any operating conditions in advance, the device 4 can perform image analysis under any operating conditions.

[0081] Furthermore, according to the device 4 of the above-described system 1, of the image data stored in the image storage unit 40, image data within a period within a reference time width that includes the time at which an alert signal is generated in response to the situation within the monitored area, and alert response information including the content of the alert signal, are transmitted to the file server 5, thereby reducing communication costs compared to when image data for the entire captured period is transmitted to the file server 5. Furthermore, unlike when image data is not transmitted to the file server 5 but is stored only in the image storage unit 40 within the device 4, the time and cost required to access the image data can be reduced.

[0082] Furthermore, each image data acquired by the multiple surveillance cameras 3 is stored in the image storage unit 40 of the device 4, and alert response information is sent to the file server. Therefore, communication costs can be significantly reduced compared to when image data captured by each surveillance camera 3 over an entire period is sent to the file server 5. The file server 5 is suitable for simultaneous access from multiple devices, and, except for communication with the mobile terminal 8, allows communication at a fixed rate regardless of the amount of data communicated, so communication costs can be kept constant even if the image data is accessed repeatedly.

[0083] In addition, upon acquisition of an alert signal, each of the multiple surveillance cameras 3 having a shooting area corresponding to the location where the cause of the alert signal occurred is identified, and alert response information including image data acquired within the period by each of the identified surveillance cameras 3 is transmitted, allowing the status of each area related to the alert to be confirmed.

[0084] Furthermore, an alert signal from the security sensor 2 is acquired by the acquisition unit 42, and an alert signal is acquired in response to an approaching danger in the monitored area, and alert response information is transmitted to the file server 5. Therefore, by monitoring via the file server 5, danger in the monitored area can be avoided.

[0085] Furthermore, image analysis is performed on the image data, and an alert signal is output in response to the detection of a target subject set in the device 4, so by setting any target subject (for example, a subject that may pose a danger) in advance, alert response information is sent to the file server 5 when the target subject is present. Therefore, by monitoring via the file server 5, target subjects present in the monitoring area can be discovered early.

[0086] Furthermore, in the analysis unit 41, a plurality of image analysis engines operate in parallel or with staggered timing, so that when a target subject is present in the monitoring area, the target subject can be reliably detected.

[0087] Furthermore, according to the monitoring terminal 7, the content of the alert signal included in the alert response information and the image data within the period corresponding to the alert signal are displayed on the monitoring terminal 7, so that the monitoring area can be monitored in an integrated manner using the alert signal and the image data within the period.

[0088] In addition, when alert response information is sent from one of the multiple devices 4, a report is sent to the report destination corresponding to that device 4, allowing personnel to be dispatched to the monitored area early.

[0089] Furthermore, according to device 4 and file server 5, device 4 performs image analysis on the image data using one image analysis engine, and outputs an alert signal in response to detection of a target subject. Furthermore, file server 5 performs image analysis on within-period image data corresponding to the alert response information using another image analysis engine, and stores only the alert response information corresponding to within-period image data in which the same target subject as device 4 was detected. This makes it possible to prevent unnecessary within-period image data in which a target subject was not detected by image analysis in file server 5 from being wasted in file server 5.

[0090] Furthermore, the image analysis engine in the analysis unit 41 of the device 4 requires less processing power for image analysis than the image analysis engine in the analysis unit 41 of the file server 5, so the load of image analysis on the device 4 can be reduced.

[0091] [1-2.Position storage unit 43] 2 shows the contents stored in the position storage unit 43. In this embodiment, as an example, the position storage unit 43 stores the camera ID (e.g., "c-01") of the surveillance camera 3 and the position of the image capture area (e.g., "inside the building entrance") in association with each other. The position storage unit 43 also stores the sensor ID (e.g., "s-01") of the security sensor 2 and the position of the security area (e.g., "automatic entrance door") in association with each other.

[0092] [1-3. Shooting condition storage unit 62] 3 shows the contents stored in the photographing condition storage unit 62. In the present embodiment, as an example, the photographing condition storage unit 62 stores, for each of the multiple devices 4, the device ID of the device 4 (for example, "aaa1"), the camera ID of each monitoring camera 3 connected to the device 4 (for example, "c-01"), and the photographing conditions of the image data (for example, the monitoring camera 3 is for indoor use and is installed indoors), in association with each other.

[0093] [1-4. Detection target storage unit 63] 4 shows the contents stored in the detection target storage unit 63. In the present embodiment, as an example, the detection target storage unit 63 stores, for each of the multiple devices 4, the device ID of the device 4 (for example, "aaa1") and the target subject set for the device 4 (for example, "human face") in association with each other.

[0094] [1-5. System 1 Operation] [1-5.1. Image analysis engine distribution] 5 shows the operation related to the distribution of an image analysis engine. The system 1 distributes an image analysis engine to each device 4 by performing the processes of steps S11 to S13. At the start of this operation, the analysis engine storage unit 61 of the distribution server 6 may store multiple image analysis engines that differ in at least one of the shooting conditions of image data suitable for image analysis and the target subject to be detected. Each image analysis engine may include information indicating the detected subject to be detected by image analysis and the shooting conditions of image data suitable for image analysis.

[0095] In step S11, the supply unit 65 supplies one of the multiple image analysis engines stored in the analysis engine storage unit 61 to the device 4. The supply unit 65 may refer to at least one of the shooting condition storage unit 62 and the detection target storage unit 63, and supply to the device 4 an image analysis engine that corresponds to at least one of the shooting conditions of the image data acquired by the device 4 and the target subject set in the device 4. Note that in step S11, the supply unit 65 may receive operating conditions of the image analysis engine to be supplied from the setting unit 64, and further supply them to the device 4.

[0096] The supply unit 65 may update the image analysis engine in the device 4 with the supplied image analysis engine, or may add the supplied image analysis engine to the device 4. For example, the supply unit 65 may instruct the analysis unit 41 of the device 4 to update or add an image analysis engine. When instructing the analysis unit 41 to update an image analysis engine, the supply unit 65 may specify the image analysis engine to be updated from among one or more image analysis engines used by the analysis unit 41.

[0097] The supply unit 65 may supply the image analysis engine in response to an operation by an operator. For example, when an operator operates to update and / or add an image analysis engine stored in the analysis engine storage unit 61, the supply unit 65 may supply the image analysis engine to the device 4. In this case, the supply unit 65 may detect the detected subject of each updated or added image analysis engine and the shooting conditions of image data suitable for image analysis, and supply the image analysis engine to the device 4 corresponding to at least one of the detected subject and the shooting conditions.

[0098] Furthermore, the supply unit 65 may supply an image analysis engine when the information in at least one of the detection target storage unit 63 and the shooting condition storage unit 62 is changed by an operator's operation. In this case, the supply unit 65 may supply an image analysis engine according to the changed detected subject to each device 4 corresponding to the detected subject changed in the detection target storage unit 63. Furthermore, the supply unit 65 may supply an image analysis engine according to the changed shooting conditions to each device 4 corresponding to the shooting conditions changed in the shooting condition storage unit 62.

[0099] In step S13, the analysis unit 41 of the device 4 acquires an image analysis engine via the communication unit 45. When the analysis unit 41 is instructed by the supply unit 65 to update an image analysis engine, it updates one of the image analysis engines with the supplied image analysis engine. When the analysis unit 41 is instructed by the supply unit 65 to add an image analysis engine, it adds the supplied image analysis engine to the list of engines to be used. When the analysis unit 41 receives operating conditions for the image analysis engine from the supply unit 65, it sets the supplied image analysis engine to operate under those operating conditions.

[0100] According to the above operation, the image analysis engine in device 4 is updated with the supplied image analysis engine, so that even if device 4 has limited storage space, it can perform image analysis using the new image analysis engine.

[0101] Furthermore, since the image analysis engine to be supplied is added to the device 4, it is possible to have the device 4 perform image analysis using a plurality of image analysis engines.

[0102] [1-5.2. Monitoring] 6 shows the operation related to monitoring. The system 1 monitors the monitoring area by performing the processes of steps S21 to S37. During this operation, the device 4 may store image data acquired by the monitoring camera 3 in the image storage unit 40. The analysis unit 41 of the device 4 may perform image analysis on the acquired image data using one or more image analysis engines.

[0103] In step S21, in response to an alert signal being output from at least one of the security sensor 2 and the analysis unit 41, the acquisition unit 42 of the device 4 acquires the alert signal.

[0104] In step S23, the identification unit 44 identifies, from among the plurality of monitoring cameras 3, each monitoring camera 3 having a shooting area corresponding to the position where the cause of the alert signal occurred.

[0105] In step S25, the communication unit 45 transmits alert response information including image data within a reference time width including the time when the alert signal was generated, from the image data acquired by each identified monitoring camera 3 and stored in the image storage unit 40, to the file server 5. The communication unit 45 may also transmit a notification that the alert response information has been output to the monitoring terminal 7 or the mobile terminal 8. As an example in the present embodiment, the communication unit 45 transmits only the content of the alert signal to the monitoring terminal 7 or the mobile terminal 8, but it may also transmit the alert response information itself together with the image data within the period. Note that although in FIG. 6 the device 4 ends the processing after transmitting the alert response information, the processing may proceed to step S21.

[0106] In step S27, the storage unit 52 of the file server 5 stores the alert response information. When the file server 5 receives the alert response information including the alert signal output from the analysis unit 41 of the device 4, the analysis unit 51 performs image analysis, and the storage unit 52 may store the alert response information only when the same target subject as that detected by the analysis unit 41 of the device 4 that sent the alert response information is detected from the image data within the period.

[0107] In step S31, in response to receiving an alert signal, the display control unit 71 of the monitoring terminal 7 reads out alert response information including the alert signal from the file server 5. In response to this, in step S33, the file server 5 may provide the monitoring terminal 7 with access to the storage unit 52. Note that if an alert signal is not transmitted from the device 4, the display control unit 71 may receive the alert signal from the security sensor 2 without going through the device 4.

[0108] In step S35, the display control unit 71 displays the image data within the period included in the alert response information and the contents of the alert signal on one or more displays (not shown). Note that when alert response information including the image data within the period is transmitted from the communication unit 45 of the device 4 to the monitoring terminal 7, the display control unit 71 may display the image data within the period and the alert signal included in the received alert response information without performing the process of step S31.

[0109] When the alert response information includes multiple sets of image data within a period captured by multiple surveillance cameras 3, the display control unit 71 may preferentially display, among these sets of image data within a period, image data within a period in which an object moving between images is detected. Detection of an object moving between images may be performed by the analysis unit 51 of the file server 5, or by an analysis unit (not shown) within the surveillance terminal 7. Preferentially displaying image data within a specific period may mean displaying only the image data within that period, or may mean displaying the image data within that period larger than other image data within that period.

[0110] Here, the processing of steps S31 and S35 may be performed by the mobile terminal 8. In this case, the file server 5 provides the mobile terminal 8 with access to the storage unit 52, and the image data within the period and the contents of the alert signal are displayed on the mobile terminal 8. This allows the security guard to check the situation in the monitored area.

[0111] In step S37, in response to the alert response information being transmitted from one of the plurality of devices 4, the reporting unit 72 of the monitoring terminal 7 sends a report to the reporting destination corresponding to that one device 4.

[0112] [1-6. Modification of the first embodiment] In the first embodiment described above, the communication unit 45 transmits alert response information including image data within a period captured by each monitoring camera 3 having a shooting area corresponding to the location where the cause of the alert signal occurred. However, the communication unit 45 may transmit alert response information including image data within a period captured by each monitoring camera 3 regardless of the shooting area. For example, in response to acquisition of an alert signal by the acquisition unit 42, the communication unit 45 may transmit alert response information including multiple pieces of image data within a period captured by multiple monitoring cameras 3 within a reference time span that includes the time when the alert signal occurred, and the content of the alert signal. In this case, the situation over a wide area can be confirmed on the monitoring terminal 7.

[0113] Although the alert response information has been described as including an alert signal and image data within a period, the alert response information may include an address of the image data within the period in the image storage unit 40 instead of the image data within the period. In this case, the cost of communication between the device 4 and the file server 5 can be reduced compared to when the image data within the period is transmitted together with the alert response information. Furthermore, since the image data within the period in the image storage unit 40 is accessed using the transmitted address, the time and cost required to access the image data within the device 4 can be reduced compared to when the image data for the entire period in the image storage unit 40 is accessed to search for image data around the time the alert signal occurred. Here, if the alert response information includes an alert signal and an address of the image data within the period, the monitoring terminal 7 may read the image data within the period indicated by the address from the image storage unit 40 of the device 4 and display the content of the alert signal included in the alert response information and the image data within the period corresponding to the alert signal. Furthermore, the monitoring terminal 7 may store the image data within the period read based on the address included in the alert response information in the file server 5 in association with the alert response information.

[0114] Furthermore, although the distribution server 6 has been described as having a setting unit 64 that sets the operating conditions of each image analysis engine in each device 4, it may also have an instruction unit (not shown) that instructs each image analysis engine in the device 4 to perform image analysis. In this case, the image analysis engine can be made to perform image analysis each time depending on the situation.

[0115] Furthermore, although the device 4 has been described as having the analysis unit 41, the position storage unit 43, and the identification unit 44, it is not necessary for the device 4 to have at least one of these. For example, if the device 4 does not have the analysis unit 41, it may output alert response information in response to an alert signal from the security sensor 2.

[0116] Furthermore, although the file server 5 has been described as having the analysis unit 51, it may not have such an analysis unit 51. In this case, the file server 5 may store all of the alert response information that it receives.

[0117] [2. Second Embodiment] [2-1. System 1A] Fig. 7 shows a system 1A according to a second embodiment. In the system 1A according to this embodiment, the operations that are substantially the same as those of the system 1 shown in Fig. 1 are assigned the same reference numerals, and the description thereof will be omitted.

[0118] The system 1A includes a device 4A, a file server 5A, and a monitoring terminal 7A. Although the security sensor 2 is not shown in Fig. 7, the system 1A may include one or more security sensors 2 that supply an alert signal to the monitoring terminal 7A.

[0119] [2-1.1.Device 4A] The device 4A includes an analysis unit 41A and a communication unit 45A.

[0120] [2-1.1.1.Analysis section 41A] The analysis unit 41A performs image analysis on the image data, and generates feature data indicating the features of the target subject in response to detecting the target subject among the subjects that may be included in the image data. The analysis unit 41A may perform image analysis on the image data stored in the image storage unit 40, and may supply the feature data to the communication unit 45. The analysis unit 41A may perform image analysis on each surveillance camera 3 to generate feature data.

[0121] If the target subject is a human face, the feature data may indicate facial features in the image. The facial features in the image may be the positional relationship of facial feature points, the color of eyes, skin, or hair, or the color or shape of accessories (e.g., glasses, earrings, etc.). If the target subject is a vehicle, the feature data may indicate the model of the vehicle in the image. If the target subject is a license plate, the feature data may indicate the number of the vehicle in the image.

[0122] The type of target subject may be one or more. When there are multiple target subjects, the analysis unit 41A may generate feature data for at least one of the target subjects. The feature data may be used as an identifier for extracting specific image data from the multiple image data.

[0123] The analysis unit 41A may perform image analysis on image data using one or more image analysis engines (not shown), similar to the analysis unit 41 of the first embodiment. As an example, in the present embodiment, the analysis unit 41A may perform image analysis by operating multiple image analysis engines in parallel or at staggered timings.

[0124] [2-1.1.2. Communication Unit 45A] The communication unit 45A communicates with the file server 5A and the like via the communication network 10.

[0125] The communication unit 45A is an example of a transmission unit, and transmits metadata of the image data for which feature data has been generated to the file server 5A. When feature data is generated for multiple pieces of image data acquired by multiple surveillance cameras 3, the communication unit 45A may transmit metadata for each of the multiple surveillance cameras 3. The communication unit 45A may transmit metadata each time the content of the feature data changes.

[0126] Here, the metadata may be data indicating the content and location of the corresponding image data, and may include feature data, the shooting time of the image data from which the feature data was generated, and / or the address of the image data within the image storage unit 40.

[0127] If the metadata includes the shooting time, the metadata may further include a camera ID and a device ID. If the metadata includes the address of the image data in image storage unit 40, the address may further indicate the address of device 4A in system 1A.

[0128] The communication unit 45A may receive the image analysis engine transmitted from the distribution server 6 and supply it to the analysis unit 41, similar to the communication unit 45 of the first embodiment.

[0129] [2-1.2. File Server 5A] The file server 5A can be accessed by devices other than the device 4 (in this embodiment, as an example, the monitoring terminal 7 and the mobile terminal 8), and stores files that can be shared with the other devices. The file server 5A includes a storage unit 52A.

[0130] The storage unit 52A stores the metadata transmitted from the device 4A. The storage unit 52A may also store image data corresponding to the metadata in association with the metadata.

[0131] [2-1.3. Monitoring terminal 7A] The monitoring terminal 7A monitors the monitoring area and includes a selection unit 75, a communication unit 76, and a display control unit 71A.

[0132] [2-1.3.2. Selection section 75] The selection unit 75 selects one of the metadata stored in the file server 5A in response to an operation by an operator. The selection unit 75 may have an input unit 751 and a search unit 752, and may select the metadata detected by the input unit 751 and the search unit 752 and supply the selected metadata to the communication unit 76.

[0133] The input unit 751 inputs feature data of a subject in response to an operation by an operator. The input unit 751 may supply the input feature data to the search unit 752.

[0134] Here, the feature data input by the operator may be data indicating the features of a subject that may have been photographed by any of the surveillance cameras 3. For example, when the police or the like requests information about a specific person or vehicle, the feature data input by the operator may be data indicating the features of the person or vehicle.

[0135] The search unit 752 searches for metadata that includes the input feature data from among multiple metadata (in this embodiment, as an example, data that includes feature data and at least one of the shooting time and address of the image data) stored in the file server 5A.

[0136] [2-1.3.1. Communications Section 76] The communication section 76 is an example of a reading section, and reads image data corresponding to the selected metadata from the device 4. The communication section 76 may supply the read image data to the display control section 71A.

[0137] Here, when the metadata includes a shooting time, the image data corresponding to the metadata may be image data captured at that shooting time, or image data captured within a reference time span that includes the shooting time. In this embodiment, the reference time span is 2 minutes as an example, but may be other time spans such as 10 minutes or 1 day.

[0138] If the metadata further includes a camera ID and a device ID, the image data corresponding to the metadata may be image data stored in the device 4A corresponding to the device ID and captured by the surveillance camera 3 corresponding to the camera ID. If the metadata includes a device ID but not a camera ID, the image data corresponding to the metadata may be image data stored in the device 4A corresponding to the device ID and captured by each surveillance camera 3. If the metadata does not include a device ID and a camera ID, the image data corresponding to the metadata may be image data stored in each device 4A and captured by each surveillance camera 3.

[0139] Furthermore, if the metadata includes an address of image data within the image storage unit 40, the image data corresponding to the metadata may be image data within the image storage unit 40 of the device 4A indicated by the address, or may be image data captured within a reference time range that includes the image data.

[0140] The communication unit 76 is also an example of a storage control unit, and stores the read image data in the file server 5A. The communication unit 76 may store the read image data in the file server 5A in association with the metadata used for reading.

[0141] [2-1.3.4. Display control unit 71A] The display control unit 71A displays the image data read by the communication unit .

[0142] According to device 4A of system 1A described above, image analysis is performed on image data acquired by surveillance camera 3, and upon detection of a predetermined subject, feature data indicating the characteristics of the subject is generated. Then, metadata including the feature data and at least one of the capture time of the image data for which the feature data was generated and the address of the image data in image storage unit 40 is transmitted to file server 5A. Therefore, the metadata can be stored in file server 5A and used as an index for searching image data. This facilitates access to image data for which feature data was generated among the image data in device 4, and makes it easy to trace back and find subjects that were present in the surveillance area.

[0143] Furthermore, since metadata is transmitted for each of the multiple surveillance cameras 3, it becomes even easier to find a subject that was present in the surveillance area.

[0144] Furthermore, since metadata is transmitted each time the content of the feature data changes, metadata is transmitted each time the subject changes, which makes it easier to access image data for each subject and to find subjects that were present in the monitored area.

[0145] Furthermore, since feature data indicating at least one of the facial features in the image, the vehicle number in the image, and the model of the vehicle in the image is generated and included in the metadata, it is possible to easily find people and cars that were present in the monitored area. Similarly, when feature data indicating the features of a person's entire body or clothing, a vehicle (a moving vehicle or a parked vehicle), the color of a vehicle, a specific object (a bag, a weapon), etc. is generated and included in the metadata as feature data of the target subject, it is also possible to easily find these target subjects that were present in the monitored area.

[0146] Furthermore, in the analysis unit 41A, a plurality of image analysis engines operate in parallel or at staggered timings, so that when a target subject is present in the monitoring area, the target subject can be reliably detected.

[0147] Furthermore, the monitoring terminal 7A of the system 1A allows an operator to select one of the metadata stored in the file server 5A, and then read and display the image data corresponding to the selected metadata. This allows the operator to easily check any image data for which feature data has been generated.

[0148] Furthermore, when feature data of a subject is input in response to an operator's operation, metadata including the input feature data is detected and displayed from among the multiple metadata stored in the file server 5A, so that image data of the subject corresponding to any feature data can be easily confirmed.

[0149] Furthermore, since the read image data is stored in the file server 5A, the image data corresponding to the selected metadata can be collected in the file server 5A. Furthermore, the image data for which feature data has been generated can be prevented from being lost due to overwriting within the device 4A, and the image data can be saved reliably.

[0150] [2-2. Operation of System 1A] [2-2.1. Image analysis engine distribution] The system 1A may perform the operation of distributing the image analysis engine in the same manner as in the first embodiment described above, thereby distributing the image analysis engine for generating feature data to the device 4A.

[0151] [2-2.2. Monitoring] 8 shows the operation related to monitoring. System 1A monitors the monitoring area by performing the processes of steps S51 to S69. During this operation, device 4A may store image data acquired by monitoring camera 3 in image storage unit 40.

[0152] In step S51, analysis unit 41A of device 4A performs image analysis on the acquired image data using one or more image analysis engines, and generates feature data indicating the features of the target subject in response to the detection of the target subject.

[0153] In step S53, the communication unit 45 generates metadata including the feature data, the shooting time of the image data from which the feature data was generated, and / or the address of the image data within the image storage unit 40, and transmits the metadata to the file server 5A.

[0154] In step S55, the storage unit 52A of the file server 5A stores the transmitted metadata.

[0155] In step S57, the selection unit 75 of the monitoring terminal 7A selects one of the metadata stored in the file server 5A in response to an operation by the operator. As an example in the present embodiment, the selection unit 75 may select metadata including feature data input by the operator via the input unit 751.

[0156] In step S61, communication unit 76 reads image data corresponding to the selected metadata from image storage unit 40 of device 4. In response, in step S63, device 4A may provide monitoring terminal 7 with access to image storage unit 40.

[0157] In step S65, the display control unit 71A of the monitoring terminal 7A displays the read image data. Note that the processes of steps S57, S61, and S65 may be performed by the mobile terminal 8. In this case, the device 4 provides access to the image storage unit 40 to the mobile terminal 8, and the read image data is displayed on the mobile terminal 8. This allows the security guard to check the situation in the monitored area.

[0158] In step S67, the communication section 76 associates the read image data with the metadata and transmits the image data to the file server 5 A. The communication section 76 may perform the transmission in response to an instruction from an operator who has confirmed the displayed image.

[0159] Then, in step S69, the storage unit 52A of the file server detects, from among the multiple pieces of metadata that have already been stored, metadata that is the same as the received metadata, and stores the received image data in association with that metadata.

[0160] In the second embodiment, the image data corresponding to the metadata selected by the monitoring terminal 7A is read from the device 4A and stored in the file server 5A. However, other methods may be used to prevent the loss of the image data corresponding to the metadata. For example, the analysis unit 41 of the device 4A may lock the image data from which feature data has been generated in the image storage unit 40 so that the image data is not overwritten or erased.

[0161] 3. Modifications of the First and Second Embodiments In the first and second embodiments, the monitoring terminals 7 and 7A and the distribution server 6 are described as separate devices, but they may be the same device.

[0162] Furthermore, although the distribution server 6 has been described as having the photographing condition storage unit 62, the detection target storage unit 63, and the setting unit 64 (or instruction unit), it may not have at least one of these. For example, if the distribution server 6 does not have the setting unit 64 (or instruction unit), the operating conditions of the image analysis engine may be set on the device 4, 4A side.

[0163] Furthermore, although the devices 4, 4A have been described as storing image data acquired by the surveillance camera 3 in the image storage unit 40, they may also store image data in response to detection of a target subject by the analysis unit 41, 41A. In this case, the amount of image data stored in the devices 4, 4A can be reduced, and therefore the devices 4, 4A do not need to transmit alert response data or metadata to the file server 5, 5A.

[0164] Furthermore, although the devices 4, 4A have been described as having a single analysis unit 41, 41A, they may have a plurality of the same type of analysis units 41, 41A, in which case image analysis can be performed in parallel at high speed.

[0165] Various embodiments of the present invention may also be described with reference to flowcharts and block diagrams, where the blocks may represent (1) stages of a process in which operations are performed or (2) sections of an apparatus responsible for performing the operations. Particular stages and sections may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable medium, and / or a processor provided with computer-readable instructions stored on a computer-readable medium. Dedicated circuitry may include digital and / or analog hardware circuitry, and may include integrated circuits (ICs) and / or discrete circuits. Programmable circuitry may include reconfigurable hardware circuitry, including logical AND, OR, XOR, NAND, NOR, and other logic operations, flip-flops, registers, memory elements such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like.

[0166] A computer-readable medium may include any tangible device capable of storing instructions that are executed by an appropriate device, such that the computer-readable medium having instructions stored thereon comprises an article of manufacture containing instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable media may include electronic, magnetic, optical, electromagnetic, and semiconductor storage media. More specific examples of computer-readable media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray (RTM) disc, memory stick, integrated circuit card, and the like.

[0167] The computer readable instructions may include either assembler instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0168] The computer-readable instructions may be provided to a processor or programmable circuitry of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, either locally or over a wide-area network (WAN) such as a local area network (LAN), the Internet, etc., which executes the computer-readable instructions to create means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0169] 9 illustrates an example of a computer 2200 in which aspects of the present invention may be embodied, in whole or in part. Programs installed on the computer 2200 may cause the computer 2200 to function as or perform operations associated with an apparatus or one or more sections of the apparatus according to embodiments of the present invention, and / or to perform a process or steps of a process according to embodiments of the present invention. Such programs may be executed by the CPU 2212 to cause the computer 2200 to perform specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0170] A computer 2200 according to this embodiment includes a CPU 2212, a RAM 2214, a graphics controller 2216, and a display device 2218, which are interconnected by a host controller 2210. The computer 2200 also includes input / output units such as a communication interface 2222, a hard disk drive 2224, a DVD-ROM drive 2226, and an IC card drive, which are connected to the host controller 2210 via an input / output controller 2220. The computer also includes legacy input / output units such as a ROM 2230 and a keyboard 2242, which are connected to the input / output controller 2220 via an input / output chip 2240.

[0171] The CPU 2212 operates according to programs stored in the ROM 2230 and RAM 2214, thereby controlling each unit. The graphics controller 2216 acquires image data generated by the CPU 2212 into a frame buffer or the like provided in the RAM 2214 or into the graphics controller 2216 itself, and causes the image data to be displayed on the display device 2218.

[0172] The communication interface 2222 communicates with other electronic devices via a network. The hard disk drive 2224 stores programs and data used by the CPU 2212 in the computer 2200. The DVD-ROM drive 2226 reads programs or data from the DVD-ROM 2201 and provides the programs or data to the hard disk drive 2224 via the RAM 2214. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0173] The ROM 2230 stores therein a boot program or the like that is executed by the computer 2200 upon activation, and / or programs that depend on the hardware of the computer 2200. The input / output chip 2240 may also connect various input / output units to the input / output controller 2220 via a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0174] The programs are provided by a computer-readable medium such as a DVD-ROM 2201 or an IC card. The programs are read from the computer-readable medium, installed in the hard disk drive 2224, RAM 2214, or ROM 2230, which are also examples of computer-readable media, and executed by the CPU 2212. Information processing described in these programs is read by the computer 2200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by realizing information manipulation or processing in accordance with the use of the computer 2200.

[0175] For example, when communication is performed between the computer 2200 and an external device, the CPU 2212 may execute a communication program loaded into the RAM 2214 and instruct the communication interface 2222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 2212, the communication interface 2222 reads transmission data stored in a transmission buffer processing area provided in the RAM 2214, the hard disk drive 2224, the DVD-ROM 2201, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer processing area or the like provided on the recording medium.

[0176] The CPU 2212 may also cause all or a necessary portion of a file or database stored on an external recording medium such as the hard disk drive 2224, the DVD-ROM drive 2226 (DVD-ROM 2201), an IC card, etc. to be read into the RAM 2214, and perform various types of processing on the data on the RAM 2214. The CPU 2212 then writes back the processed data to the external recording medium.

[0177] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and may undergo information processing. The CPU 2212 may perform various types of processing on data read from the RAM 2214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 2214. The CPU 2212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored on the recording medium, the CPU 2212 may search for an entry that matches a condition specified by the attribute value of the first attribute from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0178] The above-described programs or software modules may be stored in a computer-readable medium on or near the computer 2200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable medium, thereby providing the programs to the computer 2200 via the network.

[0179] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0180] It should be noted that the execution order of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order. [Explanation of symbols]

[0181] 1 System, 1A System, 2 Security Sensor, 3 Surveillance Camera, 4 Device, 4A Device, 5 File Server, 5A File Server, 6 Distribution Server, 7 Surveillance Terminal, 7A Surveillance Terminal, 8 Mobile Terminal, 10 Communication Network, 40 Image Memory Unit, 41 Analysis Unit, 41A Analysis Unit, 42 Acquisition Unit, 43 Position Memory Unit, 44 Identification Unit, 45 Communication Unit, 45A Communication Unit, 51 Analysis Unit, 52 Memory Unit, 52A Memory Unit, 61 Analysis Engine Memory Unit, 62 Shooting Condition Memory Unit, 63 Detection Target Memory Unit, 64 Setting Unit, 65 Supply Unit, 71 Display Control Unit, 71A Display Control Unit, 72 Reporting Unit, 75 Selection Unit, 76 Communication Unit, 751 Input Unit, 752 Search Unit, 2200 Computer, 2201 DVD-ROM, 2210 Host Controller, 2212 CPU, 2214 RAM, 2216 graphics controller, 2218 display device, 2220 input / output controller, 2222 communication interface, 2224 hard disk drive, 2226 DVD-ROM drive, 2230 ROM, 2240 input / output chip, 2242 keyboard

Claims

1. An apparatus for performing image analysis on image data acquired from at least one surveillance camera, a communication unit that receives an image analysis engine included in a plurality of image analysis engines from a distribution server that stores the image analysis engines, and additionally receives another image analysis engine included in the plurality of image analysis engines; an analysis unit that receives settings of operating conditions for each image analysis engine in the device from the distribution server and operates each image analysis engine under the set operating conditions to perform image analysis on the image data; Equipped with The operating conditions of each image analysis engine determine the timing at which the image analysis engine performs image analysis or the shooting conditions of the image data to be subjected to image analysis.

2. The device according to claim 1 , wherein the communication unit receives the operating conditions of the image analysis engine from the distribution server each time the communication unit receives the image analysis engine.

3. further comprising an image storage unit for storing the image data acquired from the at least one surveillance camera; The apparatus according to claim 1 , wherein the analysis unit performs image analysis on the image data stored in the image storage unit.

4. The device according to claim 3 , wherein the image storage section stores the image data for at least a reference period, and successively overwrites and stores new image data.

5. The apparatus according to claim 1 , wherein the image analysis engine is a model obtained by machine learning.

6. The device according to claim 1 , wherein the analysis unit outputs an alert signal in response to detecting a target subject to be detected from among subjects that may be included in the image data through image analysis of the image data.

7. The device according to claim 6 , wherein the communication unit transmits the content of the alert signal to an external server.

8. 1. A method executed by a device for performing image analysis on image data acquired from at least one surveillance camera, comprising: a communication step of receiving an image analysis engine included in the plurality of image analysis engines from a distribution server that stores the plurality of image analysis engines, and additionally receiving another image analysis engine included in the plurality of image analysis engines; an analysis step of accepting settings of operating conditions for each image analysis engine in the device from the distribution server, and operating each image analysis engine under the set operating conditions to perform image analysis on the image data; Equipped with The operating conditions of each image analysis engine determine the timing at which the image analysis engine performs image analysis or the shooting conditions of the image data to be subjected to image analysis.

9. A program that causes a computer to function as a device that performs image analysis on image data acquired from at least one surveillance camera, The computer, a communication unit that receives an image analysis engine included in a plurality of image analysis engines from a distribution server that stores the image analysis engines, and additionally receives another image analysis engine included in the plurality of image analysis engines; an analysis unit that receives settings of operating conditions for each image analysis engine in the device from the distribution server, and operates each image analysis engine under the set operating conditions to perform image analysis on the image data; It functions as The operating conditions of each image analysis engine determine the timing at which the image analysis engine performs image analysis or the shooting conditions of the image data to be subjected to image analysis.

Images