Image analysis system and image analysis method

Abstract

To appropriately determine, from an image obtained by imaging a monitoring area, the height position of a person in the image.SOLUTION: In a video analysis system 1 for detecting a height position of a person in a monitoring target area, using an image obtained by imaging the monitoring target area, an input part 31 of a position determination system 3 receives one or more images for calculation which are images of the monitoring target area imaged under an imaging condition that the size of a person at a position higher than a predetermined reference height appears larger than the size of a person at the reference height and that can specify the person at the reference height, a detection part 32 detects regions of a plurality of reference persons at the reference height from the one or more images for calculation, and a calculation part 33 calculates a feature related to the sizes of the reference persons on the basis of the regions of the plurality of reference persons, and calculates a relation for the sizes of the reference persons in the image under the imaging condition in the monitoring target area on the basis of the feature.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a technique for detecting a person from an image such as a still image or a moving image (video) captured in a predetermined monitoring area, and determining the height position of the person. [Background technology]

[0002] In work areas such as building and construction sites, workers perform high-altitude work using ladders, stepladders, scaffolding, etc. Measures must be taken to prevent accidents from falling from high places. The number of supervisors who check whether workers are taking appropriate measures is limited, so effective and efficient supervision of workers is required.

[0003] In response to this, Patent Document 1 focuses on the fact that workers wear safety belts and hooks as fall arrest devices to prevent accidents caused by falls, and discloses a configuration for providing a safety belt usage monitoring system that can determine the usage status of a safety belt by attaching an acceleration sensor to the hook and determining whether the hook is being used based on the received acceleration data.

[0004] Furthermore, Patent Document 2 discloses a configuration for providing a safety monitoring system that monitors whether fall arrest equipment is being used properly, in which an imaging device is prepared to photograph the work site, a worker is detected from the photographed image, the height of the worker's working position is calculated based on the size or length of a specific object detected in the image, and a high-altitude work judgment is made to determine whether the height is above a predetermined threshold value. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2022-029791 [Patent Document 2] Japanese Patent Publication No. 2023-074846 Summary of the Invention [Problem to be solved by the invention]

[0006] However, with the technology disclosed in Patent Document 1, the determination of hook attachment / detachment can be affected by body movements, so even when the worker is on flat ground where safety is not an issue, other actions may be mistakenly detected as attachment / detachment actions. Therefore, in order to reduce false detections, it is necessary to determine whether the worker is at a high altitude before determining whether the worker is attaching or detaching.

[0007] Furthermore, the technology disclosed in Patent Document 2 is a method for determining the height of a person based on the size of a predefined object captured around the person. Therefore, it is not possible to determine the height of a person for an object that is not predefined. Furthermore, in an environment where it is difficult to predefine objects, such as when a person may climb onto various objects, it is difficult to apply image recognition of objects. Furthermore, if a person appears to be located on top of an object in the image but is actually located behind it, this can lead to a false detection of the person's height. Therefore, there is a demand for a method for determining the working height of a worker without relying on object detection.

[0008] The present invention has been made in consideration of the above circumstances, and its purpose is to provide a technology that can appropriately determine the height position of a person in an image captured of a monitored area. [Means for solving the problem]

[0009] In order to achieve the above-mentioned object, an image analysis system according to one aspect is an image analysis system that detects the height position of a person in a monitored area using an image captured of the area, and includes at least one arithmetic unit. The arithmetic unit receives one or more calculation images that are images of the monitored area captured under imaging conditions set so that the size of a person at a position higher than a predetermined reference height appears larger than the size of a person at the reference height, and that can identify a person at the reference height. The arithmetic unit detects areas of multiple reference persons at the reference height from the one or more calculation images, calculates feature values related to the size of the reference persons based on the areas of the multiple reference persons, and calculates a relationship between the size of the reference persons in an image of the monitored area under the imaging conditions based on the feature values. [Effects of the Invention]

[0010] According to the present invention, the height position of a person in an image captured in a monitored area can be appropriately determined from the image. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is an explanatory diagram of a video analysis system according to a first embodiment. [Figure 2] FIG. 2 is a functional configuration diagram of the video analysis system according to the first embodiment. [Figure 3] FIG. 3 is a hardware configuration diagram of the video analysis system according to the first embodiment. [Figure 4] FIG. 4 is an explanatory diagram of the position determination system according to the first embodiment. [Figure 5] FIG. 5 is a flowchart of the relationship calculation process according to the first embodiment. [Figure 6] FIG. 6 is a flowchart of the height position determination process according to the first embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of setting the determination region according to the first embodiment. [Figure 8] FIG. 8 is a diagram illustrating the relationship according to the first embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of a determination region setting screen according to the first embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a threshold condition setting screen according to the first embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of a search processing screen according to the first embodiment. [Figure 12] FIG. 12 is a diagram illustrating an example of a threshold condition setting screen according to the second embodiment. [Figure 13] FIG. 13 is a functional configuration diagram of a video analysis system according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] The following description of the embodiments will be given with reference to the drawings. Note that the embodiments described below do not limit the scope of the invention as claimed, and not all of the elements and combinations thereof described in the embodiments are necessarily essential to the solution of the invention.

[0013] A video analysis system will be described below as an example of an image analysis system according to an embodiment. The video analysis system aims to determine whether a person is located at a higher or lower elevation compared to a reference height in a video (image) captured (photographed) of a monitored area, without relying on the result of object detection. The video analysis system calculates feature amounts related to the size of a person from an area where the person is detected in a calculation video (calculation image) for calculating the relationship, calculates a relationship between a predetermined reference point on the person and the feature amounts and the person's feature amounts at the reference height, and determines the person's height position from the feature amounts and the relationship by referring to a threshold condition set in advance for determining the person's height position.

[0014] In the image analysis system according to the embodiment, a person detected in the image is determined to be higher or lower than a reference height in the image without relying on object detection results. To determine the height position, the system first calculates and stores a relationship between the person's feature values from multiple images (video frame images) based on multiple feature values related to the size of the person at the reference height. Next, to determine the height position, the system determines whether the feature value related to the person's size reaches a threshold value using the person's feature values, the calculated relationship, and a predetermined threshold condition related to the height position, thereby detecting the person's height position. In this embodiment, the system can determine the person's height position based solely on the person detection results, regardless of the object detection results in the image. Therefore, the system can determine the person's height position in an area containing undefined objects, enabling height position determination even in environments where it is difficult to define objects in advance. Furthermore, because object information is not used, false detections can be reduced when a person appears to be standing on an object in the image. Furthermore, according to this embodiment, since the method is based on anomaly detection, in which a person's appearance deviates from the expected appearance of a person at a reference height, it is determined that the person's height position is at a high or low altitude, and there is no need to prepare a video (image) of the person actually at a high or low altitude in order to calculate the relationship, which is safe and easy for the person to realize. In a system that determines whether a person (e.g., a worker) has a safety hook attached when working at a height, by performing a high altitude position determination as a preliminary process for the determination, it is possible to reduce the possibility of erroneously detecting an action of the worker when he or she is in a safe place, for example, on flat ground, as an attaching or detaching action.

[0015] In this specification, "relationship" refers to the relationship between multiple feature quantities related to the size of a person at a reference height, and can be expressed, for example, by a frequency distribution, a mathematical formula, etc. "User" refers to a person who can access the video analysis system, operate the system settings, or check the video and analysis results. For example, a user could be a manager, monitor, or on-site security guard of a space where a camera that captures images is installed.

[0016] Hereinafter, an embodiment will be described with reference to the drawings. [Example]

[0017] FIG. 1 is an explanatory diagram of a video analysis system according to a first embodiment.

[0018] The video analysis system 1 includes a shooting system 2, a position determination system 3, and a monitoring center system 4.

[0019] The photographing system 2 has at least one camera unit 21 installed in at least one area 5 to be monitored.

[0020] The position determination system 3 detects people who have been photographed (pictured) within the monitored area by analyzing the input image (video) from the photographing system 2, calculates the relationship between multiple feature amounts related to the size of the person based on a predetermined reference point for the person, and determines the height position of the person by determining whether the feature amount for the person whose height position is to be detected reaches a predetermined threshold value based on the relationship.

[0021] The monitoring center system 4 receives the height position determination result from the position determination system 3, effectively displays it to the user (such as the monitor 6 or security guard), and records part or all of the video. The monitoring center system 4 also transmits setting information about the threshold (for example, threshold conditions, specification of the threshold itself, etc.) to the position determination system 3. The monitoring center system 4 may also receive and display the video transmitted from the imaging system 2.

[0022] Next, the photographing system 2, the position determining system 3, and the monitoring center system 4 will be described in detail.

[0023] FIG. 2 is a functional configuration diagram of the video analysis system according to the first embodiment.

[0024] The photographing system 2 includes one or more camera units 21. The camera units 21 are surveillance cameras that are installed in an area to be monitored (monitored area 5) and capture images of the monitored area. The camera units 21 capture images of the monitored area 5 and sequentially transmit the captured images to the position determination system 3.

[0025] The camera unit 21 is set, for example, so as to face the monitored area 5 so that the height and depression angle of the camera do not change. The camera unit 21 is not limited to a fixed camera, but may also be a mobile camera. If the video can be converted into a video from a free viewpoint, the height and depression angle of the camera of the camera unit 21 may be changed, and the video may be converted into a video from a desired viewpoint for use in processing. The camera unit 21 does not necessarily need to transmit the entire acquired image to the position determination system 3; it may perform mask processing on the acquired image and transmit only a partial area of the image to be used by the position determination system 3 to the position determination system 3.

[0026] The photographing system 2 and the position determination system 3 are connected by wired communication and / or wireless communication, and the camera unit 21 sequentially transmits video, i.e., continuously transmits frame images of the video, to the position determination system 3. The photographing system 2 and the monitoring center system 4 are also connected by wired communication and / or wireless communication, and the camera unit 21 continuously transmits frame images to the monitoring center system 4.

[0027] In the position determination system 3, when time-series data that assumes the input of multiple frame images is used to calculate feature quantities, it is desirable that the frame rate at which frame images are continuously transmitted by the camera unit 21 is equal to or higher than the required value for the functional units (described later) of the position determination system 3. However, if a decrease in recognition accuracy due to a frame rate lower than the required value is acceptable, the frame rate may be lower than the required value. In this case, it is desirable to perform processing to suppress accuracy degradation, such as interpolation or extrapolation of the time-series data, when inputting it to each functional unit.

[0028] The camera units 21 and the position determination systems 3 do not have to correspond one-to-one, and the camera units 21 and the monitoring center systems 4 do not have to correspond one-to-one. For example, one position determination system 3 and one monitoring center system 4 may be provided for multiple camera units 21. When one position determination system 3 processes video data from multiple camera units 21 through multiple processes, the frame rate from each camera unit 21 required by each functional unit conforms to the above-mentioned constraints. Also, multiple position determination systems 3 may be provided for one camera unit 21, and in this case, video analysis processing may be distributed among the multiple position determination systems 3 for video from one camera unit 21.

[0029] The camera unit 21 may be equipped with some or all of the functions of the position determination system 3, which will be described later. For example, the camera unit 21 may include at least one arithmetic unit and at least one storage unit, perform edge processing for person detection processing, and transmit only information related to the area of the person to the position determination system 3, thereby reducing the processing load on the position determination system 3.

[0030] The position determination system 3 includes an input unit 31, a detection unit 32, a calculation unit 33, a determination unit 34, an output unit 35, and a storage unit 36. The position determination system 3 may be an on-premise system constructed on a server within an operation facility, or may be constructed by a server outside the facility by utilizing a cloud service, for example.

[0031] The input unit 31 sequentially receives input of video from at least one camera unit 21 and transmits the video data to the detection unit 32. The input unit 31 is not limited to receiving video directly from the camera unit 21, but may also receive video stored in any other location (e.g., the storage unit 36 or a storage device not shown). If the frame rate of the received video is higher than the frame rate of subsequent processing (e.g., person area detection processing, position determination processing, etc.), the input unit 31 may thin the frame rate to a frame rate sufficient for subsequent processing in order to reduce the amount of analysis processing. In this embodiment, in the relationship calculation processing for calculating the relationship, the input unit 31 receives one or more calculation images that are images of the monitored area captured under imaging conditions set so that a person at a position higher than a predetermined reference height appears larger than a person at the reference height, and that can identify a person at the reference height. Such calculation images are, for example, prepared in advance by the user.

[0032] The detection unit 32 performs person detection on the input image (video data) using a trained person detection model, and obtains the area of the person (bounding box). The bounding box is a circumscribing rectangle of the person, and is expressed by the vertex coordinates of the rectangle in the input image. An existing person detection model can be used as the person detection model. The person detection model outputs, for example, the bounding box of the person and the degree of certainty that the bounding box is a person.

[0033] For example, a person in an image may not necessarily be standing, but may be crouching. Therefore, the detection unit 32 may apply a posture estimation technique to the image of the person to identify the coordinates of the person's joints, and calculate a circumscribing rectangle corresponding to a predetermined posture of the person (e.g., standing posture) from this information.

[0034] The calculation unit 33 uses the acquired information about the person's bounding box to calculate a feature amount related to the person's size from the person's reference point and information about the person's area. For example, the calculation unit 33 may acquire the midpoint of the rectangle base (corresponding to the person's feet) from the person's circumscribing rectangle as a reference point, and calculate the pixel size in the height direction as a feature amount. The reference point may be the center of the human body, the head, or multiple locations (head, neck). The feature amount may also be, for example, the area of the circumscribing rectangle, as long as it is an amount that represents the size of the person in the image.

[0035] The calculation unit 33 calculates the relationship between the feature amounts of a person at a predetermined reference height based on the plurality of feature amounts, and stores the relationship in the storage unit 36. The calculation unit 33 acquires, for example, a plurality of feature amounts for one or more people at the reference height in an image for calculation captured by the camera unit 21 in the monitored area 5, and calculates a frequency distribution of the feature amounts of the people at the reference height as the relationship. Note that, when a plurality of determination areas are set in the height direction of the image, the calculation unit 33 calculates the relationship for each determination area. Here, which determination area a person is in may be determined, for example, by the position of the person's reference point.

[0036] Note that since the bounding box detected by the person detection model may contain noise, in order to increase robustness against noise, it may be possible to select whether or not to use the bounding box for calculating the features based on the confidence level output by the detection unit 32, and it may also be possible to perform person tracking processing on the same person based on time-series video, thereby correcting the features of the person based on the time-series information of the bounding box of the same person.

[0037] The determination unit 34 compares the feature calculated by the calculation unit 33 with the relationship and threshold condition stored in the storage unit 36, and determines the relationship with the threshold, thereby determining the height position of the target person and notifying the output unit 35. The determination regarding the height position of the person may be, for example, a determination that the person is at a high place or a low place.

[0038] The output unit 35 collects information such as the content of the event determined by the determining unit 34, the time of the event occurrence, the monitored area where the event occurred, and the like, and transmits the collected information to the monitoring center system 4.

[0039] The storage unit 36 stores the relationship obtained by the calculation unit 33. The storage unit 36 also stores threshold conditions for the relationship. For example, the threshold conditions include a condition that a 3 sigma section of the obtained frequency distribution is within a reference height range, and other areas are considered to be high or low. The threshold conditions can be set by the user via the control unit 43.

[0040] The monitoring center system 4 includes a recording unit 41 , a display unit 42 , and a control unit 43 .

[0041] The recording unit 41 receives information on the determination results, such as the event obtained by the location determination by the location determination system 3, the monitored area where the event occurred, and the time of the event occurrence, and stores this information as a database. The database may be constructed in a storage server (not shown) of the monitoring center system 4, or in a cloud storage server (not shown), and may be constructed in any location.

[0042] The display unit 42 displays images transmitted from the camera unit 21. When a predetermined event defined in advance occurs, the display unit 42 displays information about some or all of the frames at the time of the event occurrence. In this case, the display unit 42 may highlight in the image the person (target person) for whom the predetermined event occurred. The display unit 42 may also have a function to search for event information and desired video from the recording unit 41 based on at least one condition such as the type of event, the area to be monitored, and the time of occurrence, and a function to display them.

[0043] The control unit 43 receives setting information from the user, such as threshold conditions used by the determination unit 34 of the position determination system 3 and threshold designations, and stores the information in the storage unit 36 .

[0044] Next, the hardware configuration of the video analysis system 1 will be described.

[0045] FIG. 3 is a hardware configuration diagram of the video analysis system according to the first embodiment.

[0046] The video analysis system 1 includes one or more camera units 71 and computers 72 and 73. The camera unit 71 and the computer 72 are connected via a network 10. The computers 72 and 73 are connected via a network 11. The camera unit 71 may be configured to communicate directly with the computer 73 without going through the computer 72. The one or more camera units 71 constitute an imaging system 2, the computer 72 constitutes a position determination system 3, and the computer 73 constitutes a monitoring center system 4. At least one camera unit 71 is installed in the area to be monitored, and transmits video data to the computer 72 and the computer 73 as appropriate.

[0047] The computer 72 has at least one arithmetic unit and at least one storage device. For example, the computer 72 includes a central processing unit (CPU) 721 as the arithmetic unit, a random access memory (RAM) 722 as the main storage device, a hard disk drive (HDD) 723 or a solid state drive (SSD) as the auxiliary storage device, and a communication interface (IF) 724. The computer 72 may also include a graphical processing unit (GPU) as the arithmetic unit, as necessary. The computer 72 may also be connected to input / output devices such as a keyboard and a display via a predetermined IF.

[0048] The IF 724 is an interface such as a wired LAN card or a wireless LAN card, and communicates with other devices (for example, the camera unit 71 and the computer 73) via the networks 10 and 11.

[0049] The CPU 721 executes various processes according to programs stored in the RAM 722 and / or the HDD 723 .

[0050] The RAM 722 stores the programs executed by the CPU 721 and necessary information.

[0051] The HDD 723 stores programs executed by the CPU 721 and data used by the CPU 721 .

[0052] In the computer 72, various programs are read from the HDD 723, loaded into the RAM 722, and executed by the CPU 721 or GPU, thereby configuring the input unit 31, detection unit 32, calculation unit 33, determination unit 34, output unit 35, and storage unit 36.

[0053] The computer 73 has at least one arithmetic unit and at least one storage device. For example, the computer 73 includes a CPU 731 as an arithmetic unit, a RAM 732 as a main storage device, a HDD 733 as an auxiliary storage device, and a communication interface (IF) 734. The computer 73 may also be connected to input / output devices such as a keyboard and a display via a predetermined IF.

[0054] The IF 734 is an interface such as a wired LAN card or a wireless LAN card, and communicates with other devices (for example, the computer 72) via the network 11.

[0055] The CPU 731 executes various processes according to programs stored in the RAM 732 and / or the HDD 733 .

[0056] The RAM 732 stores the programs executed by the CPU 731 and necessary information.

[0057] The HDD 733 stores programs executed by the CPU 731 and data used by the CPU 731 .

[0058] In the computer 73, various programs are read from the HDD 733, loaded into the RAM 732, and executed by the CPU 731 or the GPU, thereby configuring the recording unit 41, the display unit 42, and the control unit 43.

[0059] When part or all of the processing of the position determination system 3 is performed by the photographing system 2, the photographing system 2 is configured by the hardware of the camera unit 71 and part or all of the computer 72.

[0060] Next, the processing of the position determination system 3 will be described.

[0061] FIG. 4 is an explanatory diagram of the position determination system according to the first embodiment.

[0062] In the example image shown in Figure 4, the camera unit 21 is positioned to take pictures from a high position relative to the monitored area 5, and within the monitored area 5 being photographed by the camera unit 21, there are people 511, 5121, 5122, 5131, 5132, and 514, as well as an object 521 that the people can climb to a high place and a space 522 that the people can enter at a low place.

[0063] In this example, person 5121 and person 5122 are climbing on object 521, and person 514 is in space 522. Furthermore, person 5121, person 5122, person 5131, person 5132, and person 514 use the midpoint of the bottom of each person's bounding box (the person's feet) as a reference point, and the image height in the height direction on the image from this reference point is a common height 53. In this case, person 5121 and person 5122 are located at a high altitude, person 5131 and person 5132 are located at the same reference height as person 511, and person 514 is located at a low altitude.

[0064] The position determination system 3 determines the height positions of these people and finds people located in particular at high and low places.

[0065] In this embodiment, for example, the pixel size in the height direction of a person or the area of a circumscribed rectangle is used as a feature related to the person's size. However, in this case, because person 5122 and person 5132 are crouching, their feature values are smaller than those of person 5121 and person 5131, making it difficult to accurately determine their position using the feature values. Therefore, the detection unit 32 calculates the feature values by estimating the posture using posture estimation technology and correcting the posture around a reference point under the assumption that person data generally shows an upright posture. For example, when focusing on a person's lower body, the lower body bends around the knees when the human body crouches. However, the detection unit 32 estimates the positions of joints such as the waist, knees, and ankles using posture estimation technology and integrates the distances between the joints on the image to pseudo-correct the size to that of a predetermined posture (upright posture), i.e., height.

[0066] Furthermore, since there are individual differences in person height, in order to make the detection unit 32 robust to this, the detection unit 32 may perform a process of correcting the reference height of the person using an object of known size that the person is wearing. For example, if there is little variation in size of objects such as a safety belt hook or a helmet depending on the person wearing them, the detection unit 32 may use these objects and the reference points of the person to artificially change the size of the person in the image.

[0067] In an image captured in such a state, for persons 5121, 5122, 5131, 5132, and 514, whose reference point heights are common to each other at height 53 in the image height direction on the image, the feature amounts after the above-described correction are such that the feature amounts of persons 5121 and 5122 located at a high altitude are larger than the feature amounts of persons 5131 and 5132 located at the reference height because these persons are closer to the camera unit 21, and the feature amount of person 514 located at a low altitude is smaller than the feature amounts of persons 5131 and 5132 located at the reference height. In this embodiment, the feature amount of person 5121, who is closer to the camera unit 21, is larger than the feature amount of person 5131, so that the calculation unit 33 determines that person 5121 is located at a high altitude. Specifically, the calculation unit 33 calculates a relationship between the feature amounts of multiple persons located at the reference height, and determines the height position of this person based on the feature amounts of the person whose height position is to be detected.

[0068] Next, the processing in the video analysis system 1 will be described.

[0069] 5 is a flowchart of the relationship calculation process according to Example 1. The relationship calculation process is a process that is executed in advance before starting the operation of the height position determination process that determines the height position of a person.

[0070] The input unit 31 inputs an image (calculation image) prepared in advance for calculating the relationship (S1). The calculation image is an image captured by the camera unit 21 that captures an image for executing the height position determination process, under imaging conditions set so that the size of a person at a position higher than a predetermined reference height appears larger than the size of a person at the reference height in at least a partial area in the height direction of the captured image. The calculation image includes one or more identifiable people at the reference height. Here, being able to identify people at the reference height means, for example, that all people in the image are at the reference height, or that people not at the reference height cannot be recognized as people by masking or the like.

[0071] Next, the detection unit 32 detects a person at the reference height (reference person) from the image (S2), and determines whether or not there is one or more people at the reference height (S3). As a result, if one or more people at the reference height cannot be detected (S3: NO), the detection unit 32 proceeds to step S1.

[0072] On the other hand, if one or more people at the reference height have been detected (S3: YES), the calculation unit 33 performs the processing of loop 1 (S4, S5) for each person as a processing target.

[0073] Specifically, the calculation unit 33 calculates reference points and calculates feature amounts based on the bounding box of the person to be processed (S4). Next, the calculation unit 33 stores information on the calculated reference points and feature amounts (S5).

[0074] Next, if there are any unprocessed people remaining, the calculation unit 33 performs the processing of loop 1 with the unprocessed people as the next processing target, and when all people have been processed, the calculation unit 33 exits the processing of loop 1.

[0075] Next, the calculation unit 33 calculates the relationship based on the calculated feature amount (S6). Here, if multiple determination areas are set in the height direction of the image, the calculation unit 33 calculates the relationship for each determination area.

[0076] Next, the calculation unit 33 stores the calculated relationship in the storage unit 36 (S7), and the process proceeds to step S1.

[0077] Next, the height position determination process will be described.

[0078] FIG. 6 is a flowchart of the height position determination process according to the first embodiment.

[0079] The input unit 31 receives an image (detection image) from the camera unit 21 for accepting a person determination (S11).

[0080] Next, the detection unit 32 detects people from the image (S12) and determines whether or not one or more people are detected (S13). As a result, if one or more people cannot be detected (S13: NO), the detection unit 32 proceeds to step S11.

[0081] On the other hand, if one or more people have been detected (S13: YES), the calculation unit 33 performs the processing of loop 2 (S14, S15) for each person as a processing target.

[0082] Specifically, the calculation unit 33 calculates a reference point based on the bounding box of the person to be processed and calculates a feature amount (S14). Next, the determination unit 34 determines the position of the person based on the feature amount and the relationship (S15). Here, the determination unit 34 calculates a threshold for determining the height position of the person based on the relationship and the threshold condition in the storage unit 36. Furthermore, the determination unit 34 determines the height position of the person based on the feature amount and the threshold. In this embodiment, it determines whether the person is located at a position higher than the reference height or lower than the reference height.

[0083] Next, if there are any unprocessed people remaining, the judgment unit 34 performs the processing of loop 2 with the unprocessed people as the next processing target, and when all people have been processed, the judgment unit 34 exits the processing of loop 2.

[0084] The output unit 35 collects information such as the details of the event determined by the determination unit 34 (details of the person's height position), the time the event occurred, and the monitored area where the event occurred, and transmits it to the monitoring center system 4.

[0085] As a result, the monitoring center system 4 receives information such as the content of the event, the time of the event occurrence, and the monitored area where the event occurred, and displays various types of information.

[0086] Next, the setting of a determination region that is a unit for calculating the relationship in an image will be described.

[0087] Fig. 7 is a diagram showing an example of setting a determination area according to Example 1. Fig. 7 shows an image of the area 5 to be monitored taken by the camera unit 21.

[0088] The image of the monitored area 5 includes a person 515 climbing an object 52, and people 516 and 517 located at a reference height. In this embodiment, the height position of the person is determined using the feature amount of the person. For example, if the reference point is set as the coordinate of the person's feet and the feature amount is the number of pixels in the person's height direction, the feature amount of person 515 is greater than the feature amount of person 516, and this is used to determine that person 515 is located at a high altitude. In this image, people 515 and 516 are located in relatively close areas in the height direction of the image, and in this case, the feature amount of the person located at a higher height is guaranteed to be greater. On the other hand, depending on the shooting conditions of the camera unit 21 (such as the height and depression angle of the camera unit), if a person such as person 517 exists whose reference point is located in a lower area in the height direction of the image, this person 517 may be located closer to the camera unit 21 than person 515, and as a result, the feature amount of person 517 may be greater than the feature amount of person 515. In such a case, it is no longer guaranteed that the feature amount of the person located at a higher height is greater.

[0089] Therefore, in this embodiment, the image is divided into a plurality of regions in the height direction (determination regions: regions 541, 542, 543) and the relationship between the feature amounts is calculated for each of the determination regions, so that the height position of a person can be appropriately determined based on the feature amounts even when the monitored area 5 extends in the perspective direction of the camera unit 21. Note that, depending on the shooting conditions of the camera unit 21, if the height position of a person can be appropriately determined based on the feature amounts without dividing the image into a plurality of regions, it is not necessary to divide the image into a plurality of determination regions.

[0090] Next, the relationship calculated by the calculation unit 33 will be described.

[0091] Fig. 8 is a diagram illustrating a relationship according to Example 1. The relationship shown in Fig. 8 is a frequency distribution in which the horizontal axis indicates the class of feature amounts of a person positioned at a reference height, and the vertical axis indicates the frequency of the feature amounts. Note that, although one-dimensional feature amounts are used in the example of Fig. 8, feature amounts having two or more dimensions may also be used.

[0092] In this embodiment, the relationship shown in FIG. 8 is calculated for each of the set regions set in the monitoring target area.

[0093] An area 83 indicates the frequency in a certain class. The number of classes in the relationship can be set arbitrarily by the user.

[0094] In this embodiment, once the relationship is calculated, the determination unit 34 calculates thresholds 821 and 822 for determining the height position of a person based on the calculated relationship and the threshold conditions stored in the storage unit 36. Here, the threshold 821 is a threshold for determining that a person is at a height lower than the reference height, and the threshold 822 is a threshold for determining that a person is at a height higher than the reference height. Note that the thresholds may be directly specified by the user.

[0095] For example, if a person's feature amount belongs to region 811 that is lower than threshold 821, it indicates that the person is at a low altitude, and if it belongs to region 812 that is higher than threshold 822, it indicates that the person is at a high altitude. Note that, in order to set region 811 and region 812 in more detail, the discrete frequency distribution may be approximated to a curve such as curve 84, and curve 84 may be used as the relationship.

[0096] For example, the control unit 43 of the monitoring center system 4 may select a person who is confirmed by the administrator to be at the new reference height in a new image (additional image) and add the feature amount of this person to the frequency distribution to correct the correlation, or may select a person whose height position is confirmed by the administrator to have been erroneously determined and correct the correlation based on the feature amount of this person. In this way, the correlation can be adaptively corrected according to the operation time in the monitored area, and the occurrence of undetected events and erroneous detection events can be reduced.

[0097] Next, a description will be given of a setting region setting screen, which is an example of a GUI (Graphical User Interface) displayed by the display unit 42 for setting a setting region in an image.

[0098] FIG. 9 is a diagram illustrating an example of a determination region setting screen according to the first embodiment.

[0099] The determination area setting screen 91 includes an image display area 910 , a setting input area 914 , an item deletion button 915 , and a setting save button 916 .

[0100] The setting input area 914 is an area for inputting entries for each of any number of judgment areas to be set. The entry includes fields for selection 914a, area 914b, and vertical range 914c. A selection instruction for the entry is input in selection 914a. Area 914b displays the number of the area corresponding to the entry. Vertical range 914c stores the vertical range of the image of the judgment area corresponding to the entry. In this embodiment, the vertical pixel range of the image of the judgment area is stored. Pre-registered judgment area setting information may be called into the setting input area 914 based on information such as the installation height, depression angle, and lens focal position of the camera unit 21. Alternatively, an optimal judgment area may be calculated based on information such as the installation height, depression angle, and lens focal position of the camera unit 21, and the setting information may be displayed in the setting input area 914. This allows for easy and efficient judgment area setting.

[0101] The image display area 910 is an area that promptly displays on the image the determination areas set in the setting input area 914. In the example of Fig. 9, the image display area 910 displays a state in which determination areas 911, 912, and 913 have been set.

[0102] The delete item button 915 is a button that accepts an instruction to delete the judgment area of an entry selected in the setting input area 914, and when the delete item button 915 is pressed, the control unit 43 deletes the entry of the selected setting area in the setting input area 914. The save setting button 916 is a button for saving the settings input in the setting input area 914, and when the save setting button 916 is pressed, the control unit 43 stores the setting information of the judgment area set in the setting input area 914 in the storage unit 36.

[0103] Note that even if a determination area is set on the determination area setting screen 91, if there is no person in that determination area (a person whose reference point is in the determination area) in the calculation image in the relationship calculation process, the relationship cannot be calculated. For a determination area for which no relationship can be obtained, the height position of the person in the determination area is determined to be high or low in the person height position determination process. Also, for an image area not specified as a determination area on the determination area setting screen 91, for example, an area above the determination area 911 in the image display area 910 in FIG. 9, the relationship is not calculated in the relationship calculation process, but in the height position determination process, if a person is present in that area, the height position of this person is determined to be high.

[0104] Next, a threshold condition setting screen, which is an example of a GUI displayed by the display unit 42 for setting the threshold condition, will be described.

[0105] FIG. 10 is a diagram illustrating an example of a threshold condition setting screen according to the first embodiment.

[0106] The threshold condition setting screen 92 includes a target selection area 921 , a threshold condition input area 922 , an item deletion button 923 , and a setting save button 924 .

[0107] The target selection area 921 is an area for selecting a detection area to be subject to the threshold condition. In the example of Fig. 10, "all areas" is selected, indicating that the threshold condition applies to all detection areas. Note that by selecting a specific detection area in the target selection area 921, it becomes possible to set a threshold condition for that detection area.

[0108] The threshold condition input area 922 is an area for inputting entries for each threshold condition for detecting a predetermined event in the detection area of interest.

[0109] An entry includes fields for selection 922a, setting 922b, display 922c, and range 922d. A selection instruction for the entry is input into selection 922a. Setting 922b displays the number of the threshold condition corresponding to the entry. Display 922c is a field for inputting the name of an event related to the height of a person corresponding to the entry. Range 922d is a field for inputting a range of the relationship for determining that the event corresponds to the entry (in the example of FIG. 10, the range of the frequency distribution corresponding to the relationship). Note that in the example of FIG. 10, a range using the variance of the relationship is input as the threshold condition, but this is not limiting, and for example, a range of a person's feature may be specified.

[0110] In the example of FIG. 10 , the first entry is an entry for a threshold condition for detecting that a person is at a high altitude, where the display 922c is set to “high altitude,” and the range 922d indicates that the range in the relationship is 2.5 sigma or greater. Note that in the case of high altitude, there is no upper limit, so no upper limit is set. Based on this setting and the relationship, the threshold 822 in FIG. 8 is determined. The second entry is an entry for a threshold condition for detecting that a person is at a low altitude, where the display 922c is set to “low altitude,” and the range 922d indicates that the range in the relationship is less than −3.0 sigma. Note that in the case of low altitude, there is no lower limit, so no lower limit is set. Based on this setting and the relationship, the threshold 821 in FIG. 8 is determined. The third entry is an entry for a threshold condition for detecting that a person is not at a high altitude but is at a height that may be high altitude, and the display 922c is set to "Possible high altitude", and the range 922d is set to a range of 2.0 sigma or more and less than 2.5 sigma. Note that the entries to be set may be only entries for some of the events described above, or may be entries for events other than those described above.

[0111] The delete item button 923 is a button that accepts an instruction to delete a threshold condition selected in the threshold condition input area 922, and when the delete item button 923 is pressed, the control unit 43 deletes the entry of the selected threshold condition in the threshold condition input area 922. The save setting button 924 is a button for saving the settings input in the threshold condition input area 922, and when the save setting button 924 is pressed, the control unit 43 stores the threshold condition set in the threshold condition input area 922 in the memory unit 36.

[0112] Next, a search processing screen displayed by the display unit 42 for searching past events stored in the recording unit 41 and displaying the search results will be described.

[0113] FIG. 11 is a diagram illustrating an example of a search processing screen according to the first embodiment.

[0114] The search processing screen 93 includes a search specification area 931 and a search result display area 936 .

[0115] The search specification area 931 is an area for specifying search conditions for searching the history of an event, and includes an event type specification area 932 , a monitored area specification area 933 , an occurrence date and time specification area 934 , and a search button 935 .

[0116] The event type specification area 932 is an area for specifying the type of event to be searched. The monitored area specification area 933 is an area for specifying the monitored area to be searched. The occurrence date and time specification area 934 is an area for specifying the occurrence date and time to be searched. It is not necessary to specify anything in the event type specification area 932, monitored area specification area 933, or occurrence date and time specification area 934. If any of the event type specification area 932, monitored area specification area 933, or occurrence date and time specification area 934 is set, events that meet the specified conditions will be searched for, and if none is specified, information on all recorded events will be searched.

[0117] The search button 935 is a button for executing a search process based on the specified search conditions. When the search button 935 is pressed, the display unit 42 searches the database for information on events that correspond to the search conditions specified in the event type specification area 932, the monitored area specification area 933, and the occurrence date and time specification area 934, and displays the search results in a result display area 936.

[0118] The result display area 936 includes a text information display area 937 that displays information about the event type, monitored area, and date and time of occurrence of the searched event, and an image display area 938 that displays images or videos corresponding to the searched event.

[0119] In the image display area 938, the display unit 42 displays a frame image of the moment the event occurred, a thumbnail of the frame image, and video for a certain period of time including before and after the event. In order to make the person who caused the event distinguishable from other people on the same screen, the display unit 42 may highlight the person in the image display area 938 by performing image processing such as superimposing a person detection frame or cropping the image.

[0120] Furthermore, when multiple events are detected as search results, the display unit 42 may be configured to scroll through and display information about all of the events in the result display area 936. Furthermore, when the relationship can be expressed as a normal distribution as in Fig. 8, the display unit 42 may be configured to display events with feature quantities closer to the tail of the distribution at the top of the result display area 936 or to highlight them.

[0121] In the above example, the search results are displayed on the monitoring center system 4, but the present invention is not limited to this. At least a part of the search processing screen 93 may be displayed on a smartphone terminal, tablet terminal, AR goggles, or the like used by personnel responding at the site, and the search conditions may be accepted and the search results may be displayed. In addition, the display unit 42 may accept designation of an event that does not need to be saved from the user, and delete information about this event from the database.

[0122] As described above, according to this embodiment, the position of a person can be appropriately determined in comparison with the reference height in the image based on the result of detecting the person from the image.

[0123] Furthermore, according to this embodiment, the height position of a person can be determined solely from the person detection results, regardless of the object detection results in the image. This eliminates the need for pre-definition of objects, enabling appropriate person position determination even in environments where pre-definition of objects is difficult. Furthermore, because no object information is used, it is possible to reduce false positive detection of a person as being on an object in an image in which a person who is not actually on the object appears to be on the object. Furthermore, the position determination method in this embodiment is based on anomaly detection, calculating a relationship from feature values for a person at a reference height and determining whether a person is at a high or low altitude based on a feature value that deviates from the relationship. Therefore, feature values for when a person is actually at a high or low altitude are not required for the relationship calculation, eliminating the need to place the person at a dangerous height. Furthermore, in a system that determines whether a worker has a safety hook attached when working at height, performing high altitude determination as a preliminary step in the determination process reduces the possibility of false positive detection of other actions as attaching or detaching a safety hook when the worker is on safe, flat ground. [Example]

[0124] Next, a video analysis system according to a second embodiment will be described.

[0125] The video analysis system according to the second embodiment differs from the video analysis system 1 according to the first embodiment in the method of setting the threshold conditions. The configuration of the video analysis system according to the second embodiment is the same as the configuration of the video analysis system 1, and only the differences from the first embodiment will be described below.

[0126] FIG. 12 is a diagram illustrating an example of a threshold condition setting screen according to the second embodiment.

[0127] The threshold condition setting screen 94 is a screen displayed by the display unit 42, and includes a relationship display area 941 and a threshold operation area 946.

[0128] The relationship display area 941 displays the calculated relationship, and also displays a low altitude determination area 942 that is determined to be a low altitude, a threshold value 943 for determining low altitudes, a high altitude determination area 944 that is determined to be a high altitude, and a threshold value 945 for determining high altitudes.

[0129] The threshold operation area 946 is an area for operating the threshold, and includes a slider 947 for operating the threshold 943 for determining low places, and a slider 948 for operating the threshold 945 for determining high places.

[0130] When the slider 947 is selected and operated, the display unit 42 slides the threshold 943 in the relationship display area 941 in conjunction with the operation of the slider 947, and grasps the value corresponding to the position as the threshold. Furthermore, when the slider 948 is selected and operated, the display unit 42 slides the threshold 945 in the relationship display area 941 in conjunction with the operation of the slider 948, and grasps the value corresponding to the position as the threshold. When a setting button (not shown) is pressed, the display unit 42 notifies the control unit 43 of the values of the threshold 943 and threshold 945 at that time as the thresholds.

[0131] According to this embodiment, the user can intuitively set the threshold value as to what percentage of the desired event should be set while visually referring to the relationship, and can effectively set the range. Note that the threshold condition setting screen 94 in Fig. 12 is a screen on which threshold values for two types of events, low and high, can be set, but if three or more types of events are to be detected, for example, the threshold condition setting screen can be a screen on which threshold values for each event can be set. [Example]

[0132] Next, a video analysis system according to a third embodiment will be described.

[0133] Fig. 13 is a functional configuration diagram of a video analysis system according to Example 3. In Fig. 13, the same components as those in the video analysis system according to Example 1 are denoted by the same reference numerals.

[0134] When a predetermined event such as a person being in a high or low place is determined, it is desirable to immediately notify the user or the person being monitored of the event in order to prevent accidents. Therefore, the video analysis system 1A according to the third embodiment performs a process of calling attention when a predetermined event occurs.

[0135] The video analysis system 1A includes a position determination system 3A instead of the position determination system 3 of the video analysis system 1 of the first embodiment, and a monitoring center system 4A instead of the monitoring center system 4.

[0136] The position determination system 3A includes an input unit 31, a detection unit 32A, a calculation unit 33, a determination unit 34, an output unit 35A, and a storage unit 36A.

[0137] In addition to the information stored in the storage unit 36, the storage unit 36A stores a face image of the worker to be captured in the image and contact information for the worker's receiving device (for example, a smartphone).

[0138] In addition to the functions of the detection unit 32, the detection unit 32A performs processing to identify the person by face recognition based on the detected image of the person and the facial image of the worker in the storage unit 36A. Furthermore, the detection unit 32A determines, based on the image of the person, whether or not the person is wearing a safety belt hook (a fall arrest device) (for example, whether or not the person has performed the action of putting on the safety belt hook and / or whether or not the safety belt hook is being worn).

[0139] In addition to the functions of the output unit 35, the output unit 35A adds contact information of the person who caused the specified event and information on whether the person is wearing a safety belt hook to the position determination result and outputs the result to the monitoring center system 4A.

[0140] The monitoring center system 4A includes a recording unit 41, a display unit 42, a control unit 43, a first notification unit 44, and a second notification unit 45.

[0141] The first notification unit 44 issues a warning (first warning) about the detected event based on the location determination result. The first notification unit 44 warns the monitor by displaying information in real time on the display unit 42. The first notification unit 44 also uses the contact information of the person who caused the event, which is included in the location determination result, to warn the person's receiving device. This allows the monitor and the person who caused the event to be immediately warned, thereby improving the accident prevention effect.

[0142] If, based on the position determination result, the person (worker) who caused the incident at a high altitude is not wearing a safety harness hook, the second notification unit 45 uses the contact information of the person who is not wearing a safety harness hook to issue a warning (second warning) to the person urging them to wear a safety harness hook. This makes it possible to issue an appropriate warning to those people who are determined to be at a high altitude and who truly require a warning, as they are at risk of falling. Note that the monitoring center system 4A may be provided with only one of the first notification unit 44 and the second notification unit 45.

[0143] The present invention is not limited to the above-described embodiment, and can be modified appropriately without departing from the spirit of the present invention.

[0144] For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the configurations described. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment, or the configuration of one embodiment may be added to the configuration of another embodiment.

[0145] In addition, in the above embodiments, some or all of the processing performed by the processor may be performed by a hardware circuit. Also, the program in the above embodiments may be installed from a program source. The program source may be a program distribution server or a portable recording medium such as an IC card, an SD card, or a DVD. [Explanation of symbols]

[0146] 1,1A...Video analysis system, 2...Photographing system, 3,3A...Location determination system, 4,4A...Monitoring center system, 21...Camera unit, 31...Input unit, 32...Detection unit, 33...Calculation unit, 34...Determination unit, 35...Output unit, 36...Memory unit, 41...Recording unit, 42...Display unit, 43...Control unit, 44...First notification unit, 45...Second notification unit, 71...Camera unit, 72,73...Computer, 721,731...CPU, 722,732...RAM, 723,733...HDD

Claims

1. An image analysis system that detects a height position of a person in a monitoring area using an image of the monitoring area, at least one computing device; The computing device receiving one or more calculation images that are images of the area to be monitored that have been captured under imaging conditions that are set so that a person at a position higher than a predetermined reference height appears larger than a person at the reference height, and that can identify a person at the reference height; detecting areas of a plurality of reference persons at the reference height from the one or more calculation images; calculating a feature amount relating to the size of the reference person based on the regions of the plurality of reference persons; Based on the feature amount, a relationship is calculated regarding the size of the reference person in an image of the monitoring target area under the imaging conditions. Image analysis system.

2. the image analysis system includes at least one storage device; The computing device calculating a threshold value for determining that a person is at a predetermined height position different from the reference height based on the relationship, and storing the calculated threshold value in the storage device; receiving a detection image, which is an image captured under the imaging conditions in the area to be monitored and is a target for detecting a height position of a person in the image; Detecting a region of a target person from the detection image; Calculating a feature amount related to the size of the target person based on the area of the target person; Identifying the height position of the target person based on the calculated feature amount and the threshold value. The image analysis system according to claim 1 .

3. The relationship is a frequency distribution of the feature amounts of a plurality of reference persons. The image analysis system according to claim 1 .

4. The computing device When detecting a region of a reference person from the calculation image and when detecting a region of a target person from the detection image, a degree of certainty that the region is a person is calculated; Based on the degree of certainty, it is determined whether or not to use the information for calculating the feature amount of the reference person and / or the feature amount of the target person. The image analysis system according to claim 2 .

5. The computing device Correcting the feature amount of the reference person based on the area of the same reference person detected from the time-series calculation images, and / or correcting the feature amount of the target person based on the area of the same target person detected from the time-series detection images. The image analysis system according to claim 2 .

6. The computing device Estimating the posture of the reference person and calculating the feature amount of the reference person in a predetermined posture, and / or estimating the posture of the target person and calculating the feature amount of the target person in a predetermined posture. The image analysis system according to claim 2 .

7. The computing device Calculating the feature amount of the reference person based on an object of a known size carried by the reference person, and / or calculating the feature amount of the target person based on an object of a known size carried by the target person. The image analysis system according to claim 2 .

8. The computing device The relationship is calculated for each of a plurality of determination regions divided in the height direction of the calculation image. The image analysis system according to claim 1 .

9. The computing device Accept additional calculation images, Detecting an area of a reference person at the reference height from the additional calculation image; calculating a feature amount relating to the size of the reference person based on the region of the reference person; Correcting the relationship based on the feature amount The image analysis system according to claim 1 .

10. The computing device Accepts changes to the threshold from the administrator The image analysis system according to claim 2 .

11. The computing device Detecting an event in which the target person is at a predetermined height; The type of the detected event, the date and time of the event, an image taken at the time the event occurred, and the area to be monitored that is the subject of the image are stored in the storage device in association with each other; Searching for information about the event from the storage device based on at least one of the type of the event, the date and time of the event occurrence, and the area to be monitored where the event occurred, and displaying the search results. The image analysis system according to claim 2 .

12. The computing device Detecting an event in which the target person is at a predetermined height; When an event in which the target person is at a predetermined height is detected, the target person who caused the event is highlighted in the detection image. The image analysis system according to claim 2 .

13. The computing device When an event is detected in which the target person is at a predetermined height, a first warning regarding the height of the target person is issued. The image analysis system according to claim 2 .

14. The computing device It is determined whether the target person who caused the event is wearing a fall arrest device, and if it is determined that the target person is not wearing the fall arrest device, a second warning is issued regarding wearing the fall arrest device. The image analysis system according to claim 13.

15. 1. An image analysis method using an image analysis system that detects a height position of a person in a monitoring area using an image captured of the monitoring area, comprising: The image analysis system includes: receiving one or more calculation images that are images of the area to be monitored that have been captured under imaging conditions that are set so that a person at a position higher than a predetermined reference height appears larger than a person at the reference height, and that can identify a person at the reference height; detecting areas of a plurality of reference persons at the reference height from the one or more calculation images; calculating a feature amount relating to the size of the reference person based on the regions of the plurality of reference persons; Based on the feature amount, a relationship is calculated regarding the size of the reference person in an image of the monitoring target area under the imaging conditions. Image analysis methods.

Images