False detection determination device, method, and program

Abstract

To provide an erroneous detection determination device, a method, and a program for determining whether or not an object other than a person has been erroneously detected.SOLUTION: In an erroneous detection determination system including an erroneous detection determination device and an imaging device that is an infrared camera, an erroneous detection determination device 10 includes: an image acquisition unit that acquires a plurality of images captured by the imaging device; an object detection unit that detects an object like a person in the images; a blown out highlights determination unit that determines presence of blown out highlights in an area where the object has been detected in the image where the object has been detected; a movement determination unit that determines whether or not movement of the object is that of a person when the blown out highlights are present; and an erroneous detection determination unit that determines that the detection of the object is erroneous detection if it is determined that the movement of the object is not that of a person.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an apparatus, method, and program for determining false detection. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a technique for detecting an intruder by using a security or surveillance camera to detect a person who has entered an office, a residence, a vehicle, or the like.

[0003] Patent Document 1 discloses a technique for detecting people in which the shape and size of the bounding box of a human body candidate detected from a fisheye image obtained by a fisheye camera are compared with predetermined shape and size standards. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-107070 Summary of the Invention [Problem to be solved by the invention]

[0005] However, intruders in images captured by security and surveillance cameras often have parts of their body hidden (i.e., only parts of their body can be identified) and may be in various positions (for example, they may not only be standing, but also crouching or crawling). Therefore, when detecting intruders in images captured by security and surveillance cameras, using the shape and size of the bounding box may result in the false detection of objects other than people in the image.

[0006] Furthermore, when the technology in Patent Document 1 is combined with a night-vision camera and used for nighttime surveillance, images of people taken at night resemble whiteout caused by insects or other objects approaching the camera lens, and whiteout caused by insects or other objects may be mistakenly detected as a person.

[0007] Therefore, an object of the present invention is to determine whether an object other than a person has been erroneously detected in a system that detects an intruder from an image captured at night. [Means for solving the problem]

[0008] An erroneous detection determination device according to one embodiment of the present invention comprises an image acquisition unit that acquires a plurality of images, an object detection unit that detects human-like objects within the images, a blown-out highlight determination unit that determines whether or not there is blown-out highlighting in the area in the image in which the object is detected, a movement determination unit that, when blown-out highlighting is present, determines whether the movement of the object is human movement, and an erroneous detection determination unit that, when it is determined that the movement of the object is not human movement, determines that the detection of the object is an erroneous detection. [Effects of the Invention]

[0009] According to the present invention, it is possible to determine whether or not an object other than a person has been erroneously detected. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram for explaining an overview according to one embodiment of the present invention. [Figure 2] 1 is a diagram illustrating an overall system configuration according to an embodiment of the present invention. [Figure 3] 1 is a functional block diagram of an erroneous detection determination device according to an embodiment of the present invention; [Figure 4] 10 is a flowchart of an erroneous detection determination process according to an embodiment of the present invention. [Figure 5] 10 is a flowchart of an insect movement determination process according to an embodiment of the present invention. [Figure 6] 10 is a flowchart of a human movement determination process according to an embodiment of the present invention. [Figure 7] FIG. 10 is a diagram illustrating determination of insect movement according to an embodiment of the present invention. [Figure 8]FIG. 10 is a diagram illustrating a determination of a person's movement according to an embodiment of the present invention. [Figure 9] 1 is a block diagram showing an example of a hardware configuration of an erroneous detection determination device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] This specification describes an embodiment in which an intruder detection system using an infrared camera equipped with infrared lighting (hereinafter simply referred to as an "infrared camera") at night detects overexposure caused by insects as a person, but the present invention can also be applied to overexposure caused by small animals other than insects (any object other than a person will do).

[0013] 1 is a diagram illustrating an overview of one embodiment of the present invention. In one embodiment of the present invention, when a human-like object (specifically, an object with a size and shape that is assumed to be a human) is detected in an image captured by an infrared camera installed in a monitored area, it is possible to determine whether the object has been detected incorrectly (i.e., is a human) or not (i.e., is not a human) depending on whether there is blown-out highlights in the area where the object was detected.

[0014] In step 1 (S1), an image is captured by an infrared camera.

[0015] Assume that in step 2 (S2), a human-like object (specifically, an object with a size and shape that is assumed to be a human) is detected in the image of S1.

[0016] In step 3 (S3), it is determined whether the detection score of S2 (specifically, the detection reliability) is greater or less than a threshold. If it is greater than the threshold, the process proceeds to step 8, where it is determined that the object detected in S2 is a person (i.e., not a false detection). If it is less than the threshold, the process proceeds to step 4.

[0017] In step 4 (S4), it is determined whether or not there is blown-out highlights in the area where the object was detected in the image in S2. If there is no blown-out highlights, the process proceeds to step 6. If there is blown-out highlights, the process proceeds to step 5.

[0018] In step 5 (S5), it is determined whether the object detected in S2 has any insect-like movement (whether it is an insect-like movement or not). If there is any insect-like movement, the process proceeds to S9, and the detection of the human-like object in S2 is determined to be a false detection. If there is no insect-like movement, the process proceeds to step 6.

[0019] In step 6 (S6), it is determined whether the human-like object detected in S2 is moving. If there is movement, the process proceeds to S8, where the object detected in S2 is determined to be a person. If there is no movement, the process proceeds to S7, where the object detected in S2 is determined to be not a person.

[0020] Fig. 2 is a diagram showing the overall system configuration according to one embodiment of the present invention. As shown in Fig. 2, the false detection determination system 1 includes an false detection determination device 10 and an infrared camera imaging device 20. The false detection determination device 10 can acquire images captured by the imaging device 20 via any network or storage medium. Each of these will be described below.

[0021] When an object (specifically, an object of a size and shape that is assumed to be a person) is detected in an image captured by the imaging device 20, the false detection determination device 10 determines whether the object is not a false detection (i.e., it is a person) or whether it is a false detection (i.e., it is not a person). The false detection determination device 10 is a computer such as a server. The false detection determination device 10 will be described in detail later with reference to FIG. 3.

[0022] The imaging device 20 is an infrared camera, such as a security or surveillance camera, that captures images of people who intrude into an office, a residence, a vehicle, or the like. For example, when an intruder detection unit (not shown), such as an infrared sensor, detects an object, the imaging device 20 records images captured within several seconds before and after the detection in an internal memory (not shown) and transmits the images to the false detection determination device 10. The infrared sensor or the like may be installed separately from the imaging device 20, and when the infrared sensor or the like detects an object, the imaging device 20 may be linked to capture an image of the target area. Furthermore, when the infrared sensor or the like transmits a detection signal to the false detection determination device 10, the false detection determination device 10 may request the imaging device 20 to transmit the captured image to itself, depending on the content of the detection signal.

[0023] Fig. 3 is a functional block diagram of an error detection determination device 10 according to one embodiment of the present invention. As shown in Fig. 3, the error detection determination device 10 can include an image acquisition unit 101, an object detection unit 102, a detection score determination unit 103, a blown-out highlight determination unit 104, an insect movement determination unit 105 (having a similarity calculation unit 151 and a determination unit 152), a human movement determination unit 106 (having a similarity calculation unit 161 and a determination unit 162), and an error detection determination unit 107. The insect movement determination unit 105 and the human movement determination unit 106 are also collectively referred to as a movement determination unit. In addition, by executing a program, the false detection determination device 10 can function as an image acquisition unit 101, an object detection unit 102, a detection score determination unit 103, a blown-out highlight determination unit 104, an insect movement determination unit 105 (having a similarity calculation unit 151 and a determination unit 152), a human movement determination unit 106 (having a similarity calculation unit 161 and a determination unit 162), and a false detection determination unit 107.

[0024] The image acquiring unit 101 acquires a plurality of images (i.e., a video consisting of a plurality of frames (still images)) captured by the imaging device 20. Alternatively, the image acquiring unit 101 may receive a detection signal from the aforementioned infrared sensor or the like, request the imaging device 20 to transmit the captured images to itself in accordance with the content of the detection signal, and acquire the plurality of images transmitted by the imaging device 20 in response to the request.

[0025] Object detection unit 102 detects an object (specifically, a body part or the entire body that is assumed to be a person) in each of the multiple images (i.e., each frame) acquired by image acquisition unit 101. Object detection unit 102 stores coordinate information of an area (e.g., a rectangle) in which an object is detected in the image, and a detection score (detection reliability) in a memory or the like (not shown) in error detection determination device 10. Hereinafter, an image (frame) in which object detection unit 102 detects an object will be referred to as an object detection image, and an area in which an object is detected in an object detection image will be referred to as an object detection area.

[0026] For example, the object detection unit 102 can detect objects in an image using machine learning such as deep learning. Specifically, the object detection unit 102 inputs an image into a trained model that has previously trained images of entire and partial human bodies using a neural network or the like, detects any objects in the image that are assumed to be humans, and outputs the object detection area in the image and a detection score (detection reliability). Here, "human and object detection using anchor boxes" is shown as an example. For example, object detection methods such as "Single Shot Multibox Detector (SSD)," "M2Det," and "Faster R-CNN" apply anchor boxes with multiple aspect ratios to each cell of a feature map obtained by inputting an image, obtaining candidates for "human rectangles" and "human detection scores." Then, non-maximum suppression (NMS) is applied, and if the overlap of the "human rectangles" is above a certain level, only rectangles with high "human detection scores" are retained and output. There are various methods for detecting people and objects, but any method can be used as long as it outputs a "human rectangle (object detection area)" and a "human detection score."

[0027] Note that an object may be detected in one or more images (frames).

[0028] The detection score determination unit 103 determines whether the score of the detection by the object detection unit 102 (detection reliability (indicating the accuracy of prediction / output by machine learning)) is smaller than a threshold value.

[0029] The blown-out highlight determination unit 104 determines whether or not there is blown-out highlight in an image in which an object has been detected. Specifically, the blown-out highlight determination unit 104 calculates the percentage of pixels in an area (e.g., a rectangle) containing the object, whose pixel values ​​of brightness are equal to or greater than a predetermined value (e.g., 250). The blown-out highlight determination unit 104 determines that there is blown-out highlight if the percentage of the number of pixels whose pixel values ​​of brightness are equal to or greater than a predetermined value (e.g., 250) to the total number of pixels (number of pixels in width x number of pixels in height) is equal to or greater than a predetermined numerical value (e.g., 20 percent).

[0030] The insect movement determination unit 105 determines whether or not there is movement of the object detected by the object detection unit 102 (whether or not it is movement of an object other than a human, such as an insect). Specifically, the insect movement determination unit 105 determines whether or not there is movement of the object based on whether or not there is similarity between the object detection image (the frame at time t in which a human-like object is detected) and the images before and after it (the frame at time t-1 and the frame at time t+1). Below, the similarity calculation unit 151 and the determination unit 152 of the insect movement determination unit 105 will be explained separately.

[0031] Similarity calculation unit 151 calculates the similarity between the object detection image (a frame at time t in which a human-like object is detected) and the images before and after it (a frame at time t-1 and a frame at time t+1). Specifically, similarity calculation unit 151 compares an area (e.g., a rectangle) in the object detection image that includes the object with areas (e.g., rectangles) in the images before and after that have the same coordinate position as the area in the object detection image.

[0032] In addition, when a human-like object is detected in multiple images (frames), the similarity calculation unit 151 calculates the similarity between each object detection image and the images before and after it (i.e., the frames before and after it included in the video).

[0033] <<Calculating similarity>> Here, the calculation of similarity will be described in detail. For example, the similarity calculation unit 151 can use the ZNCC (Zero-mean Normalized Cross Correlation) method. For example, suppose that the similarity between a rectangle of M pixels by N pixels including an object and a rectangle of M pixels by N pixels in the previous and next images is calculated. The pixels (assumed to be T(i,j) and I(i,j)) at the same coordinates of the two rectangles are used. The similarity calculation unit 151 calculates the similarity by:

[0034]

number

[0035]

number

[0036]

number

[0037]

number

[0038] The similarity is not limited to the ZNCC method, but can be calculated by any template matching method such as NCC (Normalized Cross-Correlation), SSD (Sum of Squared Difference), or SAD (Sum of Absolute Difference).

[0039] For each object detection region in an object detection image in which the object detection unit 102 detected an object, the determination unit 152 calculates the similarity between regions at the same coordinate position as the object detection region in the previous and next images and determines whether at least one of the similarities is below a threshold. Specifically, the determination unit 152 determines that there is object movement if the similarity between at least one of the previous and next images for at least one object detection image is below a threshold. The determination unit 152 determines that there is no object movement if the similarity between all of the previous and next images is not below the threshold. This utilizes the characteristics of a situation in which an insect is photographed close to the camera. The interval between each of the approximately 15 images photographed within a few seconds is approximately 0.2 seconds. If an object assumed to be a person in the image data is actually a person, comparing adjacent images before and after the object detection region will result in a high similarity. However, in the case of a small organism such as an insect photographed directly in front of the camera lens, the movement will appear larger, resulting in a low image similarity. Using this, the image is compared with the immediately preceding image and the immediately following image, and if the similarity of either one is low, it can be determined that the movement is due to something other than a human, such as an insect.

[0040] The human movement determination unit 106 determines whether or not there is movement (whether or not it is human movement) of the human-like object detected by the object detection unit 102. Note that, in the following, an embodiment will be described in which the presence or absence of object movement is determined based on whether or not the object detection image (i.e., the frame in which the object is detected) is similar to other images (i.e., other frames included in the video), but this is not limiting, and the movement of a human-like object may be determined by any method. Below, the similarity calculation unit 161 and the determination unit 162 of the human movement determination unit 106 will be described separately.

[0041] Similarity calculation unit 161 calculates the similarity between the object detection image (a frame in which a human-like object is detected) and another image (i.e., another frame included in the video). Specifically, similarity calculation unit 161 compares an area in the object detection image that includes an object (object detection area) with an area in the other image that has the same coordinate position as the object detection area in the object detection image. Note that the other frames may be all other frames included in the video, or may be some other frames included in the video.

[0042] In addition, when a human-like object is detected in multiple images (frames), the similarity calculation unit 161 calculates the similarity between each object detection image (each frame) and each other image (i.e., other frames included in the video).

[0043] <<Calculating similarity>> Here, the calculation of similarity will be described in detail. For example, the similarity calculation unit 161 can use the ZNCC (Zero-mean Normalized Cross Correlation) method. For example, suppose that the similarity between an M pixel x N pixel rectangle containing an object and an M pixel x N pixel rectangle in another image is calculated. The pixels at the same coordinates of the two rectangles (assumed to be T(i,j) and I(i,j)) are used. The similarity calculation unit 161 calculates the similarity by:

[0044]

number

[0045]

number

[0046]

number

[0047]

number

[0048] The similarity is not limited to the ZNCC method, but can be calculated by any template matching method such as NCC (Normalized Cross-Correlation), SSD (Sum of Squared Difference), or SAD (Sum of Absolute Difference).

[0049] For each object detection region in an object detection image in which object detection unit 102 has detected an object, determination unit 162 determines whether or not there are a predetermined number of other images whose similarity is equal to or less than the threshold. Specifically, determination unit 162 determines that there is object movement if the number of other images whose similarity is equal to or less than the threshold is equal to or greater than the predetermined number in at least one object detection region. Determination unit 162 determines that there is no object movement if the number of other images whose similarity is equal to or less than the threshold is less than the predetermined number in all object detection regions. Note that the predetermined number may be one or two or more. Furthermore, if there are multiple object detection regions in the same object detection image, these determination processes are performed for each object detection region, and if the number of other images whose similarity is equal to or less than the threshold is equal to or greater than the predetermined number in at least one object detection region, it can be determined that there is object movement.

[0050] The erroneous detection determination unit 107 determines that the object detected by the object detection unit 102 is a person because it is moving, or determines that the object detected by the object detection unit 102 is not a person because it is not moving.

[0051] <Method> Hereinafter, the erroneous detection determination process will be described with reference to FIG. 4, the insect movement determination process will be described with reference to FIG. 5, and the human-like object movement determination process will be described with reference to FIG.

[0052] FIG. 4 is a flowchart of an erroneous detection determination process according to one embodiment of the present invention.

[0053] In step 11 (S11), the image acquisition unit 101 acquires a plurality of images captured by the imaging device 20 (that is, a moving image made up of a plurality of frames (still images)).

[0054] In step 12 (S12), the object detection unit 102 detects an object that is assumed to be a person (human-like) in each of the multiple images (i.e., each frame) acquired in S11. Note that an object may be detected in one or more images (frames). If a human-like object is detected, the process proceeds to step 13; if a human-like object is not detected, the process ends.

[0055] In step 13 (S13), the detection score determination unit 103 determines whether the detection score (detection reliability) of S12 is smaller than a threshold value. If it is determined that it is less than the threshold value, the process proceeds to step 14, and if it is determined that it is equal to or greater than the threshold value, it is determined that it is a person.

[0056] In step 14 (S14), the blown-out highlight determination unit 104 determines whether or not there is blown-out highlight in the area where the object was detected in S12. If there is blown-out highlight, the process proceeds to step 15, and if there is no blown-out highlight, the process proceeds to step 16.

[0057] In step 15 (S15), the insect movement determination unit 105 determines whether or not there is movement of the object detected in S12 (whether or not it is movement that is typical of an insect, etc.). If it is determined that there is no movement that is typical of an insect, etc., the process proceeds to step 16, and if it is determined that there is movement that is typical of an insect, etc., the process proceeds to step 17.

[0058] In step 16 (S16), the human movement determination unit 106 determines whether or not the human-like object detected in S12 is moving (whether or not the human-like object is moving). If it is determined that there is no movement, the process proceeds to step 17, and if it is determined that there is movement, the process proceeds to step 18.

[0059] In step 17 (S17), the erroneous detection determination unit 107 determines that the object detected in S12 is an erroneous detection (that is, the object detected in S12 is not a person).

[0060] In step 18 (S18), the erroneous detection determination unit 107 determines that the object detected in S12 is not an erroneous detection (that is, the object detected in S12 is a person).

[0061] FIG. 5 is a flowchart of the insect movement determination process (S15 in FIG. 4) according to one embodiment of the present invention.

[0062] In step 21 (S21), similarity calculation section 151 selects one of the object detection regions in the object detection image in which object detection section 102 has detected an object.

[0063] In step 22 (S22), the similarity calculation unit 151 calculates the similarity between the object detection area in the object detection image selected in S21 (i.e., the frame in which the object was detected) and the area at the same coordinate position as the object detection area in the previous and next images (i.e., the previous and next frames included in the video).

[0064] In step 23 (S23), similarity calculation unit 151 determines whether or not calculation of similarities has been completed for object detection regions in all images (object detection images) in which objects have been detected by object detection unit 102. If so, proceed to step 24; if not, return to step 21.

[0065] In step 24 (S24), for each object detection region in an image (object detection image) in which object detection unit 102 detected an object, determination unit 152 calculates the similarity between regions in the previous and next images at the same coordinate position as the object detection region, and determines whether at least one of the similarities is equal to or less than a threshold. If the similarity of at least one of the previous and next images for at least one object detection image is equal to or less than the threshold, the process proceeds to step 25; if the similarity of none of the previous and next images for all object detection images is equal to or less than the threshold, the process proceeds to step 26.

[0066] In step 25 (S25), the determination unit 152 determines that there is movement of the insect.

[0067] In step 26 (S26), the determining unit 152 determines that the insect is not moving.

[0068] FIG. 6 is a flowchart of the human movement determination process (S16 in FIG. 4) according to one embodiment of the present invention.

[0069] In step 31 (S31), similarity calculation section 161 selects one of the object detection regions in the object detection image in which object detection section 102 has detected an object.

[0070] In step 32 (S32), the similarity calculation unit 161 calculates the similarity between the object detection area in the object detection image selected in S31 (i.e., the frame in which the object was detected) and an area at the same coordinate position as the object detection area in another image (i.e., another frame included in the video).

[0071] In step 33 (S33), similarity calculation unit 161 determines whether or not calculation of similarities has been completed for object detection regions in all images (object detection images) in which objects have been detected by object detection unit 102. If so, proceed to step 34; if not, return to step 31.

[0072] In step 34 (S34), determination unit 162 determines whether or not there are a predetermined number or more of other images whose similarity is equal to or less than the threshold for each object detection region in which object detection unit 102 has detected an object. If the number of other images whose similarity is equal to or less than the threshold is equal to or greater than the predetermined number in at least one object detection region, the process proceeds to step 35, and if the number of other images whose similarity is equal to or less than the threshold is less than the predetermined number in all object detection regions, the process proceeds to step 36.

[0073] In step 35 (S35), the determination unit 162 determines that there is human-like object movement.

[0074] In step 36 (S36), the determination unit 162 determines that there is no human-like object movement.

[0075] <Determining insect movement> The determination of insect movement will now be described in detail.

[0076] 7 is a diagram illustrating the determination of insect movement according to one embodiment of the present invention. Assume that a video captured by a security or surveillance camera (i.e., a video acquired by the image acquisition unit 101) is composed of 15 frames in time series from the first frame to the fifteenth frame. Assume that an object is detected in the tenth frame (i.e., the object detection unit 102 detects an object in the tenth frame). Assume that the detection score (detection reliability) is 0.5, which is less than the threshold.

[0077] First, the similarity calculation unit 151 of the insect movement determination unit 105 calculates the similarity between the frame in which the object is detected (the 10th frame in FIG. 7) and the previous and next frames included in the video (the 9th and 11th frames in FIG. 7).

[0078] Next, for each object detection region in the object detection image, determination unit 152 of insect movement determination unit 105 calculates the similarity between regions at the same coordinate position as the object region in the previous and next images, and determines whether at least one of the similarities is equal to or less than a threshold. In the example of Figure 7, it is assumed that the similarity of the object detection region in the 11th frame is equal to or less than the threshold. In this case, insect movement determination unit 105 determines that there is insect movement because the similarity of at least one of the previous and next images is equal to or less than the threshold.

[0079] If an object is detected in multiple images (frames) (i.e., if an object is detected in a frame other than the 10th frame), similarity calculation unit 151 calculates the similarity between the object detection region in each object detection image (each frame) and the region at the same coordinate position as the object detection region in each of the previous and next images (i.e., the previous and next frames included in the video). Then, for each image in which an object is detected (object detection image), determination unit 152 determines whether or not at least one of the previous and next images has a similarity below a threshold (i.e., a determination similar to the determination for the 10 frames described above is made). If at least one of the previous and next images has a similarity below the threshold in at least one object detection image, it is determined that there is movement, and if none of the previous and next images has a similarity below the threshold in all object detection images, it is determined that there is no movement.

[0080] <Detecting human movement> The determination of human movement will now be described in detail.

[0081] 8 is a diagram for explaining human movement determination according to one embodiment of the present invention. Assume that a video captured by an imaging device 20 such as a security or surveillance camera and acquired by an image acquisition unit 101 is composed of 15 frames in time series from frame 1 to frame 15. Assume that the object detection unit 102 detects a human-like object within frame 10. The detection score (detection reliability) is 0.5, which is less than a threshold.

[0082] First, similarity calculation unit 161 of human movement determination unit 106 calculates the similarity between the object detection area in which the object is detected in the image in which the object is detected (the 10th frame in Figure 8) and areas at the same coordinate position as the object detection area in which the aforementioned object is detected in other frames included in the video (frames 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, and 15 in Figure 8).

[0083] Next, the determination unit 162 of the human movement determination unit 106 counts the number of similarities calculated by the similarity calculation unit 161 that are equal to or less than a threshold, and determines whether or not this number is equal to or greater than a predetermined number (e.g., 8). In the example of FIG. 8, the threshold for similarity is set to 0.7, and there are 11 frames whose similarity is equal to or less than the threshold (marked with a circle in FIG. 8). In this case, the human movement determination unit 106 determines that the human-like object is moving because the number of other images whose similarity is equal to or less than the threshold is equal to or greater than a predetermined number.

[0084] If an object is detected in multiple images (frames) (i.e., if an object is detected in a frame other than the tenth frame), similarity calculation unit 161 calculates the similarity between each object detection region in an image (object detection image) in which the object is detected and regions in other images (i.e., other frames included in the video) that have the same coordinate position as the object detection region. Then, determination unit 162 determines, for each object detection region in which the object is detected, whether there are a predetermined number of regions with the same coordinate position as the object detection region in other images whose similarity is less than a threshold (i.e., a determination similar to the determination for the tenth frame described above). If the number of regions with the same coordinate position as the object detection region in other images whose similarity is less than the threshold is equal to or greater than a predetermined number in at least one object detection region, it is determined that there is movement. If the number of similarities less than the threshold in all object detection images is less than a predetermined number, it is determined that there is no movement. Similarly, if there are multiple object detection areas within the same image, the similarity between each object detection area and other images is calculated, the number of other images (areas) whose similarity is below a threshold is counted, and movement is determined.

[0085] Here, we will explain the determination based on the condition that there are multiple images with similarity below a threshold. When comparing the object detection region of an image in which a human-like object has been detected with other images, the similarity may be calculated to be low if there happens to be an insect or ambient light such as a car light reflected in the object. In this case, if the method calculates and compares the similarity only with the region of the same position range in the previous and next images, the insect or ambient light may be erroneously detected as a human. By comparing not only the previous and next images but also multiple other images and setting the determination condition to include the number of images (regions) with similarity below a threshold, it is possible to suppress the occurrence of erroneous detection due to noise such as insects or ambient light, and to accurately determine that the object is a moving human body.

[0086] <Other embodiments> Other embodiments will be described below.

[0087] <<Detection of non-human movements such as insects>> The above describes an embodiment in which the presence or absence of object movement (whether it is the movement of an object other than a human, such as an insect) is determined based on whether an object detection image is similar to previous and subsequent images. However, the present invention can also be applied to an embodiment in which the presence or absence of object movement is determined based on the characteristics of the object movement. For example, the insect movement determination unit 105 can determine that the movement is of a fast-moving insect or the like when the change in the proportion of white pixels between the object detection area in the object detection image and the area at the same position as the object detection area in the previous and subsequent images is equal to or greater than a threshold. Furthermore, for example, the insect movement determination unit 105 can determine that the movement is of a fast-moving insect or the like when the change in the size of the area (e.g., a rectangle) containing the object between the object detection area in the object detection image and each object detection area in the previous and subsequent images is equal to or greater than a threshold. In this case, if there is no object detection area in the previous or subsequent image, object tracking technology is used to specify a rectangular area that identifies the object's position in the previous or subsequent image to make the determination. Furthermore, for example, if the change in the movement direction of an area (e.g., a rectangle) containing an object between the object detection area in the object detection image and each object detection area in the previous or next image is equal to or greater than a threshold, the insect movement determination unit 105 can determine that the movement is that of a fast-moving insect, etc. In this case, if there is no object detection area in the previous or next image, an object tracking technique is used to specify a rectangular area that identifies the object position in the previous or next image and make the determination.

[0088] <<Movement determination for each body part>> In one embodiment of the present invention, the false detection determination device 10 can detect parts of a person's body (for example, not limited to the whole body, but also parts such as the head, arms, upper body, lower body, left and right halves of the body, etc.) and determine the movement of the detected parts of the person's body.

[0089] Specifically, the object detection unit 102 detects human body parts using machine learning or the like. The detection score determination unit 103 determines whether the detection score (detection reliability) is smaller than a threshold. The human movement determination unit 106 determines whether or not the human body parts are moving. In this case, a threshold for the detection score (detection reliability) and a threshold for determining human-like object movement may be set for each human body part.

[0090] For example, when learning the shapes of heads and arms, a hat or helmet placed indoors may be mistakenly detected as a "head," and long, thin objects such as chairs, desk frames, and pillars as "arms." To prevent these false detections, if a condition is set to uniformly determine a person (moving object) when the similarity is below a threshold in a larger number of images, there is a risk of missing a person. Therefore, for object detection regions detected as arms, legs, and other parts of the body that tend to move significantly in typical human movements, the system determines that the object is a person when the similarity is below a threshold in a larger number of images than for object detection regions detected as parts of the body that move relatively little, such as the head. This enables accurate detection of human objects.

[0091] <<Motion detection based on brightness>> The above describes an embodiment in which whether or not the object detection areas of two images (frames) are similar is determined by template matching such as ZNCC, but the present invention can also be applied to an embodiment in which whether or not two images (frames) are similar is determined by an evaluation value based on the brightness of the object detection areas of the two images (frames) (e.g., the average brightness difference).

[0092] Furthermore, multiple images obtained by comparing the brightness differences between pixels when comparing object detection regions can be trained by an AI model that classifies moving images, such as 3DCNN (an AI that inputs a series of multiple images (moving images) and outputs their contents), and the model can output which movement pattern the image corresponds to. Note that the trained model may be a model trained based on the similarity between an image (e.g., the first frame) other than the image in which the object detection unit 102 detected the object (e.g., the tenth frame in the example of FIGS. 7 and 8) and other images. Alternatively, the trained model may be a model trained based on the similarity between images of adjacent frames (e.g., the similarity between the images of the first and second frames, the similarity between the images of the second and third frames, etc.).

[0093] <<Exclusion of Cyclical Movements>> In one embodiment of the present invention, the human movement determination unit 106 can further determine whether the object movement is periodic. If the object movement is periodic, the false detection determination unit 107 determines that the object detection is a false detection (for example, a light, flag, curtain swaying in the wind, or a flashing car headlight, etc., other than a human).

[0094] Specifically, the human movement determination unit 106 can input the similarity between the object detection region in which the object detection unit 102 detected an object and a region in another image at the same coordinate position as the object detection region (in the example of FIG. 8, the similarity between the 10th frame and frames 1 to 9 and 11 to 15) into a trained model using a method such as a support vector machine or a random forest, and output whether or not the movement is periodic. Note that the trained model may be a model trained based on the similarity between an image (e.g., the 1st frame) other than the image in which the object detection unit 102 detected an object (in the example of FIG. 8, the 10th frame) and another image. Alternatively, the trained model may be a model trained based on the similarity between images of adjacent frames (e.g., the similarity between the images of the 1st and 2nd frames, the similarity between the images of the 2nd and 3rd frames, etc.).

[0095] <<Learning human-like movements>> The trained model may have previously learned patterns of human-like movements (such as a person moving or standing still). The human movement determination unit 106 may output which movement pattern corresponds to which of multiple classes from multiple frames.

[0096] <Effects> In this way, in one embodiment of the present invention, it is possible to determine whether an object detected in an image captured by a security or surveillance camera or the like is a false positive (i.e., a person) or a false positive (i.e., not a person). Furthermore, in one embodiment of the present invention, if there is overexposure and it is determined to be insect movement (i.e., fast-moving movement), it is possible to reduce false positive detections of intruders by excluding it from the detection target.

[0097] <Hardware configuration> 9 is a block diagram showing an example of the hardware configuration of an error detection determination device 10 according to one embodiment of the present invention. The error detection determination device 10 has a CPU (Central Processing Unit) 1001, a ROM (Read Only Memory) 1002, and a RAM (Random Access Memory) 1003. The CPU 1001, ROM 1002, and RAM 1003 form a so-called computer.

[0098] The false detection determination device 10 may also have an auxiliary storage device 1004, a display device 1005, an operation device 1006, an I / F (Interface) device 1007, and a drive device 1008. The hardware components of the false detection determination device 10 are connected to each other via a bus B.

[0099] The CPU 1001 is a computing device that executes various programs installed in the auxiliary storage device 1004 .

[0100] The ROM 1002 is a non-volatile memory. The ROM 1002 functions as a main storage device that stores various programs, data, etc. required for the CPU 1001 to execute various programs installed in the auxiliary storage device 1004. Specifically, the ROM 1002 functions as a main storage device that stores boot programs such as a BIOS (Basic Input / Output System) and an EFI (Extensible Firmware Interface).

[0101] The RAM 1003 is a volatile memory such as a dynamic random access memory (DRAM) or a static random access memory (SRAM). The RAM 1003 functions as a main storage device that provides a working area in which various programs installed in the auxiliary storage device 1004 are expanded when the CPU 1001 executes them.

[0102] The auxiliary storage device 1004 is an auxiliary storage device that stores various programs and information used when the various programs are executed.

[0103] The display device 1005 is a display device that displays the internal state of the error detection determination device 10 and the like.

[0104] The operation device 1006 is an input device through which the administrator of the false detection determination device 10 inputs various instructions to the false detection determination device 10 .

[0105] The I / F device 1007 is a communication device that connects to a network and communicates with other devices.

[0106] Drive device 1008 is a device for loading storage medium 1009. The storage medium 1009 here includes media that record information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, and magneto-optical disks. Storage medium 1009 may also include semiconductor memories that record information electrically, such as EPROMs (Erasable Programmable Read Only Memory) and flash memories.

[0107] The various programs to be installed in the auxiliary storage device 1004 are installed, for example, by setting the distributed storage medium 1009 in the drive device 1008 and reading out the various programs recorded on the storage medium 1009 by the drive device 1008. Alternatively, the various programs to be installed in the auxiliary storage device 1004 may be installed by being downloaded from a network via the I / F device 1007.

[0108] Although the examples of the present invention have been described in detail above, the present invention is not limited to the specific embodiments described above, and various modifications and variations are possible within the scope of the gist of the present invention as set forth in the claims. [Explanation of symbols]

[0109] 1. False positive detection system 10. False detection determination device 20 Imaging device 101 Image acquisition unit 102 Object detection unit 103 Detection score determination unit 104 Overexposure judgment section 105 Insect movement detection unit 106 Human movement detection unit 107 False detection judgment unit 151 Similarity calculation part 152 Judgment section 161 Similarity calculation part 162 Judgment section 1001 CPU 1002 ROM 1003 RAM 1004 Auxiliary storage device 1005 Display device 1006 Operating device 1007 I / F device 1008 Drive device 1009 Storage medium

Claims

1. An erroneous detection determination device provided in an object detection device that acquires a plurality of images and detects an object from the plurality of images, comprising: a blown-out highlight determination unit that determines whether or not there is blown-out highlights whose brightness exceeds a predetermined brightness in an object detection image, which is an image in which an object is detected, among the plurality of images; If it is determined that there is blown-out highlights, it is determined that the detection of the object is an erroneous detection based on the correlation or similarity between the object detection image and each of the images immediately before and after the object detection image among the plurality of images; If it is determined that there is no overexposure, it is determined that the detection of the object is a false detection based on the correlation or similarity between the object detection image and the plurality of images excluding the object detection image. An erroneous detection determination device comprising:

2. When it is determined that there is overexposure, if there is no correlation or similarity between the object detection image and either one of the images immediately before and after the object detection image among the plurality of images, it is determined that the detection of the object is an erroneous detection; When it is determined that there is no overexposure, if the number of images that are not correlated with or similar to the object detection image among the plurality of images excluding the object detection image is less than a predetermined number, it is determined that the detection of the object is a false detection.

2. The erroneous detection determination device according to claim 1.

3. If it is determined that there is overexposure, the detection of the object is determined to be an erroneous detection based on the correlation or similarity between an object detection region in which the object is detected in the object detection image and regions at the same coordinate position as the object detection region in images immediately before and after the object detection image; If it is determined that there is no overexposure, it is determined that the detection of the object is an erroneous detection based on the correlation or similarity between the object detection area and an area at the same coordinate position as the object detection area in an image other than the object detection image among the plurality of images.

2. The erroneous detection determination device according to claim 1.

4. A method executed by an erroneous detection determination device provided in an object detection device that acquires a plurality of images and detects an object from the plurality of images, comprising: determining whether or not there is blown-out highlights in an object detection image, which is an image in which an object has been detected, among the plurality of images, where the brightness exceeds a predetermined brightness; If it is determined that there is blown-out highlights, it is determined that the detection of the object is an erroneous detection based on the correlation or similarity between the object detection image and each of the images immediately before and after the object detection image among the plurality of images; If it is determined that there is no blown-out highlights, the method determines that the detection of the object is a false detection based on the correlation or similarity between the object detection image and the plurality of images excluding the object detection image.

5. An object detection device that acquires a plurality of images and detects an object from the plurality of images includes an erroneous detection determination device, determining whether or not there is blown-out highlights in an object detection image, which is an image in which an object has been detected, among the plurality of images, where the brightness exceeds a predetermined brightness; If it is determined that there is blown-out highlights, it is determined that the detection of the object is an erroneous detection based on the correlation or similarity between the object detection image and each of the images immediately before and after the object detection image among the plurality of images; and if it is determined that there is no overexposure, the device determines that the detection of the object is an erroneous detection based on a correlation or a similarity between the object detection image and the plurality of images excluding the object detection image. A program to function as a

Images