Person detection device, system, method and program

Abstract

The purpose of the present disclosure is to provide a person detection device capable of accurately detecting a person. This person detection device (100) comprises: an acquisition unit (11) which acquires a video; a detection unit (120) which detects a person from frames, which constitute the video, on the basis of a threshold value of a prescribed person detection score; an adjustment unit (130) which adjusts the threshold value of the person detection score according to an aspect ratio of an area from which the person is detected when the person is detected from the frames. The person detection device (100) adjusts the threshold value of the person detection score according to the aspect ratio of the area, thereby being capable of accurately detecting a person.

Description

Person detection device, system, method and non-transitory computer-readable medium having program stored thereon

[0001] The present disclosure relates to a person detection device, a system, a method, and a non-transitory computer-readable medium having a program stored thereon.

[0002] When a person is included in a video captured by an in-vehicle camera while a vehicle is traveling, it may be necessary to blur the person from the viewpoint of protecting personal information. In order to blur the person included in the video, it is required to accurately detect the person from the captured video.

[0003] Patent Document 1 discloses a technique for determining whether an object captured in a video is a person.

[0004] International Publication No. 2010 / 073929

[0005] If a false positive occurs when detecting a person in a video, there is a risk that areas of the video where there is no person, i.e., areas that do not need to be blurred, will be blurred. Therefore, there is a need to improve the accuracy of detecting people in video.

[0006] The present disclosure has been made to solve such problems, and aims to provide a human detection device, system, method, and non-transitory computer-readable medium on which a program is stored that can accurately detect people.

[0007] The human detection device according to the present disclosure includes an acquisition unit that acquires a captured image, a detection unit that detects a person from a frame that constitutes the image based on a predetermined human detection score threshold, and an adjustment unit that, when a person is detected from the frame, adjusts the human detection score threshold according to the aspect ratio of the area in which the person is detected.

[0008] The human detection system according to the present disclosure comprises an imaging device that captures video of the surrounding area, and a human detection device capable of communicating with the imaging device, wherein the human detection device comprises: an acquisition unit that acquires video captured by the imaging device; a detection unit that detects a person from frames that constitute the video based on a predetermined human detection score threshold; and an adjustment unit that, when a person is detected from the frame, adjusts the human detection score threshold according to the aspect ratio of the area in which the person is detected.

[0009] The person detection method according to the present disclosure includes the steps of: acquiring a captured image by a computer; detecting a person from frames constituting the image based on a predetermined person detection score threshold; and, when a person is detected from the frame, adjusting the person detection score threshold according to the aspect ratio of the area in which the person is detected.

[0010] The non-transitory computer-readable medium according to the present disclosure stores a person detection program that causes a computer to execute the following processes: acquiring a captured image; detecting a person from frames that constitute the image based on a predetermined person detection score threshold; and, when a person is detected from the frame, adjusting the person detection score threshold according to the aspect ratio of the area in which the person is detected.

[0011] The present disclosure makes it possible to provide a human detection device, a system, a method, and a non-transitory computer-readable medium storing a program capable of detecting a human being with high accuracy.

[0012] FIG. 1 is a block diagram showing the configuration of a human detection device according to embodiment 1. FIG. 2 is a flowchart showing the flow of a human detection method according to embodiment 1. FIG. 3 is a block diagram showing the configuration of a human detection system according to embodiment 2. FIG. 4 is a block diagram showing the configuration of a human detection device according to embodiment 2. FIG. 5 is a diagram showing an example of a frame in which a person is detected. FIG. 6 is a flowchart showing the flow of a human detection method according to embodiment 2. FIG. 7 is a block diagram showing the configuration of a human detection device according to embodiment 3. FIG. 8 is a flowchart showing the flow of a human detection method according to embodiment 3.

[0013] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are designated by the same reference numerals, and for clarity of explanation, duplicate explanations will be omitted as necessary.

[0014] First Embodiment Fig. 1 is a block diagram showing the configuration of a human detection device 100 according to a first embodiment. The human detection device 100 includes an acquisition unit 110, a detection unit 120, and an adjustment unit 130. The human detection device 100 is connected to a network 500 (not shown), which may be wired or wireless. An image capture device 300 (not shown) and the like are connected to the network 500. The image capture device 300 is installed in a vehicle 310 (not shown), and is a device that captures images of the surroundings of the vehicle. The video captured by the image capture device 300 is typically a moving image and includes multiple frames.

[0015] The acquisition unit 110 acquires video captured by the imaging device 300 installed in the vehicle 310. Note that the video includes at least one frame, and typically includes multiple frames. The detection unit 120 detects people from the frames constituting the video acquired by the acquisition unit 110 based on a predetermined person detection score threshold. The person detection score is a numerical value calculated for each region within a frame. The person detection score is higher in regions where a person is likely to be present than in other regions. Note that the method for calculating the person detection score is not particularly limited, and existing technology can be applied. The predetermined person detection score threshold (hereinafter sometimes referred to as the "threshold") is a predetermined numerical value used to detect people from the frames constituting the video. The threshold may be set to a different value depending on the region within the frame. If the person detection score for a region within a frame is equal to or lower than the threshold for that region, the detection unit 120 determines that a person is present in that region.

[0016] When the threshold value of the human detection score is lowered, it becomes easier to detect a person from a frame, but there is a higher risk of erroneously detecting an object other than a person. Therefore, when the detection unit 120 detects a person, the adjustment unit 130 adjusts the threshold value of the human detection score according to the aspect ratio of the area in which the person is detected.

[0017] 2 is a flowchart showing the flow of the person detection method according to the first embodiment. First, the acquisition unit 110 acquires a captured video (step S101). Next, the detection unit 120 detects a person from frames constituting the video based on a predetermined person detection score threshold (step S102). If a person is detected from a frame in step S102, the adjustment unit 130 adjusts the person detection score threshold (step S103). In this way, the person detection method according to this embodiment adjusts the person detection score threshold according to the aspect ratio of the area in which the person is detected, thereby enabling accurate detection of people.

[0018] The human detection device 100 includes a processor, a memory, and a storage device (not shown). The storage device stores a computer program that implements the processing of the human detection method according to this embodiment. The processor then loads the computer program from the storage device into the memory and executes the computer program. As a result, the processor realizes the functions of the acquisition unit 110, the detection unit 120, and the adjustment unit 130.

[0019] The acquisition unit 110, the detection unit 120, and the adjustment unit 130 may each be realized by dedicated hardware. Some or all of the components of each device may be realized by general-purpose or dedicated circuits, processors, etc., or a combination of these. These may be configured by a single chip, or by multiple chips connected via a bus. Some or all of the components of each device may be realized by a combination of the above-mentioned circuits, etc., and a program. A CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), etc. may be used as the processor.

[0020] Furthermore, when some or all of the components of the human detection device 100 are realized by multiple information processing devices, circuits, etc., the multiple information processing devices, circuits, etc. may be centrally or decentralized. For example, the information processing devices, circuits, etc. may be realized as a client-server system, a cloud computing system, or the like, in a form in which each is connected via a communication network. Furthermore, the functions of the human detection device 100 may be provided in a SaaS (Software as a Service) format.

[0021] <Embodiment 2> Embodiment 2 is a specific example of the above-described embodiment 1. Fig. 3 is a block diagram showing the configuration of a person detection system 200 according to embodiment 2. The person detection system 200 includes a photographing device 300 and a person detection device 400. The photographing device 300 is connected to the person detection device 400 via a network 500. Note that descriptions that overlap with embodiment 1 will be omitted as appropriate.

[0022] The person detection system 200 is a system for detecting a person from video captured by a vehicle 310. The vehicle 310 is, for example, an automobile, but may also be a vehicle other than an automobile, such as a motorcycle or a bicycle. An imaging device 300 is installed in the vehicle 310. The imaging device 300 is a device that captures the scenery around the vehicle 310, such as a drive recorder. The imaging device 300 includes an imaging unit 301 and a communication unit 302. The imaging unit 301 is a camera. The imaging unit 301 captures, for example, the scenery ahead of the vehicle 310, i.e., the scenery that can be seen by a driver seated in the driver's seat of the vehicle 310. The communication unit 302 is a communication interface with a network 500. The communication unit 302 transmits the video captured by the imaging unit 301 to the person detection device 400 via the network 500.

[0023] Next, the configuration of the human detection device 400 will be described in detail with reference to Fig. 4. Fig. 4 is a block diagram showing the configuration of the human detection device 400. The human detection device 400 includes a memory 410, a communication unit 420, a storage unit 430, and a control unit 440.

[0024] The memory 410 is a storage area that temporarily stores the processing contents of the control unit 440, and is a volatile storage device such as a RAM (Random Access Memory). The communication unit 420 is an interface that communicates with the outside of the human detection device 400. The storage unit 430 is a storage device that stores a threshold 431, a program 432, and the like. The threshold 431 is a numerical value used when detecting a person based on the human detection score, and different values ​​may be set depending on the area within the frame. The program 432 is a computer program that implements the human detection process according to this embodiment.

[0025] The control unit 440 includes an acquisition unit 441, a detection unit 442, and an adjustment unit 443. The control unit 440 is a control device that controls the operation of the human detection device 400, and is, for example, a processor such as a CPU. The control unit 440 loads the program 432 from the storage unit 430 into the memory 410 and executes it. In this way, the control unit 440 realizes the functions of the acquisition unit 441, the detection unit 442, the adjustment unit 443, and the determination unit 444.

[0026] The acquisition unit 441 acquires the video transmitted from the image capturing device 300. The video usually includes multiple frames. The video may also include identification information, etc. The identification information is information for identifying the vehicle 310 on which the image capturing device 300 that captured the video is installed.

[0027] The detection unit 442 detects people from the video acquired by the acquisition unit 441. Specifically, the detection unit 442 calculates a person detection score for each frame constituting the video acquired by the acquisition unit 441. Next, the detection unit 442 determines whether the person detection score is equal to or less than a threshold 431. The detection unit 442 calculates and determines the person detection score for each of a plurality of frames. The threshold 431 is a preset numerical value used when detecting people based on the person detection score, and different values ​​may be set depending on the area within the frame. The detection unit 442 determines that a person is present in a location within the frame where the person detection score is equal to or greater than the threshold.

[0028] Fig. 5 is a diagram showing an example of a frame in which a person has been detected. Frame 10 shown in Fig. 5 is a frame constituting the video acquired by acquisition unit 441. When person 20 appears in frame 10 as shown in Fig. 5, the person detection score in the vicinity of person 20 is calculated to be higher than the person detection score in other areas. When the person detection score in the vicinity of person 20, i.e., in area 30, is equal to or greater than a threshold, detection unit 442 determines that a person appears in area 30.

[0029] As shown in FIG. 5 , the region 30 is rectangular. Here, the direction perpendicular to the ground in the frame is defined as the vertical direction, and the direction parallel to the ground is defined as the horizontal direction. The sides of the region 30 extending vertically are defined as vertical sides 30a, and the sides extending horizontally are defined as horizontal sides 30b. The determination unit 444 calculates the aspect ratio by dividing the length of the vertical side 30a by the length of the horizontal side 30b, and determines that human detection has been successful if the aspect ratio is within a predetermined range. Here, the predetermined range is 1 or greater, preferably 1 to 4, and more preferably 2 to 3.5. If the aspect ratio is less than 1, i.e., if the region 30 is square or horizontally elongated, it is considered that an object other than a human is captured in the region 30, or that only a portion of a human is captured. Therefore, the determination unit 444 determines that human detection has been successful if the aspect ratio is 1 or greater, i.e., if the region 30 is vertically elongated. 5, a person is usually detected as an area 30 with an aspect ratio of approximately 1 to 4, and a walking person in particular is detected as an area 30 with an aspect ratio of approximately 2 to 3.5. Therefore, if the predetermined range is set to 1 to 4, and more preferably 2 to 3.5, a person can be determined with higher accuracy.

[0030] Returning to FIG. 4 , the explanation will be continued. When the determination unit 444 determines that person detection has failed, the adjustment unit 443 adjusts the threshold value of the person detection score. Specifically, when the determination unit 444 determines that the aspect ratio of the area 30 is less than 1, the adjustment unit 443 raises the threshold value of the person detection score. The detection unit 442 re-detects people from the frames constituting the video based on the adjusted threshold value. When the threshold value of the person detection score is raised, the detection unit 442 compares it with the threshold value before the increase and detects fewer areas as areas that are likely to contain people. When an area is detected again by re-detecting a person, the determination unit 444 may calculate the aspect ratio of the area and determine whether the aspect ratio is within a predetermined range.

[0031] In this way, when an area in which a person is likely to be photographed is detected, the human detection device 400 according to this embodiment determines whether the object photographed in the area is a person by determining whether the aspect ratio of the area is within a predetermined range. Therefore, the human detection device 400 can accurately detect people. Furthermore, when the human detection device 400 determines that person detection has failed, it increases the threshold value of the human detection score and performs person detection again. Therefore, compared to when the threshold value of the human detection score is not adjusted, the human detection device 400 can accurately detect people.

[0032] Next, the operation of human detection device 400 when detecting a human will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the flow of human detection processing. In the example shown in Fig. 6, a case where a video includes multiple frames will be described.

[0033] First, the acquisition unit 441 acquires video from the image capture device 300 (step S201). Next, the detection unit 442 detects a person based on a preset threshold 431 for each frame constituting the video acquired in step S201 (step S202). If a person is not detected in any of the frames constituting the video (step S202: No), the person detection ends. If a person is detected in any frame (step S202: Yes), the determination unit 444 calculates the aspect ratio of the area in which the person is detected and determines whether the aspect ratio is within a predetermined range (step S203: Yes). If the aspect ratio is within the predetermined range (step S203: Yes), the person detection ends. If the aspect ratio is outside the predetermined range (step S203: No), the adjustment unit 443 raises the threshold of the person detection score (step S204). Next, the detection unit 442 performs person detection again for each frame constituting the video based on the raised threshold (step S205). In this way, when it is determined that human detection has failed, the human detection device 400 according to this embodiment increases the threshold value of the human detection score and performs human detection again, thereby enabling accurate detection of humans.

[0034] Furthermore, if a person is detected again in step S205, the determination unit 444 may calculate the aspect ratio of the area in which the person is detected again and determine whether the aspect ratio is within a predetermined range. Furthermore, if the aspect ratio of the area in which the person is detected again is outside the predetermined range, the adjustment unit 443 may further increase the threshold value of the person detection score, and the detection unit 442 may perform person detection again for each frame constituting the video based on the further increased threshold value. In this way, by repeatedly detecting people and adjusting the threshold value, the person detection device 400 can detect people with higher accuracy.

[0035] <Embodiment 3> Embodiment 3 is a modification of the above-described embodiment 2. When it is determined that human detection has been successful, the human detection device 600 according to this embodiment performs a masking process on the area in which the human is detected. FIG. 7 is a block diagram showing the configuration of the human detection device 600 according to embodiment 3. Compared to the human detection device 400 shown in FIG. 4, the human detection device 600 is different in that it includes a control unit 640 instead of the control unit 440. The control unit 640 includes a processing unit 645 in addition to the components included in the control unit 440. Since the other components overlap with those of embodiment 1 or 2, their description will be omitted as appropriate.

[0036] The control unit 640 includes an acquisition unit 441, a detection unit 442, an adjustment unit 443, a determination unit 444, and a processing unit 645. The acquisition unit 441 acquires video from the image capture device 300. The detection unit 442 detects people from frames constituting the video acquired by the acquisition unit 441 based on a threshold value of the person detection score. The determination unit 444 calculates the aspect ratio of the detected area 30 and determines that person detection has been successful if the aspect ratio is within a predetermined range. If the determination unit 444 determines that person detection has failed, the adjustment unit 443 adjusts the threshold value of the person detection score. If the determination unit 444 determines that person detection has been successful, the processing unit 645 performs a masking process on the determined area 30, i.e., blurs it. The method of the masking process is not particularly limited, and processing is performed using existing technology.

[0037] Next, the operation of the human detection device 600 when detecting a human will be described with reference to Fig. 8. Fig. 12 is a flowchart showing the flow of the human detection process.

[0038] First, the acquisition unit 441 acquires video from the image capture device 300 (step S301). Next, the detection unit 442 detects a person based on a preset threshold 431 for the frames constituting the video acquired in step S301 (step S302). If a person is not detected from the frames constituting the video (step S302: No), the person detection ends. If a person is detected from the frames constituting the video (step S302: Yes), the determination unit 444 calculates the aspect ratio of the region 30 in which the person is detected and determines whether the aspect ratio is within a predetermined range (step S303). If the aspect ratio of the region 30 is within the predetermined range (step S303: Yes), the processing unit 645 performs a masking process on the region 30 (step S307). If the aspect ratio of the region 30 is outside the predetermined range (step S303: No), the adjustment unit 443 increases the threshold value of the person detection score (step S304). Next, the detection unit 442 performs person detection again for the frames constituting the video based on the increased threshold (step S305). The determination unit 444 calculates the aspect ratio of the region detected in step S305 and determines whether the aspect ratio is within a predetermined range (step S306). If the aspect ratio of region 30 is outside the predetermined range (step S306: No), person detection ends. If the aspect ratio of region 30 is within the predetermined range (step S306: Yes), the processing unit 645 performs a masking process on region 30 (step S307).

[0039] In this way, the human detection device 600 according to this embodiment determines whether a detected object is a human based on the aspect ratio of the region. Therefore, the human detection device 600 can perform masking processing only on regions where it is determined that a human has been successfully detected, without performing masking processing on regions where an object other than a human has been erroneously detected.

[0040] Although the above-described embodiment has been described as a hardware configuration, the present disclosure is not limited to this. Any processing in the present disclosure can also be realized by causing a CPU to execute a computer program.

[0041] In the above examples, the program can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-RWs, DVDs (Digital Versatile Discs), and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program may also be supplied to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire or an optical fiber, or via a wireless communication path.

[0042] The present disclosure is not limited to the above-described embodiments, and may be modified as appropriate without departing from the spirit and scope of the present disclosure. In addition, the present disclosure may be implemented by appropriately combining the respective embodiments.

[0043] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0044] 10 Frame 20 Person 30 Area 30a Vertical Side 30b Horizontal Side 100 Person Detection Device 110 Acquisition Unit 120 Detection Unit 130 Adjustment Unit 200 Person Detection System 300 Shooting Device 301 Shooting Unit 302 Communication Unit 310 Vehicle 400 Person Detection Device 410 Memory 420 Communication Unit 430 Storage Unit 431 Threshold 432 Program 440 Control Unit 441 Acquisition Unit 442 Detection Unit 443 Adjustment Unit 444 Determination Unit 500 Network 600 Person Detection Device 640 Control Unit 645 Processing Unit

Claims

1. An acquisition means for acquiring the captured video; a detection means for detecting a person from frames constituting the video based on a predetermined threshold value of the person detection score; an adjustment means for adjusting a threshold value of the human detection score in accordance with an aspect ratio of an area in which a person is detected when the person is detected from the frame; Person detection device.

2. Further, a determination unit is provided for determining that human detection has been successful when the aspect ratio is within a predetermined range, the aspect ratio is a value obtained by dividing a vertical side by a horizontal side, where the vertical side is a side perpendicular to the ground in the area where the person is detected and the horizontal side is a side perpendicular to the vertical side, The predetermined range is a range in which the aspect ratio is equal to or greater than 1. The human detection device according to claim 1 .

3. If it is determined that person detection has failed, the adjusting means increases a threshold value of the person detection score, the detection means re-detects people from frames constituting the video based on the adjusted threshold of the person detection score; The human detection device according to claim 2 .

4. If a person is detected again in the frame, The determination means determines that the person detection has been successful when the aspect ratio of the area in which the person is detected again is within a predetermined range. The human detection device according to claim 3 .

5. The image processing device further includes a processing unit that, when it is determined that the person detection has been successful, performs a masking process on the area in which the person is detected. The human detection device according to claim 2 .

6. A camera that captures images of the surrounding area; a person detection device capable of communicating with the photographing device, The person detection device Acquire an image captured by the imaging device; Detecting a person from frames constituting the video based on a predetermined person detection score threshold; When a person is detected from the frame, a threshold value of the person detection score is adjusted according to an aspect ratio of an area in which the person is detected. People detection system.

7. The person detection device If the aspect ratio is within a predetermined range, it is determined that the person detection has been successful; the aspect ratio is a value obtained by dividing a vertical side by a horizontal side, where the vertical side is a side perpendicular to the ground in the area where the person is detected and the horizontal side is a side perpendicular to the vertical side, The predetermined range is a range in which the aspect ratio is equal to or greater than 1. The person detection system of claim 6 .

8. The computer Obtain the captured footage, Detecting a person from frames constituting the video based on a predetermined person detection score threshold; When a person is detected from the frame, a threshold value of the person detection score is adjusted according to an aspect ratio of an area in which the person is detected. Person detection methods.

9. On the computer, A process of acquiring video; A process of detecting a person from frames constituting the video based on a predetermined person detection score threshold; a process of adjusting a threshold value of the person detection score according to an aspect ratio of an area in which a person is detected when the person is detected from the frame; A person detection program that executes the following.