Information processing device, control method and program for information processing device

Abstract

To reduce erroneous detection of a human body.SOLUTION: An information processing device comprises: detection means which detects an object from an image; acquisition means which acquires information on a predetermined portion of an object that is designated to be an exception target; and output means which outputs an object that is designated to be a detection target, among the detected objects, on the basis of the acquired information. The output means outputs the detected object as the detection target, when a portion other than the predetermined portion indicated by the acquired information is detected.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a technique for detecting a specific object from an image. [Background technology]

[0002] Conventionally, there is known a technology that analyzes people in a captured image to determine whether or not there are people and how many people are present. When using this technology to detect people from an image, people in photographs or posters in the image may be mistakenly determined to be people in the captured space. As a result, the number of people is counted incorrectly. Therefore, Patent Document 1 discloses a system in which a user determines a range corresponding to a specific object, changes the threshold for the determined range to a value higher than the reliability of the specific object, and detects the face as a person if the reliability of the face detection results is higher than the threshold for each area. Patent document 2 also discloses a system that excludes a target from the human body detection results if the degree of change over time in the matching rate (reliability) of the face detection results with a predetermined template of an area including a person's face, head, or body is less than a threshold value. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-125058 [Patent Document 2] Japanese Patent Application Publication No. 2019-135618 Summary of the Invention [Problem to be solved by the invention]

[0004] When dealing with the reliability of human body detection as described in Patent Documents 1 and 2, the reliability varies greatly depending on the camera's installation angle of view, lighting changes, the orientation of the human body, etc. As a result, there are cases where a person who should be the target is excluded from the detection results, or a person who appears on a poster or the like and should be excluded from the target is mistakenly detected.

[0005] In view of the above-mentioned problems, an object of the present invention is to reduce false detection of human bodies. [Means for solving the problem]

[0006] The information processing device according to the present invention includes a detection means for detecting an object from an image, an acquisition means for acquiring information relating to a predetermined portion of the object to be excluded, and an output means for outputting an object to be detected from among the detected objects based on the acquired information, wherein the output means If the detected object is present within an area for identifying the object to be excluded and a part of the detected object matches the predetermined part to be excluded, the detected object is not included in the objects to be detected; When a part other than the predetermined part is detected, the detected object is treated as the detection target. The object that becomes The present invention is characterized in that it outputs the signal as [Effects of the Invention]

[0007] According to the present invention, it is possible to reduce false detection of a human body. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram illustrating an example of the configuration of a communication system according to an embodiment of the present invention. [Figure 2] 1 is a block diagram illustrating an example of a functional configuration of an information processing device according to an embodiment of the present invention. [Figure 3] 10A to 10C are diagrams for explaining an example of a process for determining the part, orientation, etc. of a human body by a human body detection unit. [Figure 4] 10A and 10B are diagrams for explaining an example of a process for setting a human body region by a human body detection unit. [Figure 5] FIG. 10 is a diagram showing an example of a screen for setting information about exclusion targets. [Figure 6]5 is a flowchart illustrating an example of processing according to the first embodiment. [Figure 7] 10 is a flowchart illustrating an example of processing according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] (First embodiment) Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. The embodiment described below is an example of a means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions, and the present invention is not limited to the following embodiment. In this embodiment, a human body is detected from an image, and based on information about the detected position of the human body, and information about the body part, orientation, etc., it is determined whether or not to include the detected human body in the detection target.

[0010] First, an example of the configuration of a communication system according to this embodiment will be described with reference to Fig. 1. The communication system 1 includes an information processing device 100, a network 101, and at least one network camera 102. The information processing device 100 processes image signals supplied from the network camera 102 via the network 101. The information processing device 100 also supplies signals for setting and controlling the network camera 102 to the network camera 102 via the network 101. The network 101 is a wired network, but may also be a wireless network.

[0011] Next, a description will be given of an example of the hardware configuration of the information processing device 100. The information processing device 100 includes a system bus 110, a CPU (Central Processing Unit) 111, a RAM (Random Access Memory) 112, a ROM (Read-Only Memory) 113, an external storage device 114, an input / output I / F 115, and a network I / F 116. The CPU 111 is connected to the RAM 112, the ROM 113, the external storage device 114, the input / output I / F 115, and the network I / F 116 via the system bus 110. The CPU 111 is a device that performs overall control of the information processing device 100 and performs calculations and processing of data. The CPU 111 can be configured with one or more processors.

[0012] The RAM 112 is a volatile memory and is used as a temporary storage area such as the main memory and work area of ​​the CPU 111. The ROM 113 is a non-volatile memory that stores image data and other data, as well as various computer programs for the operation of the CPU 111. The CPU 111 controls each unit of the information processing device 100 in accordance with the computer programs stored in the ROM 113, using the RAM 112 as a work memory. Note that the computer programs for the operation of the CPU 111 do not necessarily have to be stored in the ROM 113, and may also be stored in the external storage device 114.

[0013] The external storage device 114 is configured by a magnetic recording medium such as a hard disk drive (HDD) or flash memory. Computer programs such as application programs, an OS, control programs, and related programs are stored in the external storage device 114. Data can be read from and written to the external storage device 114 under the control of the CPU 111. The external storage device 114 may be used instead of the RAM 112 or the ROM 113.

[0014] The input / output I / F 115 is an interface for receiving operational input from a user via an input device and outputting data to an output device. The input device can be configured as, for example, a keyboard, a mouse, a touch panel, or buttons. The input device may also receive operational input via voice. The output device is a display that displays received images, images stored in the external storage device 114, or a predetermined input screen, and is configured, for example, as a liquid crystal panel.

[0015] The network I / F 116 is a communication interface for communicating with the network camera 102 via the network 101 under the control of the CPU 111. If the network camera 102 is connected to a wired network, the network I / F 116 includes a wired communication module for a wired connection. The wired communication module enables communication with other devices via one or more external ports. If the network 101 is a wireless network, the network I / F 116 includes a wireless communication module. The wireless communication module has well-known wireless circuitry including an antenna system, an RF transceiver, one or more amplifiers, etc. The network I / F 116 may also include various software components for processing data.

[0016] Next, an example of the functional configuration of the information processing device 100 according to this embodiment will be described with reference to Fig. 2. The functions shown in Fig. 2 are realized by the CPU 111 executing a computer program, for example. The information processing device 100 includes a communication unit 201 , an image acquisition unit 202 , a preprocessing unit 203 , a human body detection unit 204 , a result determination unit 210 , and a setting unit 211 .

[0017] The communication unit 201 receives a signal from the network camera 102 via the network 101. The signal from the network camera 102 includes an image captured by the network camera 102. The image acquisition unit 202 decodes the video data included in the signal from the network camera 102, and performs white balance adjustment, gamma processing, noise reduction processing, etc. on the decoded video data. The image acquisition unit 202 stores the processed video data in the external storage device 114 of the information processing device 100. The image acquisition unit 202 also provides the processed video data to the preprocessing unit 203.

[0018] The preprocessing unit 203 performs preprocessing on the video image data received from the image acquisition unit 202 so that it is easy to detect a human body from the image. The preprocessing includes, for example, grayscaling, threshold processing, and filtering. The human body detection unit 204 detects human bodies in each frame image (hereinafter also simply referred to as an image) that constitutes the moving image processed by the preprocessing unit 203. Furthermore, when the human body detection unit 204 detects multiple human bodies, it determines the body part, orientation, etc. of each detected human body, and stores the determination results in the ROM 113 or the external storage device 114. Furthermore, the human body detection unit 204 sets a human body region in the image based on the determination results. In this embodiment, an example of detecting a human body will be described, but the detection target is not limited to a human body, and may be, for example, a car or a robot.

[0019] An example of a process for determining the part, orientation, etc. of a human body performed by the human body detection unit 204 will now be described with reference to Fig. 3. First, the human body detection unit 204 determines the part, orientation, etc. of a human body from feature amounts in an image. Specifically, for example, as shown in group A in Fig. 3, when the outline of the head, both eyes, nose, and mouth of a human body are all detected from the image, the human body detection unit 204 determines that a "face" has been identified. When the outline of the head is detected but any of the eyes, nose, and mouth is not detected, the human body detection unit 204 determines that a "head" has been identified. In other words, the human body detection unit 204 identifies the parts of the human body in the image. 3, the human body detection unit 204 determines whether the human body in the image is facing the left half, the right half, or the front toward the camera. That is, the human body detection unit 204 determines the orientation of the human body in the image.

[0020] 3, the human body detection unit 204 determines whether the "upper body," "lower body," or "whole body" of the human body in the image has been identified. In other words, the human body detection unit 204 determines the range in which the human body in the image has been detected. Furthermore, as shown in group D in Fig. 3, the human body detection unit 204 determines whether the human body in the image was captured at a "high angle." That is, the human body detection unit 204 determines the angle of view at which the human body in the image was captured.

[0021] Note that groups A, B, and C in FIG. 3 are not necessarily mutually exclusive. For example, when a "head" is detected, one of the orientations in group B is also determined at the same time. When a "head" is determined, only the head may be determined, or the "upper body" or "whole body" may be determined at the same time. When a "face" is determined, the head outline, both eyes, nose, and mouth are all detected, so the human body is facing forward. When a "face" is determined, only the face may be determined, or the "upper body" or "whole body" may be determined at the same time. Sometimes neither a "face" nor a "head" is determined, but one of the orientations in group B is determined, and the "upper body" or "whole body" is determined. Furthermore, group D is not discriminated independently, but is discriminated together with one or more of group A, group B, and group C.

[0022] The human body detection unit 204 may be configured, for example, by cascading multiple weak classifiers. The weak classifiers detect patterns of image features such as edges and colors. The optimal detection pattern for image features can be obtained by machine learning. In this way, the human body detection unit 204 determines the part, orientation, etc. of each of the detected multiple human bodies, and stores information such as the part, orientation, etc. of each human body as the determination result in the ROM 113 or the external storage device 114. The method for determining the part and orientation of the human body is not limited to the above-mentioned method.

[0023] Next, an example of the process of setting a human body region by the human body detection unit 204 will be described with reference to Fig. 4. Fig. 4 shows an example of an image captured by the network camera 102 or an image read from the external storage device 114. The human body detection unit 204 sets human body regions 401-403 in the image based on the discrimination results of the parts and orientations of the human body in the image, as shown in discrimination results 401A, 402A, and 403A in FIG. 4 (thick bold indicates discrimination results). At this time, for example, an evaluation function using a weighted sum of each part can be used. Each human body region is set so as to include the part discriminated by the human body detection unit 204. Alternatively, each human body region may be slightly shifted up or down or to the left or right from the discriminated part, or may be set so as to include at least a part of the human body.

[0024] The result determination unit 210 determines whether or not to include a human body detected by the human body detection unit 204 in the detection result, based on information set by the setting unit 211, which will be described later. Specifically, for example, when a human body is detected within an area set by the setting unit 211 and information such as a part and orientation of the detected human body matches information such as a part and orientation of the human body set by the setting unit 211, the result determination unit 210 excludes the detected human body from the detection result. Note that when a human body has previously been determined in a moving image to be included in the detection result, the detected human body may be excluded from the objects to be determined by the result determination unit 210 as a detection result. The setting unit 211 receives information about human bodies to be excluded from the detection results of human body detection (hereinafter referred to as exclusion targets) from the user of the information processing device 100 via the input / output I / F 115. The information about the exclusion targets includes, for example, information about an area for identifying the exclusion targets (hereinafter referred to as exclusion target areas), and information about the parts and orientations of the exclusion targets. Furthermore, the setting unit 211 stores information about the exclusion target in the external storage device 114.

[0025] Next, a method for setting information on exclusion targets in the setting unit 211 will be described with reference to FIG. 5A shows an example of a screen for setting information about exclusion targets. The screen shown in FIG. 5A is displayed on the output device by the setting unit 211 via the input / output I / F 115. The setting dialog 500 can include a check box 501 for specifying whether or not to set an exclusion target, and a setting button 502 . 5B shows an example of an input screen for inputting information about exclusion targets. The screen shown in FIG. 5B is displayed on the output device by the setting unit 211 via the input / output I / F 115.

[0026] Screen 510 displays an image captured by network camera 102 or an image read from external storage device 114. Screen 510 also includes human body regions 511 to 513, a cursor 514, exclusion target regions 515 and 516, and information 517 such as human body parts and orientations. Also included at the bottom of screen 510 are a specification field 518 for specifying information such as human body parts and orientations to be excluded, an OK button 519, and a cancel button 520.

[0027] Human body regions 511 to 513 are regions set by human body detection unit 204. A cursor 514 is displayed on screen 510, and the user sets the exclusion target region at any position (corresponding to a pixel in the displayed image) within screen 510 using an input device such as a mouse. At this time, the user can set the exclusion target region by referring to human body regions 511 to 513. In the example shown in Fig. 5(b), an exclusion target area 515 has already been set by a user operation, and exclusion targets are being set in an exclusion target area 516. As shown in Fig. 5(b), when the exclusion target area 516 is selected using a cursor 514, information 517 such as the main human body parts and orientations included in the exclusion target area 516 is displayed. The user can refer to the information 517 such as the human body parts and orientations determined by the human body detection unit 204 and select information such as the human body parts and orientations to be excluded using a specification field 518 for information such as the human body parts and orientations to be excluded. When the user operates the OK button 519, information such as the part of the human body to be excluded from the set exclusion target region, orientation, etc. is set. When the cancel button 520 is operated, the setting is canceled.

[0028] In this embodiment, the information processing device 100 has all of the functional configuration shown in FIG. 2 , and image analysis is performed by the information processing device 100. However, some or all of the processing may be performed by the network camera 102. For example, the network camera 102 may be provided with the functionality of the human body detection unit 204, causing the network camera 102 to operate as an image analysis device. In this case, information on the body part and orientation of each human body detected from the image, as well as information on the human body region set for each human body, is provided to the information processing device 100, along with image data captured and generated by the network camera 102. Then, the information processing device 100 executes a process for determining the human body detection result. Therefore, the information processing device according to the embodiment of the present invention may be considered to be the information processing device 100 separate from the network camera 102. Furthermore, the information processing device according to the embodiment of the present invention may be considered to be a communication system that combines the network camera 102 and the information processing device 100, or may be considered to be the network camera 102.

[0029] Next, an example of the flow of processing according to this embodiment will be described with reference to the flowchart in Fig. 6. This processing is realized, for example, by the CPU 111 executing a computer program. This processing starts when a signal including video image data is transmitted from the network camera 102 to the information processing device 100 via the network 101. First, in S601, the image acquisition unit 202 performs predetermined image processing on video data received from the network camera 102 via the network I / F 116, and provides the processed video data to the pre-processing unit 203. In addition, the pre-processing unit 203 performs pre-processing on the video data received from the image acquisition unit 202.

[0030] Next, in S602, the human body detection unit 204 detects human bodies in the acquired image and determines information such as the body part, orientation, etc. of each detected human body. Furthermore, the human body detection unit 204 sets multiple human body regions in the image based on the determined body parts. The subsequent processing is performed for each human body region set by the human body detection unit 204. In S603, the result determination unit 210 acquires information about the exclusion target from the ROM 113 or the external storage device 114, and performs a process of determining whether or not the human body detected by the human body detection unit 204 is present in the exclusion target area. If the result determination unit 210 determines that a human body is present in the exclusion target region, the process proceeds to S604. On the other hand, if the result determination unit 210 determines that a human body is not present in the exclusion target region, the result determination unit 210 determines that the detected human body is not to be excluded, and the process proceeds to S606.

[0031] In S604, the result determination unit 210 determines whether the information on the part and orientation of the human body determined by the human body detection unit 204 matches the information on the part and orientation of the exclusion target within the exclusion target region. If the result determination unit 210 determines that there is a match, the process proceeds to S605. On the other hand, if the result determination unit 210 determines that there is no match, the result determination unit 210 determines that the detected human body is not to be excluded, and the process proceeds to S606. In S605, the result determination unit 210 excludes the detected human body from the detection result.

[0032] In S606, the result determination unit 210 determines whether the processing from S603 onwards has been performed on all human body regions. If the result determination unit 210 determines that the processing from S603 onwards has been performed on all human body regions, the processing ends. On the other hand, if the result determination unit 210 determines that there is a human body region on which the processing from S603 onwards has not been performed, the processing returns to S603 and the processing of S603 is performed on the next human body region. Note that, in the determination of S604, time series information may be used, and if the information on the body part, orientation, etc. matches at a threshold rate or more within a predetermined number of frames (predetermined time), the process may proceed to S605 and the person may be excluded from the detection results. Also, if the determination of S603 or S604 is No, the person may be set as a non-exclusion target, and may be excluded from the determination targets in the result determination unit 210 from the next frame onwards.

[0033] As described above, according to this embodiment, information on the position, part, and orientation of a human body to be excluded from the detection results is acquired, and based on this information, it is determined whether a human body detected at the same position should be excluded from the detection results. As a result, even if a person is standing in front of a poster containing a human body, it is possible to determine that the detected human body should be detected based on the part and orientation of the human body. As a result, it is possible to reduce false detection of human bodies and reduce the possibility that a specific person that should be detected will not be detected.

[0034] (Second embodiment) In the first embodiment described above, a process for setting the exclusion target area, and the parts and orientations of the human body to be excluded in advance using a UI such as that shown in Fig. 5. In this embodiment, the flow of a method for adding and deleting information related to exclusion targets using time-series information will be described with reference to the flowchart in Fig. 7. The processing according to this flowchart is performed between S602 and S603.

[0035] First, in S701, the setting unit 211 determines whether or not one or more exclusion target regions have been set in the image. If the setting unit 211 determines that an exclusion target region has been set, the process proceeds to S702. On the other hand, if the setting unit 211 determines that an exclusion target region has not been set, the process proceeds to S704. In S702, the setting unit 211 determines whether a human body that was detected in the set exclusion target region has not been detected for a predetermined number of frames (a predetermined time). If the setting unit 211 determines that a human body has not been detected for the predetermined number of frames, the setting unit 211 proceeds to S703. On the other hand, if the setting unit 211 determines that a human body has not been detected for the predetermined number of frames, the setting unit 211 proceeds to S704. In S703, the setting unit 211 deletes the setting of the exclusion target area in which it is determined in S702 that no human body has been detected, that is, deletes information related to the exclusion target in the exclusion target area.

[0036] Next, in S704, the setting unit 211 determines whether or not information such as the same human body part, orientation, etc. has been detected at the same position for a predetermined number of frames (predetermined time). If the setting unit 211 determines that information has been detected, the process proceeds to S705. On the other hand, if the setting unit 211 determines that information has not been detected, the process ends. It is also possible to set a threshold percentage and determine whether or not the above-mentioned condition is satisfied for at least the threshold percentage within a predetermined number of frames (predetermined time). In S705, the setting unit 211 sets the region information of the human body detected in S704, as well as information such as the part and orientation of the human body, as information related to the exclusion target. At this time, the exclusion target region may be presented to the user from an output device. Note that the position and size of the exclusion target region are determined according to the part of the detected human body, but the size, position, etc. may be changed. Furthermore, the order of processing S701 to S703 and S704 to S705 may be reversed. Furthermore, in S702, it is determined whether a human body that had been detected within the set exclusion target region has become undetectable for a predetermined number of frames (predetermined time), but the determination may also be made under the following conditions. That is, it may be determined whether information such as the body part and orientation of a human body that had been detected within the set exclusion target region for a threshold percentage or more of the predetermined number of frames (predetermined time) no longer matches.

[0037] As described above, according to this embodiment, it is possible to add or delete information about an object to be excluded using time-series information. Then, it is possible to determine whether or not to include a detected human body in the detection result based on information such as the position information of the human body in the image, the body part, and the orientation of the human body. As a result, it is possible to reduce false detection of a human body and reduce the possibility that a specific person who should be detected will not be detected.

[0038] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0039] 204 human body detection unit, 210 result determination unit

Claims

1. a detection means for detecting an object from an image; an acquisition means for acquiring information about a predetermined portion of an object to be excluded; an output means for outputting an object to be detected among the detected objects based on the acquired information; and The information processing device is characterized in that the output means does not include the detected object among the objects to be detected when the detected object is present within an area for identifying the object to be excluded and a part of the detected object matches the specified part to be excluded, and outputs the detected object as the object to be detected when a part other than the specified part is detected.

2. An information processing device as described in Claim 1, characterized in that if a part of the detected object matches the specified part to be excluded at a specified percentage of a specified time, the detected object is not included in the detection targets.

3. 3. The information processing apparatus according to claim 1, wherein the acquisition unit acquires information about the exclusion target in response to a user operation.

4. 4. The information processing device according to claim 1, wherein, when a part of the detected object matches the predetermined part at a predetermined rate within a predetermined time, the output means outputs information about the matching object as the object to be excluded.

5. 5. The information processing apparatus according to claim 4, wherein the acquiring means deletes the information about the predetermined part when the object to be excluded has not been detected for a period longer than a predetermined time.

6. 6. The information processing apparatus according to claim 1, wherein the acquisition means further acquires information about a position or an orientation of the object to be excluded.

7. 7. The information processing apparatus according to claim 6, wherein the output means outputs the object to be detected based on information about the position or orientation of the object to be excluded.

8. the acquiring means includes, as the information about the predetermined body part, the position of at least one body part selected from the face, head, left half body, right half body, front, upper body, lower body, and whole body of the human body; The information processing device according to any one of claims 1 to 7, characterized in that the output means includes in the detection target an object in which the position of at least one of the face, head, left half body, right half body, front, upper body, lower body, and whole body of a human body is detected as a part not included in the information regarding the specified part.

9. An information processing device described in any one of claims 1 to 8, characterized in that a human body that has previously been determined to be included in the detection result is not used as the subject of the determination of whether or not it is included in the object to be detected.

10. a detection step of detecting an object from the image; an acquisition step of acquiring information about a predetermined portion of an object to be excluded; an output step of outputting an object to be detected among the detected objects based on the acquired information; and The control method for an information processing device, characterized in that the output process does not include the detected object among the objects to be detected if the detected object is present within an area for identifying the object to be excluded and a part of the detected object matches the specified part to be excluded, and outputs the detected object as the object to be detected if a part other than the specified part is detected.

11. a detection step of detecting an object from the image; an acquisition step of acquiring information about a predetermined portion of an object to be excluded; an output step of outputting an object to be detected among the detected objects based on the acquired information; A program for causing a computer to execute the above, The output process is a program characterized in that, if the detected object is present within an area for identifying the object to be excluded and a part of the detected object matches the specified part to be excluded, the detected object is not included in the objects to be detected, and if a part other than the specified part is detected, the detected object is output as the object to be detected.

Images