Information processing apparatus, information processing method, and program

Abstract

A first portion rectangle inference means infers a particular portion of a human body from an image, and outputs a first portion rectangle including the particular portion. A first human body rectangle inference means infers, on the basis of the coordinates of the first portion rectangle, a human body corresponding to the particular portion included in the first portion rectangle, and outputs a first human body rectangle including the human body. A second human body rectangle inference means infers a human body from an image, and outputs a second human body rectangle including the human body. A second portion rectangle inference means infers, on the basis of the coordinates of the second human body rectangle, a particular portion corresponding to the human body included in the second human body rectangle, and outputs a second portion rectangle including the particular portion. A human body integration means integrates, for a plurality of pairs of portion rectangles and human body rectangles which correspond to each other, overlapping human body rectangles on the basis of at least the relation between the portion rectangles.

Description

Technical Field

[0001] This disclosure relates to a technique for detecting a human body from an image.

Background Art

[0002] In recent years, many object detection techniques using neural networks based on deep learning have been proposed. Object detection is to estimate what object is shown in an image or video, and at the same time, estimate the position and size of the object by obtaining the circumscribed rectangle of the object, called a "bounding box". Therefore, an object detector outputs the position coordinates of the bounding box of the object, the category of the object, and the confidence indicating the probability of that category.

[0003] An example of an object detection device is described in Non-Patent Document 1. This document includes a discriminator that outputs the bounding box position of the object candidate position and the confidence indicating the object-likeness from within the image, and a discriminator that outputs the bounding box position, the category of the object, and the confidence for the category from the object candidate position obtained above.

[0004] Among object detection fields, human body detection is one of the very important tasks in particular. Human body detection from a moving image can be considered for various applications such as autonomous driving, security monitoring, and biometric authentication. In particular, in the real world, problems of occlusion such as the overlap of people in a crowded environment and the partial hiding of the torso by obstacles can be assumed, and human body detection robust to occlusion is required. Patent Document 1 describes a method of calculating the distance between the face region and the human body region detected from an image, and deleting the human body region as inappropriate when the face region and the human body region are in a situation where they cannot actually exist.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Non-Patent Literature

[0006]

Non-Patent Literature 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] This disclosure aims to improve the technologies disclosed in prior art documents.

Means for Solving the Problems

[0008] In one aspect of this disclosure, the information processing apparatus includes first partial rectangle estimation means for estimating the head of a human body from an image and outputting a first partial rectangle including the head; first human rectangle estimation means for estimating the human body corresponding to the head included in the first partial rectangle based on the coordinates of the first partial rectangle by using prior knowledge regarding the positional relationship between the human body and the head, and outputting a first human rectangle including the human body; first pair generation means for generating a first pair including the first partial rectangle and the first human rectangle; second human rectangle estimation means for estimating a human body from the image and outputting a second human rectangle including the human body; second partial rectangle estimation means for estimating the head corresponding to the human body included in the second human rectangle based on the coordinates of the second human rectangle by using the prior knowledge, and outputting a second partial rectangle including the head; second pair generation means for generating a second pair including the second partial rectangle and the second human rectangle; integration means for integrating a plurality of pairs of partial rectangles and human rectangles including the first pair and the second pair comprising 、 The integration means determines whether the input pair is the first pair or the second pair. If the input pair is the second pair, the second pair is adopted. If the input pair is the first pair, it is determined whether the first pair is of the same person as the second pair. If it is of the same person, the first pair is rejected; if it is not of the same person, the first pair is adopted. .

[0009] In another aspect of this disclosure, an information processing method executed by a computer includes estimating a human head from an image and outputting a first partial rectangle including the head, estimating a human body corresponding to the head included in the first partial rectangle based on the coordinates of the first partial rectangle by using prior knowledge regarding the positional relationship between the human body and the head, and outputting a first human rectangle including the human body, generating a first pair including the first partial rectangle and the first human rectangle, estimating a human body from the image and outputting a second human rectangle including the human body, estimating a head corresponding to the human body included in the second human rectangle based on the coordinates of the second human rectangle by using the prior knowledge, and outputting a second partial rectangle including the head, generating a second pair including the second partial rectangle and the second human rectangle, integrating a plurality of pairs of a partial rectangle and a human rectangle including the first pair and the second pair Execute the integration process. The integration process determines whether the input pair is the first pair or the second pair. If the input pair is the second pair, the second pair is adopted. If the input pair is the first pair, it is determined whether the first pair is of the same person as the second pair. If it is of the same person, the first pair is rejected; if it is not of the same person, the first pair is adopted. .

[0010] In still another aspect of this disclosure, a program includes estimating a human head from an image and outputting a first partial rectangle including the head, estimating a human body corresponding to the head included in the first partial rectangle based on the coordinates of the first partial rectangle by using prior knowledge regarding the positional relationship between the human body and the head, and outputting a first human rectangle including the human body, generating a first pair including the first partial rectangle and the first human rectangle, estimating a human body from the image and outputting a second human rectangle including the human body, Using the prior knowledge, based on the coordinates of the second human body rectangle, estimate the head corresponding to the human body included in the second human body rectangle, and output a second partial rectangle including the head. Generate a second pair including the second partial rectangle and the second human body rectangle. Integrate a plurality of pairs of partial rectangles and human body rectangles including the first pair and the second pair. Integration Cause a computer to execute the process. 、 The integration process determines whether the input pair is the first pair or the second pair. If the input pair is the second pair, the second pair is adopted. If the input pair is the first pair, it is determined whether the first pair is of the same person as the second pair. If it is of the same person, the first pair is rejected; if it is not of the same person, the first pair is adopted. 。

Brief Description of Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Embodiments for Carrying Out the Invention

[0012] Hereinafter, with reference to the drawings, preferred embodiments of this disclosure will be described. <Basic Principle> In a scene where occlusion occurs in the object to be detected, there is a problem that the human body cannot be detected with high accuracy. In this case, two cases can be considered for occlusion. One is the overlap between different categories. For example, a case where a pedestrian overlaps with an obstacle such as a wall or a car and a part of the body is not visible can be considered. In this case, the visible area of the body becomes small, a lack of information amount occurs, and it becomes difficult to estimate the position of the bounding box of the whole body.

[0013] One of the problems caused by occlusion in object detection is that it is difficult to directly estimate the whole of the object where occlusion is occurring from the image. Therefore, among the parts subordinate to the object to be detected, those where occlusion is less likely to occur are estimated first, and the estimation of the object to be detected is performed only in the vicinity thereof. For example, in the case of the human body, the head corresponds to that part. Surveillance cameras and in-vehicle cameras in the real world are often installed at high positions, and the head tends to be relatively less likely to be occluded. Therefore, in this embodiment, first, the head region is estimated, and the human body region is estimated in the vicinity thereof, so that the target object can be detected even in an environment with a lot of occlusion.

[0014] Another problem is the overlap between objects in the same category. For example, in a crowded situation such as a public facility or an event venue, there is an overlap between people, making it difficult to estimate the position of the bounding box of the whole body by occlusion. Also, in object detection, when multiple bounding box estimation results are obtained for the same object in an image, these are integrated into one using a method called NMS (Non Maximum Suppresion). However, in a crowded environment, the overlap between people is large, and the rectangles estimated for different individuals are integrated into one by NMS.

[0015] To solve this problem, in this embodiment, in the integration process, the rectangular position information of the head is used. Specifically, the integration process is performed while emphasizing the overlap between the heads. This prevents the situation where actually separate individuals are integrated into one person by the integration process.

[0016] In addition, as an issue when performing the integration process using the head, the accuracy of estimating the human body from the head is low. When there is a large occlusion of the human body and only the head is visible, the accuracy of estimating the human body rectangle from the head rectangle is lower than the accuracy of estimating the human body rectangle when the whole human body is visible without occlusion. That is, the accuracy of the human body rectangle obtained by estimating the human body from the image (hereinafter also referred to as the "human body rectangle derived from the human body") and the human body rectangle obtained by estimating the head rectangle and then estimating the human body from the estimated head rectangle (hereinafter referred to as the "human body rectangle derived from the head") are originally different. Therefore, if many human body rectangles derived from the head remain due to the integration process, the accuracy of the entire detector will decrease. Thus, in the following embodiments, the integration process is performed so as to actively retain the human body rectangle derived from the human body rather than the human body rectangle derived from the head. This can prevent the overall accuracy of the detector from decreasing. In the above description, the head is used as an example of a part of the human body where occlusion is less likely to occur, but other specific parts of the human body may also be used.

[0017] <First Embodiment> [Human Body Detection Device] FIG. 1 shows a human body detection device according to the first embodiment. The human body detection device 100 detects a human body from the input image data. Although details will be described later, the human body detection device 100 can distinguish and detect a human body (hereinafter also referred to as "human body derived from the human body") obtained by estimating a human body from a human body rectangle, and a human body (hereinafter referred to as "human body derived from the head") obtained by estimating a head rectangle and then estimating a human body from the estimated head rectangle.

[0018] [Hardware Configuration] FIG. 2 is a block diagram showing the hardware configuration of the human body detection device according to the first embodiment. As shown in the figure, the human body detection device 100 includes an input device 12, a processor 13, a memory 14, a recording medium 15, a database (DB) 16, and a display unit 17.

[0019] The input device 12 inputs image data used for learning and inference of the human body detection device 100. The image data may be a moving image or a still image. As the input device 12, for example, a digital camera, a smartphone with a camera, an in-vehicle camera, etc. can be used. Note that the input device 12 may acquire image data stored in a server or the like through communication.

[0020] The processor 13 is a computer such as a CPU (Central Processing Unit), and controls the entire human body detection device 100 by executing a program prepared in advance. Note that the processor 13 may be a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array). Specifically, the processor 13 executes the human body detection process described later.

[0021] The memory 14 is composed of a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The memory 14 stores various programs executed by the processor 13. Also, the memory 14 is used as a working memory during the execution of various processes by the processor 13.

[0022] The recording medium 15 is a non-volatile and non-temporary recording medium such as a disk-shaped recording medium or a semiconductor memory, and is configured to be detachable from the human body detection device 100. The recording medium 15 stores various programs executed by the processor 13. When the human body detection device 100 executes various processes, the programs recorded on the recording medium 15 are loaded into the memory 14 and executed by the processor 13.

[0023] The DB 16 stores the image data input from an external device including the input device 12. Specifically, the DB 16 stores the image data used for the learning of the human body detection device 100. The display unit 17 is, for example, a liquid crystal display device or a projector, and displays the detection result by the human body detection device 100. In addition to the above, the human body detection device 100 may be provided with input devices such as a keyboard and a mouse for the user to give instructions and inputs.

[0024] [Functional configuration] FIG. 3 is a block diagram showing the functional configuration of the human body detection device 100 according to the first embodiment. The human body detection device 100 functionally includes a head rectangle estimation unit 21, a human body rectangle estimation unit 22, a human body rectangle estimation unit 31, and a head rectangle estimation unit 32.

[0025] (Overall processing) The image data is input to each of the head rectangle estimation unit 21, the human body rectangle estimation unit 22, the human body rectangle estimation unit 31, and the head rectangle estimation unit 32. The image data is input from the input device 12 and may be a color image or a grayscale image. Also, the image size does not matter.

[0026] The head rectangle estimation unit 21 receives the image data, calculates the image feature amount, estimates the head, and calculates the coordinates of the bounding box (referred to as "rectangle") of the head (hereinafter referred to as "head rectangle coordinates") and the reliability representing the head-likeness (hereinafter referred to as "head reliability"). Note that the head rectangle coordinates and the head reliability are collectively referred to as "head rectangle information".

[0027] Based on the image data and the head rectangle information output by the head rectangle estimation unit 21, the human body rectangle estimation unit 22 estimates a human body rectangle indicating the human body corresponding to the head, and calculates the coordinates of the rectangle of the human body (hereinafter referred to as "human body rectangle coordinates") and the reliability representing the humanity (hereinafter referred to as "human body reliability"). The human body rectangle coordinates and the human body reliability are collectively referred to as "human body rectangle information".

[0028] In this way, a pair of a head rectangle and a human body rectangle (hereinafter referred to as a "head-human body pair") is generated from the image data and sent to the integration unit 40. The head-human body pair generated by the head rectangle estimation unit 21 and the human body rectangle estimation unit 22 is also referred to as a "head-human body pair derived from the head". The human body rectangle estimation unit 22 outputs the number of head-derived head-human body pairs that can be estimated from the image data to the integration unit 40.

[0029] On the other hand, the human body rectangle estimation unit 31 receives the image data, calculates the image feature amount, estimates the human body, and outputs the human body rectangle information including the human body rectangle coordinates and the human body reliability. Based on the image data and the human body rectangle information output by the human body rectangle estimation unit 31, the head rectangle estimation unit 32 estimates a head rectangle indicating the head corresponding to the human body, and calculates the head rectangle information including the head rectangle coordinates and the head reliability. In this way, a head-human body pair is generated from the image data and sent to the integration unit 40. The head-human body pair generated by the human body rectangle estimation unit 31 and the head rectangle estimation unit 32 is also referred to as a "head-human body pair derived from the human body". The head rectangle estimation unit 32 outputs the number of human body-derived head-human body pairs that can be estimated from the image data to the integration unit 40.

[0030] The integration unit 40 performs integration processing on a plurality of head-body pairs including the head-body pair derived from the head input from the human body rectangle estimation unit 22 and the head-body pair derived from the human body input from the head rectangle estimation unit 32. Specifically, the integration unit 40 deletes overlapping head-body pairs based on the relationship between the head rectangles or the relationship between the head rectangles and the relationship between the human body rectangles for the input plurality of head-body pairs. At this time, the integration unit 40 performs the integration process so as to preferentially leave the head-body pair derived from the human body rather than the head-body pair derived from the head. The details will be described later. As a result, the human body rectangles detected repeatedly for the same person are integrated, and the human body rectangles corresponding to different persons remain. The integration unit 40 outputs the human body rectangle information of the remaining human body rectangles.

[0031] Note that the head rectangle estimated by the head rectangle estimation unit 21 is an example of the first partial rectangle, the human body rectangle estimated by the human body rectangle estimation unit 22 is an example of the first human body rectangle, the human body rectangle estimated by the human body rectangle estimation unit 31 is an example of the second human body rectangle, and the head rectangle estimated by the head rectangle estimation unit 32 is an example of the second partial rectangle.

[0032] (Generation of head-body pair derived from head) Next, a method for generating a head-body pair derived from the head by the head rectangle estimation unit 21 and the human body rectangle estimation unit 22 will be described.

[0033] (1) First method In the first method, the head rectangle estimation unit 21 estimates the head from the image data and outputs the head rectangle information, and the human body rectangle estimation unit 22 generates the human body rectangle information from the image data and the head rectangle information.

[0034] The estimation of the head rectangle by the head rectangle estimation unit 21 is not limited to a specific estimation process. For example, a matching method using a sliding window or a method using machine learning such as deep learning can be used.

[0035] Next, an example using machine learning will be described. The head rectangle estimation unit 21 first extracts image feature amounts using a neural network. Examples of neural networks include VGG and ResNet. The image input to the neural network is reduced by a plurality of convolutional processes, and image feature amounts are generated in the process. The image feature amount is three-dimensional information. FIG. 4 shows an example of the image feature amount and the anchor box. Now, let the size of the image feature amount generated when an image having an image size of H×W is input be h×w×c. Here, "h" and "w" are the vertical and horizontal sizes of the image feature amount, and "c" is the number of classes. The head rectangle estimation unit 21 estimates the position of the head rectangle using the extracted image feature amount and the anchor box. The anchor box is a group of rectangles with a predefined aspect ratio and number, and is scattered infinitely in the image. Actually, a fixed number of anchor boxes are arranged at each point in the spatial direction of the image feature amount. The spatial size of the image feature amount with a size of h×w×c is h×w. If four anchor boxes are arranged at each point, a total of h×w×4 anchor boxes are arranged.

[0036] Next, the head rectangle estimation unit 21 outputs the head rectangle coordinates and the head confidence respectively using the image features of each anchor box. The image features of each anchor box refer to the amount of information of 1×1×c where the anchor box is arranged. Convolutional processing is performed again on the amount of information of 1×1×c to estimate the head rectangle coordinates and the head confidence. The estimation here becomes possible by the neural network learning rectangle position regression and category classification. Specifically, the position and category of the head are given as correct data, the error between the correct data and the estimation result is calculated by a loss function, and the neural network is corrected so that the error becomes small. This process is repeated, and when the specified number of times is reached, the learning is completed. The head rectangle estimation unit 21 estimates the head rectangle coordinates and the head confidence from the image using the thus obtained learned neural network, and supplies them to the human body rectangle estimation unit 22 as head rectangle information.

[0037] Based on the head rectangle information generated by the head rectangle estimation unit 21, the human body rectangle estimation unit 22 generates human body rectangle information corresponding to the head. First, the human body rectangle estimation unit 22 determines a human body candidate region corresponding to the head indicated by the head rectangle information. Here, the "human body candidate region" is a region in the image where a human body is considered to exist. Generally, for the detected head, the human body of the same person often exists in the image. Also, based on the physical characteristics of humans and the position of the head, the region of the human body can be estimated to a certain extent. For example, it is obtained as prior knowledge that the body is often located below the head relative to the head.

[0038] Figure 5 shows an example of a method for determining a human body candidate region. In this example, based on the prior knowledge that there is a body under the head and the human body is often a vertically long rectangle, a mechanical model is created to calculate the human body candidate region from the head position. Specifically, in this example, the width of the human body is set to 3 times the width of the head, and the height of the human body is set to 2.5 times the width of the human body. If the head rectangle coordinates given from the head rectangle estimation unit 21 include the coordinates (x, y) of the upper left point of the head rectangle, the width and height (w H , h H ) of the head rectangle, then the human body rectangle estimation unit 22 sets the coordinates of the upper left point of the human body candidate region to (x - w H , y), and determines the width and height to be (3w H , 3w H / 0.4).

[0039] Next, as shown in Figure 6, the human body rectangle estimation unit 22 extracts the feature amount corresponding to the human body candidate region from the feature amount of the entire image obtained from the image data to obtain the feature amount of the human body candidate region. Next, the human body rectangle estimation unit 22 estimates the human body rectangle and the human body reliability from the obtained feature amount of the human body candidate region, and outputs them as human body rectangle information.

[0040] The human body rectangle estimation process by the human body rectangle estimation unit 22 is not limited to a specific estimation process. Similar to the head rectangle estimation unit 21, there are methods such as a matching method using a sliding window and a method using machine learning such as deep learning. Specifically, in the method using machine learning, the human body rectangle estimation unit 22 inputs the extracted feature amounts into a CNN (Convolutional Neural Network) and outputs the human body rectangle coordinates and the human body reliability. This is performed in the same way as the head rectangle estimation unit 21, where the neural network learns the regression and category classification problems of the human body candidate region, and estimates the human body rectangle coordinates and the human body reliability using the learned neural network. In this way, the head - human body pair obtained by the first method is output to the integration unit 40.

[0041] (2) The second method In the above - mentioned first method, the head rectangle estimation unit 21 estimates the head rectangle from the image data, and the human body rectangle estimation unit 22 estimates the human body candidate region based on the position of the head rectangle. In contrast, in the second method, the head rectangle estimation unit 21 estimates the center position of the human body in addition to the head rectangle, and the human body rectangle estimation unit 22 estimates the human body candidate region using the position of the head rectangle and the center position of the human body.

[0042] Specifically, the head rectangle estimation unit 21 calculates the image feature amounts from the image data, estimates the head rectangle coordinates and the head reliability, and also estimates the center position of the human body to which the head belongs. Here, the estimation process of the head rectangle can use the same method as the first method. Also, the estimation of the center position of the human body is made possible by giving the center position of the human body as correct data and having the neural network learn the regression problem. That is, information in which the head rectangle and the human body rectangle of the same person are paired is given as correct data to learn the neural network. Then, the image data is input into the learned neural network to estimate the center position of the human body.

[0043] The human body rectangle estimation unit 22 receives the head rectangle coordinates, head reliability, and the center position of the human body to which the head belongs from the head rectangle estimation unit 21, and estimates a human body candidate region where the human body of the person having the head is likely to exist. Specifically, assuming that the head rectangle coordinates output from the head rectangle estimation unit 21 include the width and height (w H , h H ) of the head rectangle, the human body rectangle estimation unit 22 estimates a rectangle with a width and height of (3w H , 3w H / 0.4) and centered at the center position of the human body received from the head rectangle estimation unit 21 as the human body candidate region. Then, the human body rectangle estimation unit 22 outputs the feature amount of the human body candidate region by cutting out the feature amount corresponding to the human body candidate region from the feature amount of the entire image obtained from the image data. And the human body rectangle estimation unit 22 estimates a human body rectangle based on the human body candidate region and outputs the human body rectangle coordinates and human body reliability as human body rectangle information in the same manner as the first method.

[0044] Note that the method for generating the head-human body pair derived from the head is not limited to the above method, and other methods may be used.

[0045] (Generation of Head-Human Body Pair Derived from Human Body) Next, a method for generating a head-human body pair derived from a human body by the human body rectangle estimation unit 31 and the head rectangle estimation unit 32 will be described. The human body rectangle estimation unit 31 calculates an image feature amount from the image data, estimates the human body rectangle coordinates and human body reliability, and outputs them to the head rectangle estimation unit 32 as human body rectangle information.

[0046] The head rectangle estimation unit 32 estimates a head candidate region from the human body rectangle based on the positional relationship between a human head and a human body, in the same way as the first method in the method for generating a head-human body pair derived from the head described above. Specifically, the head rectangle estimation unit 32 estimates a head candidate region, which is a region where a head is considered likely to exist, based on the human body rectangle coordinates input from the human body rectangle estimation unit 31 and the positional relationship between the head and the human body shown in FIG. 5. Then, the head rectangle estimation unit 32 extracts the feature amount of the obtained head candidate region from the feature amount of the entire image obtained from the image data, and estimates the head rectangle coordinates and the head reliability. Note that the head rectangle estimation unit 32 can estimate the head rectangle coordinates and the head reliability by using a matching method using a sliding window or a method using machine learning such as deep learning, in the same way as the human body rectangle estimation unit 22.

[0047] Note that the method for generating a head-human body pair derived from a human body is not limited to the above method, and other methods may be used.

[0048] (Integration process) Next, the integration process by the integration unit 40 will be described in detail. The integration unit 40 removes duplicate pairs from a plurality of head-human body pairs input from the human body rectangle estimation unit 22 and the head rectangle estimation unit 32. Hereinafter, two embodiments of the integration process will be described.

[0049] (1) First embodiment In the first embodiment of the integration process, the integration unit 40 determines whether the input head-human body pair is derived from a human body, and retains the head-human body pair derived from a human body. Further, when the input head-human body pair is derived from a head, the integration unit 40 deletes the head-human body pair if it is of the same person as the head-human body pair derived from a human body, and retains it if it is of a different person.

[0050] Figure 7 is a flowchart of the integration process of the first embodiment. This process is realized by the processor 13 shown in FIG. 2 executing a program prepared in advance. First, the integration unit 40 acquires one of a plurality of head / human body pairs as a head / human body pair to be processed (hereinafter also referred to as "processing target pair"), and determines whether the processing target pair is derived from a human body (step S21). When the processing target pair is derived from a human body (step S21: Yes), the integration unit 40 adopts the processing target pair as something to be left (step S23).

[0051] On the other hand, when the processing target pair is not derived from a human body (step S21: No), that is, when the processing target pair is derived from a head, the integration unit 40 determines whether the processing target pair is of the same person as another head / human body pair derived from a human body (step S22). Specifically, the integration unit 40 compares the processing target pair with all other head / human body pairs derived from a human body, and determines whether the processing target pair is of the same person as any of the other head / human body pairs derived from a human body. Hereinafter, the determination of whether it is of the same person is also referred to as "same person determination".

[0052] When the processing target pair is of the same person as any of the head / human body pairs derived from a human body (step S22: Yes), since those head / human body pairs are considered to be overlapping, the integration unit 40 rejects the processing target pair (step S24). On the other hand, when the processing target pair is not of the same person as any of the head / human body pairs derived from a human body (step S22: No), since those head / human body pairs are considered not to be overlapping, the integration unit 40 adopts the processing target pair (step S23). As a result, a head / human body pair derived from a head that does not overlap with other head / human body pairs derived from a human body is adopted.

[0053] The integration unit 40 sequentially executes the above processing for all of the plurality of head-body pairs for which the above processing has been input, and outputs the head-body pair adopted in step S23 as an integration result. In this way, the integration unit 40 can delete overlapping head-body pairs. Further, according to the first embodiment, all of the head-body pairs derived from the human body are adopted, and the head-body pairs derived from the head are adopted only when there is no head-body pair derived from the human body that overlaps with it. Therefore, the head-body pairs derived from the human body with higher accuracy are preferentially adopted, and the accuracy of human body detection can be ensured.

[0054] In the comparison in the same person determination in step S22, the comparison target may be only the head rectangles, or both the head rectangles and the body rectangles. When comparing only the head rectangles, the integration unit 40 compares only the head rectangles of the pair to be processed and the head-body pair derived from a certain human body, and determines whether they are of the same person. On the other hand, when comparing both the head rectangles and the body rectangles, the integration unit 40 compares the head rectangles and the body rectangles of the pair to be processed and the head-body pair derived from a certain human body, respectively, and determines whether they are of the same person. By comparing both the head rectangles and the body rectangles, the accuracy of the same person determination can be further improved.

[0055] In addition, as a method of comparison in the same person determination in step S22, the size of the overlap between rectangles or the proximity of the center coordinates of rectangles may be used. This enables the same person determination by performing simple area or distance calculations. For example, when only the head rectangles are the comparison targets and the size of the overlap between rectangles is used, the integration unit 40 determines that the pair of processing targets and the head·human body pair derived from a certain human body are of the same person if the overlap between the head rectangle of the pair of processing targets and the head rectangle of the head·human body pair derived from a certain human body is equal to or greater than a predetermined threshold, and determines that the pair of processing targets and the head·human body pair derived from a certain human body are not of the same person if the overlap is less than the predetermined threshold. Also, when both the head rectangles and the human body rectangles are the comparison targets and the size of the overlap between rectangles is used, the integration unit 40 determines that the pair of processing targets and the head·human body pair derived from a certain human body are of the same person if both the overlap of the head rectangles and the overlap of the human body rectangles are equal to or greater than a predetermined threshold for the head rectangle of the pair of processing targets and the head·human body pair derived from a certain human body, and determines that the pair of processing targets and the head·human body pair derived from a certain human body are not of the same person in other cases.

[0056] On the other hand, when only the head rectangles are the comparison targets and the proximity of the center coordinates of rectangles is used, the integration unit 40 determines that the pair of processing targets and the head·human body pair derived from a certain human body are of the same person if the distance between the center coordinates of the head rectangle of the pair of processing targets and the head rectangle of the head·human body pair derived from a certain human body is less than a predetermined threshold, and determines that the pair of processing targets and the head·human body pair derived from a certain human body are not of the same person if the distance between the center coordinates is equal to or greater than the predetermined threshold. Also, when both the head rectangles and the human body rectangles are the comparison targets and the proximity of the center coordinates of rectangles is used, the integration unit 40 determines that the pair of processing targets and the head·human body pair derived from a certain human body are of the same person if both the distance between the center coordinates of the head rectangle of the pair of processing targets and the head rectangle of the head·human body pair derived from a certain human body and the distance between the center coordinates of the human body rectangle of the pair of processing targets and the human body rectangle of the head·human body pair derived from a certain human body are less than a predetermined threshold, and determines that the pair of processing targets and the head·human body pair derived from a certain human body are not of the same person in other cases.

[0057] (2) Second Embodiment In the second embodiment of the integration process, in addition to the process of the first embodiment, when the pair to be processed is derived from the human body, the integration unit 40 further determines whether the pair to be processed overlaps with other head-body pairs derived from the human body, and adopts it only when there is no overlap.

[0058] FIG. 8 is a flowchart of the integration process of the second embodiment. This process is realized by the processor 13 shown in FIG. 2 executing a program prepared in advance. First, the integration unit 40 acquires one of a plurality of head-body pairs as the pair to be processed, and determines whether the pair to be processed is derived from the human body (step S21). Steps S22 to S24 when the pair to be processed is not derived from the human body are the same as those in the first embodiment shown in FIG. 7, so the description is omitted.

[0059] On the other hand, when the pair to be processed is derived from the human body (step S21: Yes), the integration unit 40 determines whether the pair to be processed is of the same person as other head-body pairs derived from the human body (step S25). In the same-person determination in step S25, the comparison targets may be only the head rectangles, or both the head rectangles and the body rectangles. Also, as a comparison method, the size of the overlap between the rectangles may be used, or the proximity of the center coordinates of the rectangles may be used.

[0060] When the pair to be processed is of the same person as other head-body pairs derived from the human body (step S25: Yes), the pair to be processed is considered to overlap with other head-body pairs that have already been adopted, so the integration unit 40 rejects the pair to be processed (step S26). On the other hand, when the pair to be processed is not of the same person as other head-body pairs derived from the human body (step S25: No), the pair to be processed is considered not to overlap with other head-body pairs that have already been adopted, so the integration unit 40 adopts the pair to be processed (step S23).

[0061] Also in the second embodiment, the integration unit 40 sequentially executes the above processing for all of the plurality of input head-human pairs, and outputs the head-human pair output in step S23 as an integration result. According to the second embodiment, in addition to preferentially adopting head-human pairs derived from the human body over head-human pairs derived from the head, pairs that overlap even among head-human pairs derived from the human body can be deleted, so that the accuracy of human body detection can be further improved.

[0062] [Human body detection process] FIG. 9 is a flowchart of the human body detection process by the human body detection device 100. This process is realized by the processor 13 shown in FIG. 2 executing a program prepared in advance and functioning as each element shown in FIG. 3.

[0063] First, the head rectangle estimation unit 21 estimates a head rectangle from the image data and generates head rectangle information (step S11). Next, the human body rectangle estimation unit 22 estimates a human body rectangle from the head rectangle based on the head rectangle information (step S12). Then, the human body rectangle estimation unit 22 outputs the head rectangle information obtained in step S11 and the human body rectangle information obtained in step S12 to the integration unit 40 as a head-human pair (step S13). In this way, the number of head-human pairs derived from the head that can be estimated from the image data is obtained.

[0064] Similarly, the human body rectangle estimation unit 31 estimates a human body rectangle from the image data and generates human body rectangle information (step S14). Next, the head rectangle estimation unit 32 estimates a head rectangle from the human body rectangle based on the human body rectangle information (step S15). Then, the head rectangle estimation unit 32 outputs the head rectangle information obtained in step S14 and the human body rectangle information obtained in step S15 to the integration unit 40 as a head-human pair (step S16). In this way, the number of head-human pairs derived from the human body that can be estimated from the image data is obtained. Note that either step S11 to S13 or step S14 to S16 may be performed first, or they may be performed in parallel in terms of time.

[0065] Then, the integration unit 40 performs integration processing using the plurality of head-body pairs input in steps S13 and S16 (step S20). The integration processing may be the same as that of the first embodiment or the second embodiment described above. Then, the integration unit 40 outputs the head-body pairs adopted in the integration processing as the detection results of the human body.

[0066] [Modification Example] In the above example, the head rectangle estimation unit 21 and the human body rectangle estimation unit 31 estimate the head and the human body from the image data using an object detection method that uses an anchor box (referred to as "anchor-based"). Instead, an object detection method that does not use an anchor box (referred to as "anchor-free") may be used to detect the head or the human body.

[0067] FIG. 10 schematically shows a method for detecting a head rectangle and a human body rectangle by a detector using CenterNet as an example of an anchor-free object detection method. In the detector, the image data is input into a convolutional network such as a CNN, and a heatmap indicating the probability of the presence of an object is generated. The detector detects the peak position in the heatmap as the center position (keypoint) of the object, and defines the rectangle of the object based on the width and height with respect to the center position.

[0068] Specifically, the head rectangle estimation unit 21 uses a pre-trained network to detect the head center from the image data, generates a heatmap M1 of the head center position, and outputs the head center position 51, the width 52w and the height 52h based on the head center position 51. The head rectangle 53 is defined by the head center position 51, the width 52w and the height 52h based on the head center position. In FIG. 10, for convenience of explanation, the image data includes only one person. However, the heatmap M1 of the head center position generated from the image data including a plurality of persons indicates the head center positions of the plurality of persons included in the image. In this way, the head rectangle estimation unit 21 creates a heatmap M1 indicating the head center positions of the plurality of persons included in the image data. The heatmap of the head center position generated by the head rectangle estimation unit 21 is referred to as the "head heatmap derived from the head".

[0069] Similarly, the human body rectangle estimation unit 31 uses a pre-trained network to detect the human body center from the image data, generates a heatmap M2 of the human body center position, and outputs the human body center position 54, the width 55w and the height 55h based on the human body center position 54. The human body rectangle 56 is defined by the human body center position 53, the width 55w and the height 55h based on the human body center position. In FIG. 10, for convenience of explanation, the image data includes only one person. However, the heatmap M2 of the human body center position generated from the image data including a plurality of persons indicates the human body center positions of the plurality of persons included in the image. In this way, the human body rectangle estimation unit 31 creates a heatmap M2 indicating the human body center positions of the plurality of persons included in the image data. The heatmap of the human body center position generated by the human body rectangle estimation unit 31 is referred to as the "human body heatmap derived from the human body".

[0070] The human body rectangle estimation unit 22 first estimates a human body center candidate region by using the head center position 51 estimated by the head rectangle estimation unit 21 and an offset indicating the positional relationship between the head center position and the human body center position of the same person as illustrated in, for example, FIG. 5. Next, the human body rectangle estimation unit 22 estimates the human body center position corresponding to the head center position estimated by the head rectangle estimation unit 21 by using the estimated human body center candidate region and the heat map of the human body center position generated from the image data. That is, the human body rectangle estimation unit 22 estimates, from the heat map of the human body center position including a plurality of human body center positions, the human body center position corresponding to the human body center candidate region as the human body center position corresponding to the head center position estimated by the head rectangle estimation unit 21, obtains the width and height based on the human body center position, and estimates the human body rectangle. In this way, the human body rectangle estimation unit 22 creates a heat map showing the human body center positions of a plurality of persons included in the image data. The heat map of the human body center position generated by the human body rectangle estimation unit 22 is referred to as the "human body heat map derived from the head".

[0071] Similarly, the head rectangle estimation unit 32 first estimates a head center candidate region by using the human body center position 54 and an offset indicating the positional relationship between the human body center position and the head center position of the same person. Next, the head rectangle estimation unit 32 estimates the head center position corresponding to the human body center position estimated by the human body rectangle estimation unit 31 by using the estimated head center candidate region and the heat map of the head center position generated from the image data. That is, the head rectangle estimation unit 32 estimates, from the heat map of the head center position including a plurality of head center positions, the head center position corresponding to the head center candidate region as the head center position corresponding to the human body center position estimated by the human body rectangle estimation unit 31, obtains the width and height based on the head center position, and estimates the head rectangle. In this way, the head rectangle estimation unit 32 creates a heat map showing the head center positions of a plurality of persons included in the image data. The heat map of the head center position generated by the head rectangle estimation unit 32 is referred to as the "head heat map derived from the human body".

[0072] Note that the heat map of the head center position is an example of the first heat map, and the heat map of the human body center position is an example of the second heat map.

[0073] The integration unit 40 performs integration processing using the heatmap thus obtained. According to this modification example, when performing the same-person determination in the integration processing, the proximity of the center positions of the rectangles can be determined by the distance between the peak positions on the heatmap. This determination can be performed by logical operations on two heatmaps, for example. For instance, in the first embodiment of the integration processing, consider the case where the same-person determination is made in step S22 of FIG. 7. When the comparison targets are the heads, the integration unit 40 compares the head heatmap derived from the head and the head heatmap derived from the human body, and may adopt those where the distance between the head center positions is equal to or greater than a predetermined threshold value. Thus, according to the modification example, it is possible to significantly reduce the amount of calculation compared to the case of calculating the overlap by pairwise comparison of a large number of rectangles and performing the same-person determination.

[0074] [Display example] Next, a display example of the human body rectangles detected by the human body detection device 100 will be described. FIGS. 11(A) and 11(B) show examples of the human body rectangles displayed on the display unit 17. Note that the examples in FIGS. 11(A) and 11(B) are those in which the human body rectangles detected by the human body detection device 100 are superimposed and displayed on an image in which five persons A to E are shown.

[0075] As described above, the human body detection device 100 can distinguish and detect the human body rectangle derived from the head and the human body rectangle derived from the human body. Therefore, the display unit 17 displays the human body rectangle derived from the head and the human body rectangle derived from the human body in a distinguishable manner. Specifically, the human body rectangle derived from the head and the human body rectangle derived from the human body can be distinguished and displayed by the color of the rectangle line, the type of the line, the thickness of the line, the color given inside the rectangle, etc. In the example of FIG. 11(A), the human body rectangle derived from the human body is displayed by a solid line, and the human body rectangle derived from the head is displayed by a dashed line. Since persons A to D have little occlusion, they are detected as the human body rectangles derived from the human body, and since person E has a lot of occlusion, it is detected as the human body rectangle derived from the head.

[0076] Also, for the human body rectangles derived from the head and those derived from the human body, characters, symbols, etc. indicating each of them may be assigned to distinguish between the two. In the example of FIG. 11(B), the character "Head" is attached to the human body rectangle derived from the head, and the character "Body" is attached to the human body rectangle derived from the human body. As shown in FIG. 11(B), the number of detected human body rectangles, that is, the number of persons, may be displayed.

[0077] As described above, since the person detection device 100 of the present embodiment detects by distinguishing between the human body rectangles derived from the head and those derived from the human body, it is possible to distinguish and display the persons detected based on the head and the persons detected based on the human body.

[0078] [Application Example] Next, application examples of the detection results by the human body detection device 100 will be described. The human body rectangle detected by the human body detection device 100 can be used for person verification and authentication (hereinafter collectively referred to as "authentication").

[0079] FIG. 12(A) shows a configuration example of an authentication device that performs person authentication based on the detection result of the human body detection device 100. The authentication device 200 includes a classification unit 201, an authentication unit 202, and a tracking unit 203.

[0080] The human body rectangle information output by the human body detection device 100 as a detection result shall include, in addition to the human body rectangle coordinates and the human body reliability, information indicating whether the human body rectangle is derived from the head or the human body (hereinafter referred to as "derivation information"). As described above, the human body rectangle information derived from the head is considered to have low accuracy compared to the human body rectangle information derived from the human body and is considered inappropriate for use in person authentication. Therefore, the classification unit 201 determines whether the input human body rectangle information is derived from the head or the human body based on the derivation information, and outputs the human body rectangle information derived from the human body to the authentication unit 202. The authentication unit 202 performs person authentication using the human body rectangle information derived from the human body and outputs an authentication result. Examples of person authentication here include face authentication, gait authentication, etc.

[0081] On the one hand, the human body rectangle derived from the head is not suitable for use in person authentication in terms of accuracy, but can be used for tracking a person. Therefore, the classification unit 201 outputs the human body rectangle information derived from the head to the tracking unit 203. The tracking unit 203 performs person tracking processing based on the human body rectangle derived from the head input from the classification unit 201. The tracking unit 203 assigns a person ID for tracking and the like to each person included in the image and performs tracking. The tracking unit 203 continues to track each person using the continuously input human body rectangle derived from the head, and outputs a tracking result including the person ID and the like of each person to the classification unit 201.

[0082] Based on the tracking result by the tracking unit 203, the classification unit 201 monitors the human body rectangle derived from the head detected from each frame image as the human body rectangle of the same person. When a human body rectangle derived from the human body is input for that person, the classification unit 201 outputs the human body rectangle information to the authentication unit 202. As a result, even for a person for whom only a human body rectangle derived from the head can be detected and authentication cannot be performed during a certain period of the moving image, by continuing to track using the head-derived person image, authentication can be performed when a human body rectangle derived from the human body can be detected.

[0083] FIG. 13(A) shows an example of displaying the authentication result obtained by the authentication device 200. The names A to E of the persons obtained by person authentication are assigned to the rectangles of each person. Note that since the authentication device 200 performs authentication based on the human body rectangle derived from the human body, in FIG. 13(A), all the rectangles of the persons are shown as solid-line rectangles indicating that they are derived from the human body.

[0084] FIG. 12(B) shows a configuration example of another authentication device that performs person authentication based on the detection result of the human body detection device 100. The authentication device 200x includes a classification unit 211, and authentication units 212 and 213.

[0085] The authentication device 200x performs authentication for both the human body rectangle derived from the head and the human body rectangle derived from the human body. However, since the human body rectangle derived from the head has lower accuracy compared to the human body rectangle derived from the human body, it is not appropriate to use the same method as in the case of the human body rectangle derived from the human body. Therefore, in the authentication device 200x, an authentication unit 212 that performs authentication using the human body rectangle derived from the human body and an authentication unit 213 that performs authentication using the human body rectangle derived from the head are prepared separately. The authentication units 212 and 213 are configured by, for example, a neural network. The authentication unit 212 is a neural network trained using the human body rectangle derived from the human body as learning data, and the authentication unit 213 is a neural network trained using the human body rectangle derived from the head as learning data. As a result, the authentication unit 213 can perform authentication with a certain accuracy even when a human body rectangle derived from the head is input.

[0086] The classification unit 211 classifies the human body rectangle information input from the human body detection device 100 based on the origin information, outputs the human body rectangle derived from the human body to the authentication unit 212, and outputs the human body rectangle derived from the head to the authentication unit 213. The authentication unit 212 performs authentication based on the human body rectangle derived from the human body and outputs the authentication result. Also, the authentication unit 213 performs authentication based on the human body rectangle derived from the head and outputs the authentication result.

[0087] FIG. 13(B) shows an example of the display of the authentication result obtained by the authentication device 200x. The human body rectangles of each person are assigned the person names A to E obtained by person authentication. In FIG. 13(B), the rectangles of persons A to D authenticated using the human body rectangle derived from the human body are solid-line rectangles indicating that they are derived from the human body, and the rectangle of person E authenticated using the human body rectangle derived from the head is a dashed-line rectangle indicating that it is derived from the head.

[0088] <Second Embodiment> FIG. 14 is a block diagram showing the functional configuration of the human body detection device according to the second embodiment. The human body detection device 70 includes a first partial rectangle estimation means 71, a first human body rectangle estimation means 72, a second human body rectangle estimation means 73, a second partial rectangle estimation means 74, and a human body integration means 75.

[0089] FIG. 15 is a flowchart of the processing by the human body detection device 70 according to the second embodiment. The first partial rectangle estimation means 71 estimates a specific part of the human body from the image and outputs a first partial rectangle including the specific part (step S31). The first human body rectangle estimation means 72 estimates a human body corresponding to the specific part included in the first partial rectangle based on the coordinates of the first partial rectangle and outputs a first human body rectangle including the human body (step S32). The second human body rectangle estimation means 73 estimates a human body from the image and outputs a second human body rectangle including the human body (step S33). The second partial rectangle estimation means 74 estimates a specific part corresponding to the human body included in the second human body rectangle based on the coordinates of the second human body rectangle and outputs a second partial rectangle including the specific part (step S34). The human body integration means 75 integrates overlapping human body rectangles for a plurality of pairs of mutually corresponding partial rectangles and human body rectangles based on at least the relationship between the partial rectangles (step S35).

[0090] According to the human body detection device 70 of the second embodiment, it is possible to provide a human body detection device that is robust against occlusion.

[0091] Some or all of the above embodiments may be described as follows in the following supplementary notes, but are not limited thereto.

[0092] (Supplementary Note 1) First partial rectangle estimation means for estimating a specific part of the human body from the image and outputting a first partial rectangle including the specific part, First human body rectangle estimation means for estimating a human body corresponding to the specific part included in the first partial rectangle based on the coordinates of the first partial rectangle and outputting a first human body rectangle including the human body, Second human body rectangle estimation means for estimating a human body from the image and outputting a second human body rectangle including the human body, Second partial rectangle estimation means for estimating a specific part corresponding to the human body included in the second human body rectangle based on the coordinates of the second human body rectangle and outputting a second partial rectangle including the specific part, For a plurality of pairs of corresponding partial rectangles and human body rectangles, a human body integration means for integrating overlapping human body rectangles based on at least the relationship between the partial rectangles themselves, An information processing apparatus comprising the same.

[0093] (Appendix 2) The first partial rectangle estimation means outputs the coordinates and reliability of the first partial rectangle, The first human body rectangle estimation means outputs the coordinates and reliability of the first human body rectangle, The second human body rectangle estimation means outputs the coordinates and reliability of the second human body rectangle, The information processing apparatus according to Appendix 1, wherein the second partial rectangle estimation means outputs the coordinates and reliability of the second partial rectangle.

[0094] (Appendix 3) The information processing apparatus according to Appendix 1 or 2, wherein the human body integration means integrates overlapping human body rectangles based on the overlapping ratio between the partial rectangles.

[0095] (Appendix 4) The information processing apparatus according to Appendix 1 or 2, wherein the human body integration means integrates overlapping human body rectangles based on the distance between the central positions of the partial rectangles.

[0096] (Appendix 5) The information processing apparatus according to Appendix 1 or 2, wherein the human body integration means integrates overlapping human body rectangles based on the relationship between the partial rectangles and the relationship between the human body rectangles for the plurality of pairs.

[0097] (Appendix 6) The information processing apparatus according to Appendix 5, wherein the human body integration means integrates overlapping human body rectangles based on the overlapping ratio between the partial rectangles and the overlapping ratio between the human body rectangles.

[0098] (Appendix 7) The information processing apparatus according to Appendix 5, wherein the human body integration means integrates overlapping human body rectangles based on the distance between the central positions of the partial rectangles and the distance between the central positions of the human body rectangles.

[0099] (Appendix 8) The human body integration means is the information processing apparatus according to any one of Appendices 1 to 7, which rejects the first human body rectangle and adopts the second human body rectangle when the first human body rectangle and the second human body rectangle overlap with each other.

[0100] (Appendix 9) The human body integration means is the information processing apparatus according to Appendix 8, which rejects the second human body rectangle with a lower reliability and adopts the second human body rectangle with a higher reliability when the second human body rectangles overlap with each other.

[0101] (Appendix 10) The first partial rectangle estimation means outputs the coordinates of the center position of the first partial rectangle on the first heatmap indicating the probability of the presence of a specific part of the human body in the image. The first human body rectangle estimation means outputs the coordinates of the center position of the second human body rectangle on the second heatmap indicating the probability of the presence of a human body in the image. The second human body rectangle estimation means outputs the coordinates of the center position of the second human body rectangle on the second heatmap. The second partial rectangle estimation means outputs the coordinates of the center position of the second partial rectangle on the first heatmap. The integration means is the information processing apparatus according to Appendix 1, which determines the relationship between the partial rectangles using the first heatmap.

[0102] (Appendix 11) The first partial rectangle estimation means outputs the coordinates of the center position of the first partial rectangle on the first heatmap indicating the probability of the presence of a specific part of the human body in the image. The first human body rectangle estimation means outputs the coordinates of the center position of the second human body rectangle on the second heatmap indicating the probability of the presence of a human body in the image. The second human body rectangle estimation means outputs the coordinates of the center position of the second human body rectangle on the second heatmap. The second partial rectangle estimation means outputs the coordinates of the center position of the second partial rectangle on the first heatmap. The integration means is an information processing apparatus according to appended note 4 that determines the relationship between the partial rectangles using the first heatmap and determines the relationship between the human body rectangles using the second heatmap.

[0103] (Appended note 12) The information processing apparatus according to any one of appended notes 1 to 11, comprising display means for superimposing and displaying the first human body rectangle and the second human body rectangle adopted by the integration means on the image in a distinguishable manner.

[0104] (Appended note 13) Estimate a specific part of the human body from the image, and output a first partial rectangle including the specific part. Based on the coordinates of the first partial rectangle, estimate the human body corresponding to the specific part included in the first partial rectangle, and output a first human body rectangle including the human body. Estimate the human body from the image, and output a second human body rectangle including the human body. Based on the coordinates of the second human body rectangle, estimate the specific part corresponding to the human body included in the second human body rectangle, and output a second partial rectangle including the specific part. An information processing method for integrating overlapping human body rectangles based on at least the relationship between the partial rectangles for a plurality of pairs of mutually corresponding partial rectangles and human body rectangles.

[0105] (Appended note 14) Estimate a specific part of the human body from the image, and output a first partial rectangle including the specific part. Based on the coordinates of the first partial rectangle, estimate the human body corresponding to the specific part included in the first partial rectangle, and output a first human body rectangle including the human body. Estimate the human body from the image, and output a second human body rectangle including the human body. Based on the coordinates of the second human body rectangle, estimate the specific part corresponding to the human body included in the second human body rectangle, and output a second partial rectangle including the specific part. A recording medium storing a program that causes a computer to execute a process of integrating overlapping human body rectangles based on at least the relationship between the partial rectangles for a plurality of pairs of corresponding partial rectangles and human body rectangles.

[0106] As described above, this disclosure has been explained with reference to the embodiments and examples, but this disclosure is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of this disclosure within the scope of this disclosure.

Explanation of Reference Numerals

[0107] 13 Processor 21, 32 Head Rectangle Estimation Unit 22, 31 Human Body Rectangle Estimation Unit 40 Integration Unit 100 Human Body Detection Device 200, 200x Authentication Device 201, 211 Classification Unit 202, 212, 213 Authentication Unit 203 Tracking Unit

Claims

1. First partial rectangle estimation means for estimating the head of a human body from an image and outputting a first partial rectangle including the head; First human body rectangle estimation means for estimating a human body corresponding to the head included in the first partial rectangle based on the coordinates of the first partial rectangle using prior knowledge regarding the positional relationship between the human body and the head, and outputting a first human body rectangle including the human body; First pair generation means for generating a first pair including the first partial rectangle and the first human body rectangle; Second human body rectangle estimation means for estimating a human body from the image and outputting a second human body rectangle including the human body; Second partial rectangle estimation means for estimating a head corresponding to the human body included in the second human body rectangle based on the coordinates of the second human body rectangle using the prior knowledge, and outputting a second partial rectangle including the head; Second pair generation means for generating a second pair including the second partial rectangle and the second human body rectangle; Integration means for integrating a plurality of pairs of partial rectangles and human body rectangles including the first pair and the second pair; comprising; The integration means determines whether the input pair is the first pair or the second pair. If the input pair is the second pair, the second pair is adopted. If the input pair is the first pair, it is determined whether the first pair is for the same person as the second pair. If it is for the same person, the first pair is rejected. If it is not for the same person, the first pair is adopted. An information processing apparatus.

2. The information processing apparatus according to claim 1, wherein when the integration means includes two second pairs corresponding to the same person among the plurality of pairs, one of the two second pairs is adopted and the other is rejected.

3. The information processing apparatus according to claim 1, wherein in two pairs included in the plurality of pairs, when the overlap between the partial rectangles is equal to or greater than a first threshold value, or when the distance between the center coordinates of the partial rectangles is less than a second threshold value, the two pairs are determined to be pairs corresponding to the same person.

4. An information processing method executed by a computer, comprising: estimating the head of a human body from an image and outputting a first partial rectangle including the head; estimating a human body corresponding to the head included in the first partial rectangle based on the coordinates of the first partial rectangle using prior knowledge regarding the positional relationship between the human body and the head, and outputting a first human body rectangle including the human body; Generate a first pair including the first partial rectangle and the first human body rectangle. Estimate a human body from the image and output a second human body rectangle including the human body. Using the prior knowledge, estimate the head corresponding to the human body included in the second human body rectangle based on the coordinates of the second human body rectangle, and output a second partial rectangle including the head. Generate a second pair including the second partial rectangle and the second human body rectangle. Execute an integration process for integrating a plurality of pairs of partial rectangles and human body rectangles including the first pair and the second pair. The integration process determines whether the input pair is the first pair or the second pair. If the input pair is the second pair, the second pair is adopted. If the input pair is the first pair, it is determined whether the first pair is of the same person as the second pair. If it is of the same person, the first pair is rejected. If it is not of the same person, the first pair is adopted. An information processing method.

5. Estimate the head of a human body from an image and output a first partial rectangle including the head. Using the prior knowledge regarding the positional relationship between the human body and the head, estimate the human body corresponding to the head included in the first partial rectangle based on the coordinates of the first partial rectangle, and output a first human body rectangle including the human body. Generate a first pair including the first partial rectangle and the first human body rectangle. Estimate a human body from the image and output a second human body rectangle including the human body. Using the prior knowledge, estimate the head corresponding to the human body included in the second human body rectangle based on the coordinates of the second human body rectangle, and output a second partial rectangle including the head. Generate a second pair including the second partial rectangle and the second human body rectangle. Cause a computer to execute an integration process for integrating a plurality of pairs of partial rectangles and human body rectangles including the first pair and the second pair. The integration process determines whether the input pair is the first pair or the second pair. If the input pair is the second pair, the second pair is adopted. If the input pair is the first pair, it is determined whether the first pair is of the same person as the second pair. If it is of the same person, the first pair is rejected. If it is not of the same person, the first pair is adopted. A program.

Images