Apparatus and method for determining human attributes

Abstract

To provide a personal attribute determination device and a method capable of improving determination accuracy for a personal attribute.SOLUTION: A personal attribute determination device 1 includes: a camera 100 which photographs a prescribed photographing range containing a moving route R of a person being an attribute determination object; a person image extracting unit 320 which extracts a person image from images photographed for the same person by the camera 100 at a plurality of positions along the moving route R; an image classifying unit 330; a representative image selection unit 340; a tentative attribute determination unit 350 which determines tentative attributes of persons by using each of the plurality of person images selected by the representative image selection unit 340; a weight value setting unit 360 which finally determines a person attribute by weighting the plurality of tentative attributes by using weight values respectively corresponding to the plurality of positions; and a final attribute determination unit 370.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a person attribute determination device and method for determining the attributes of a person included in a captured image.

Background Art

[0002] Conventionally, an image of the same person is extracted from a plurality of frames obtained by imaging with a camera, and for each of the extracted plurality of images, a score indicating the likelihood of the person's attributes and a reliability corresponding to an event (position, action, posture, etc.) that affects the recognition of the attributes are obtained, and the values obtained by multiplying these are added for each frame, and an attribute determination device has been known that finally determines the attribute with the largest addition result as the attribute of this person (for example, see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, in the attribute determination device disclosed in Patent Document 1 described above, an image of the same person moving within the imaging range of the camera is extracted over a plurality of frames, and for each image, an attribute and a reliability are obtained to determine the most likely person attribute. For this reason, when a high-reliability image is included among the plurality of images, the accuracy of the finally determined attribute is high, but when only low-reliability images are included, the accuracy of the attribute is low, and there is a problem that there may be variations in accuracy.

[0005] The present invention has been created in view of such points, and its object is to provide a person attribute determination device and method capable of improving the determination accuracy of person attributes. [Means for solving the problem]

[0006] To solve the above-mentioned problems, the person attribute determination device of the present invention comprises: an imaging means for imaging a predetermined imaging range that includes the movement path of a person whose attributes are to be determined; a person image extraction means for extracting person images corresponding to a person from images captured by the imaging means at multiple locations along the movement path for the same person; a provisional attribute determination means for determining the provisional attributes of a person using each of the multiple person images extracted by the person image extraction means; and a final attribute determination means for finally determining the attributes of a person by weighting the multiple provisional attributes determined by the provisional attribute determination means using weight values ​​corresponding to each of the multiple locations. Each of the multiple locations corresponds to one of several pre-set postures when the same person moves along a designated path. ru.

[0007] Furthermore, the method for determining a person's attributes according to the present invention is: A method for determining a person's attributes performed in a person attribute determination device comprising an imaging means, a person image extraction means, a provisional attribute determination means, and a final attribute determination means, The system comprises the steps of: capturing an image of a predetermined imaging range that includes the movement path of a person to be subject to attribute determination using an imaging means; extracting a person image corresponding to the person from images captured by the imaging means at multiple locations along the movement path for the same person using a person image extraction means; determining the person's provisional attributes using each of the multiple person images extracted by the person image extraction means using a provisional attribute determination means; and finally determining the person's attributes using a final attribute determination means by weighting the multiple provisional attributes determined by the provisional attribute determination means using weight values ​​corresponding to each of the multiple locations. Each of the multiple locations corresponds to one of several pre-set postures when the same person moves along a designated path. ru.

[0008] Since the movement path of the person whose attributes are to be determined is known, attribute determination can be performed on images of the person taken along this movement path. Furthermore, since the person's attributes can be comprehensively determined using images of the person corresponding to multiple locations along the movement path, it is possible to improve the accuracy of the determined person's attributes.

[0009] Furthermore, it is desirable that the weight values ​​mentioned above be set based on the posture of the person at each of the multiple locations. When determining a person's attributes, the accuracy of the determination can vary greatly depending on the person's posture. Therefore, by setting weight values ​​that take posture into account, the accuracy of attribute determination can be improved.

[0010] Furthermore, it is desirable that the weight values ​​mentioned above be larger when the person is facing forward and decrease as the person moves away from the front. The accuracy of attribute determination is considered to be highest when the person is facing forward, that is, when the person's face is visible. Therefore, by setting weight values ​​that take into account whether or not the person is facing forward, the accuracy of attribute determination can be improved.

[0011] Furthermore, it is desirable that the weight values ​​mentioned above increase when the size of the corresponding person image is large, and decrease as the size decreases. Even with the same person image, a larger image with a higher number of constituent pixels allows for more accurate identification of the person's actions and expressions, thus improving the accuracy of attribute determination. Therefore, by setting weight values ​​that take into account the size of the person image, the accuracy of attribute determination can be improved.

[0012] Furthermore, along the movement path within the imaging range described above, multiple detection frames are set corresponding to each of the multiple locations, and it is desirable that the person image extraction means focuses on the centroid of the person image and extracts person images in which the centroid is included in each of the multiple detection frames as person images to be determined for provisional attribute. This makes it clear which person images will be extracted and makes it easy to identify the person images to be extracted.

[0013] Furthermore, it is desirable that the person image extraction means described above extracts the person image whose center of gravity is closest to the center of the detection frame. This makes it possible to fix the position on the movement path for extracting person images, and since the posture of the person at those positions will also be similar, it is possible to maintain a high level of accuracy in determining attributes. [Brief explanation of the drawing]

[0014] [Figure 1] It is a diagram showing the configuration of a person attribute determination device according to an embodiment. [Figure 2] It is a diagram showing an installation example of a camera. [Figure 3] It is a diagram showing the configuration of an in-vehicle device. [Figure 4] It is a diagram showing the configuration of a person attribute determination server. [Figure 5] It is a flowchart showing an operation procedure for determining the person attribute of a passenger boarding a bus. [Figure 6] It is an explanatory diagram of cutting out a partial image including a person. [Figure 7] It is an explanatory diagram of posture classification of a person. [Figure 8] It is an explanatory diagram of representative image selection. [Figure 9] It is a diagram showing a determination result of a person attribute calculated using specific values. [Figure 10] It is a diagram showing a determination example when the number of representative images is small. [Figure 11] It is a diagram showing another example of performing person attribute determination.

Mode for Carrying Out the Invention

[0015] Hereinafter, a person attribute determination device according to an embodiment to which the present invention is applied will be described with reference to the drawings.

[0016] FIG. 1 is a diagram showing the configuration of a person attribute determination device according to an embodiment. As shown in FIG. 1, the person attribute determination device 1 according to the present embodiment includes a camera 100, an in-vehicle device 200, and a person attribute determination server 300.

[0017] The camera 100 captures a predetermined imaging range including the movement path of a person who is an object of attribute determination at a predetermined time interval. It is not necessary for the camera 100 to be one, and the imaging range including the movement path may be captured by a plurality of cameras.

[0018] Figure 2 shows an example of camera 100 installation. In the example shown in Figure 2, in order to determine the personal attributes (gender, age, etc.) of passengers using the bus 400, camera 100 is mounted near the ceiling opposite the entrance 410 of the bus 400, in a orientation that allows it to capture the entire body of a passenger moving along this path R, as the passenger moves from the entrance 410 to the exit 420.

[0019] The in-vehicle device 200 performs predetermined processing on the image data captured by the camera 100 and then transmits it to the person attribute determination server 300.

[0020] Figure 3 shows the configuration of the in-vehicle unit 200. As shown in Figure 3, the in-vehicle unit 200 is composed of an image quality adjustment unit 210, a mosaic processing unit 220, and a communication unit 230. The image quality adjustment unit 210 performs brightness correction (dimming) on ​​the image data input from the camera 100 to adjust the image data to a certain brightness. The mosaic processing unit 220 performs mosaic processing on partial images that have features to identify a person if the captured image includes a person. For example, mosaic processing is performed on a predetermined range including the person's eyes. The image data after this mosaic processing is sent from the communication unit 230 to the person attribute determination server 300.

[0021] The communication unit 230 and the person attribute determination server 300 are connected via a network line such as the Internet. It should be noted that the transmission and reception of image data via this network line does not necessarily have to be in real time; image data captured by the bus 400 while it is in motion can be stored in the onboard unit 200 and then transmitted simultaneously to the person attribute determination server 300 when the bus returns to a bus terminal or similar location.

[0022] The person attribute determination server 300 determines the attributes of a person included in an image based on the image data sent from the in-vehicle device 200. These attributes include gender and age. For example, gender may be either male or female, and age may be determined to be one of the following: infant / elementary school student / junior high / high school student / university student / working adult in their 20s / working adult in their 30s-40s / working adult in their 50s-60s / elderly. The age classification can be broad, such as child / adult, or other classifications, depending on the purpose for which the determined person attributes will be used.

[0023] Figure 4 shows the configuration of the person attribute determination server 300. As shown in Figure 4, the person attribute determination server 300 is composed of a communication unit 310, a person image extraction unit 320, an image classification unit 330, a representative image selection unit 340, a provisional attribute determination unit 350, a weight value setting unit 360, and a final attribute determination unit 370. Each component other than the communication unit 310 can be realized by executing a predetermined operation program stored in memory on a processor such as a CPU. These operations will be described later.

[0024] The camera 100 described above corresponds to the imaging means, the person image extraction unit 320, the image classification unit 330, and the representative image selection unit 340 correspond to the person image extraction means, the provisional attribute determination unit 350 corresponds to the provisional attribute determination means, and the weight value setting unit 360 and the final attribute determination unit 370 correspond to the final attribute determination means.

[0025] The person attribute determination device 1 of this embodiment has the above configuration, and its operation will now be described.

[0026] Figure 5 is a flowchart showing the procedure for determining the personal attributes of passengers boarding bus 400, and mainly illustrates the operation procedure of the personal attribute determination server 300.

[0027] When video image data obtained by capturing images of passengers is sent from the in-vehicle device 200, the communication unit 310 receives it (step 100).

[0028] Next, the person image extraction unit 320 acquires multiple consecutive image frames corresponding to the same person from the received image data, and extracts images of the same person (person images) contained in each of these multiple image frames (step 102). This image extraction is performed by cutting out a partial image demarcated by the smallest rectangular frame that includes the entire body of the person of interest. Figure 6 is an explanatory diagram of the partial image cutting out that includes a person. As shown in Figure 6, a vertically elongated rectangle K is set so that the person image is inscribed within it, and the partial image contained in this rectangle K is cut out.

[0029] Next, the image classification unit 330 classifies the multiple partial images corresponding to the multiple extracted frames, focusing on the posture of the person image contained in each partial image (step 104). For example, it classifies them into four types: front (before boarding), front (after boarding), side, and back.

[0030] While these four types of posture classifications could be performed by analyzing the posture of each person image, in this embodiment, the classification process is simplified by identifying the position of the centroid (center) of the rectangular partial image containing the person image.

[0031] Figure 7 is an explanatory diagram of the classification of human posture. As shown in Figure 7, in this embodiment, four classification frames A, B, C, and D are set up to classify human images by posture, corresponding to the number of postures to be classified. The image classification unit 330 classifies the posture of a human image by checking which of these four classification frames A, B, C, or D the centroid (center) p (Figure 6) of a partial image delimited by a rectangle K inscribed within the human image is contained within.

[0032] Specifically, if the center of gravity p is in classification category A, the posture is classified as "facing forward (before boarding)". If the center of gravity p is in classification category B, the posture is classified as "facing forward (after boarding)". If the center of gravity p is in classification category C, the posture is classified as "sideways". If the center of gravity p is in classification category D, the posture is classified as "rearward".

[0033] In order to enable the classification of postures using such classification frameworks, it is necessary to first investigate the range of possible positions for the center of gravity p of various people with different attributes, and then set up four classification frameworks A, B, C, and D. Also, for example, there may be cases where a person whose center of gravity p corresponds to a position in classification framework B is actually facing sideways, but in this embodiment, since the actual posture is not analyzed, the posture of such a sideways-facing person is also treated as "facing forward (after boarding)".

[0034] Next, the representative image selection unit 340 selects one representative image for each classified pose (four in total) (step 106). This selection is performed by identifying the partial image (person image) with the centroid p closest to the center of each classification frame.

[0035] Figure 8 is an explanatory diagram for representative image selection. In Figure 8, a1 to a3 indicate the positions of the centroid p of three sub-images including a person image, and are included in classification frame A. b1 to a4 indicate the positions of the centroid p of four sub-images including a person image, and are included in classification frame B. c1 to c3 indicate the positions of the centroid p of three sub-images including a person image, and are included in classification frame C. d1 to d5 indicate the positions of the centroid p of five sub-images including a person image, and are included in classification frame D.

[0036] In the example shown in Figure 8, for classification frame A, the partial image where the centroid p is located at a1 is selected as the representative image. For classification frame B, the partial image where the centroid p is located at b3 is selected as the representative image. For classification frame C, the partial image where the centroid p is located at c1 is selected as the representative image. For classification frame D, the partial image where the centroid p is located at d3 is selected as the representative image.

[0037] Next, the provisional attribute determination unit 350 provisionally determines the attributes of a person using each of the four selected representative images (step 108). This determination can be performed using AI (artificial intelligence). In this case, by training the AI ​​using a large number of partial images obtained by having people with various known attributes ride the bus 400 (it is not necessary to use the bus 400 for which attribute determination is being performed in this case, as long as the relative positional relationship between the camera 100 and the travel path R is the same), it is possible to realize AI processing suitable for determining a person moving along the travel path R.

[0038] For example, an example of the results of determining the provisional attributes obtained for each representative image is shown below. (Representative image 1) Classification Category A / Posture: "Front view (before boarding)" Male: 0.4 (percentage of those who apply) Female: 0.8 Working adults in their 20s: 0.6 Elderly: 0.4 (Representative image 2) Classification Category B / Posture: "Front view (after boarding)" Male: 0.3 Female: 0.7 Working adults in their 20s: 0.7 Elderly: 0.4 (Representative image 3) Classification Category C / Posture "Sideways" Male: 0.4 Female: 0.5 Working adults in their 20s: 0.7 Elderly: 0.2 (Representative image 4) Classification Category D / Posture "Back" Male: 0.4 Female: 0.6 Working adults in their 20s: 0.6 Elderly: 0.2 In this way, four sets of provisional attribute determination results are obtained, corresponding to each of the four representative images.

[0039] Next, the weight value setting unit 360 sets four weight values ​​for weighting and adding the provisional attribute values ​​obtained for each of the four representative images (step 110). In this embodiment, these weight values ​​include a first weight value s1 corresponding to the posture and a second weight value s2 corresponding to the image size.

[0040] For example, the first weight value s1 corresponding to posture is set to 0.35, 0.35, 0.20, and 0.10, respectively, for the postures "front (before boarding)", "front (after boarding)", "side", and "back", as the accuracy of attribute determination is considered to decrease as the person is viewed from the side and then from the back.

[0041] Furthermore, the second weight value s2 corresponding to the image size is set so that it represents the area ratio of each representative image, considering that the accuracy of attribute determination increases as the size of the representative image to be determined increases, and decreases as the size of the representative image to be determined decreases.

[0042] Furthermore, considering the distance from camera 100 to the person, the actual sizes of the four representative images are not expected to differ significantly. Therefore, in this embodiment, the proportions contributing to the accuracy of attribute determination are set as follows: 75% for the first weight value s1 corresponding to posture, 25% for the second weight value s2 corresponding to image size, and a combined overall weight value s3 of 0.75 × s1 + 0.25 × s2. These proportions are merely examples and can be changed as appropriate.

[0043] Next, the final attribute determination unit 370 performs weighted addition on the provisional attributes obtained for each of the four representative images using the overall weight value s3 set for each, to comprehensively determine the final attributes of the person of interest (step 112).

[0044] Figure 9 shows the results of determining the person's attributes using the specific values ​​described above. The image size shown in Figure 9 indicates the number of pixels in the width and height of the representative image.

[0045] In the example shown in Figure 9, the overall attributes of the person under focus are: Male: 0.37 Female: 0.68 Working adults in their 20s: 0.66 Elderly: 0.33 This was the determination. Additionally, the most likely attributes could be "female" for gender and "working adult in their 20s" for age group.

[0046] Thus, in the person attribute determination device 1 of this embodiment, since the movement path R of the person whose attributes are to be determined is known, attribute determination can be performed on person images captured along this movement path R. Moreover, since the person attributes can be comprehensively determined using person images corresponding to multiple locations along the movement path R, it is possible to improve the accuracy of the person's attributes being determined.

[0047] Furthermore, when determining a person's attributes, the accuracy of the determination can vary significantly depending on the person's posture. Therefore, by setting weight values ​​that take posture into account, the accuracy of the final attribute determination can be improved.

[0048] Furthermore, the weight values ​​are set higher when the person is facing forward and lower as the angle shifts away from the front. Generally, the accuracy of attribute determination is considered highest when the person is facing forward, that is, when the person's face is visible. Therefore, by setting weight values ​​that take into account whether or not the person is facing forward, the accuracy of attribute determination can be improved.

[0049] Furthermore, even with the same person image (partial image), a larger image with a higher number of constituent pixels allows for more accurate identification of the person's actions and expressions, thus improving the accuracy of attribute determination. Therefore, by setting weight values ​​that take into account the size (area) of the person image, the accuracy of attribute determination can be improved.

[0050] Furthermore, multiple non-overlapping detection frames (classification frames) A, B, C, and D are set along the movement path R. Focusing on the centroid p of the person image, person images in which the centroid p is included in each detection frame are extracted as person images to be subject to provisional attribute determination. This makes it clear which person images will be extracted, and makes it easy to identify the person images to be extracted.

[0051] In particular, by extracting the person image whose centroid p is closest to the center of the detection frame (classification frame), the position on the movement path R from which the person image is extracted can be fixed, and since the posture of the person at those positions will also be similar, it is possible to maintain a high level of accuracy in attribute determination.

[0052] It should be noted that the present invention is not limited to the embodiments described above, and various modifications can be implemented within the scope of the gist of the invention. In the embodiments described above, a representative image was selected for each of the four postures (front (before boarding), front (after boarding), side, and rear), and four provisional attributes were determined. After that, weighted addition was performed to determine the final attributes. However, the number of postures for which provisional attributes are determined may be other than four, and the number can be determined according to the shape of the travel path R, etc.

[0053] Furthermore, while the example shown in Figure 9 describes the case where four representative images are obtained, there may be cases where four representative images cannot be obtained, such as when there is another person between the person of interest and the camera 100. For example, as shown in Figure 10, if only two representative images of postures, front (after boarding) and side, are obtained, the first weight value s1 corresponding to the posture can be adjusted so that the sum of the two values ​​is 1. This allows for weighted addition of the two provisional attributes determined for the two representative images to perform the final attribute determination.

[0054] Furthermore, although the above-described embodiment described the case of determining the attributes of a person riding in bus 400, the present invention can also be applied to determining the attributes of a person traveling along other travel paths R.

[0055] Figure 11 shows another example of determining a person's attributes. The example in Figure 11 shows the movement paths when determining the attributes of a person entering and exiting a station's ticket gate. Possible movement paths include R11 when exiting ticket gate 500 and going left along passage 510, R12 when going right, and R21 and R22 when going the other way around, passing through passage 510 and then through ticket gate 500. Once these four movement paths R11, R12, R21, and R22 are determined, it becomes possible to determine the attributes of a person moving along all or part of these paths using a similar method. [Industrial applicability]

[0056] As described above, according to the present invention, since the movement path of the person whose attributes are to be determined is known, attribute determination can be performed on images of the person taken along this movement path. Furthermore, since the person's attributes can be comprehensively determined using images of the person corresponding to multiple locations along the movement path, it is possible to improve the accuracy of the determined person's attributes. [Explanation of Symbols]

[0057] 1 Person attribute determination device 100 Cameras 200 Onboard equipment 300 Person Attribute Determination Server 320 Human Image Extraction Unit 330 Image Classification Unit 340 Representative Image Selection Section 350 Temporary attribute determination unit 360 Weight Value Setting Section 370 Final attribute judgment section

Claims

1. An imaging means for imaging a predetermined imaging range that includes the movement path of a person whose attributes are to be determined, A person image extraction means for extracting a person image corresponding to the same person from images captured by the imaging means at multiple locations along the movement path, A provisional attribute determination means that determines the provisional attributes of a person using each of the multiple person images extracted by the person image extraction means, A final attribute determination means that determines the attributes of a person by assigning weights to the multiple provisional attributes determined by the provisional attribute determination means using weight values ​​corresponding to each of the multiple locations, A person attribute determination device comprising the above, wherein each of the multiple locations is a location corresponding to each of a set of pre-set postures when the same person moves along the movement path.

2. The person attribute determination device according to claim 1, characterized in that the weight values ​​are set based on the posture of the person at each of the multiple locations.

3. The person attribute determination device according to claim 2, characterized in that the weight value is large when the person is facing forward and decreases as the person deviates from facing forward.

4. The person attribute determination device according to any one of claims 1 to 3, characterized in that the weight value increases when the size of the person image corresponding to each of the multiple locations is large, and decreases as the size decreases.

5. Along the movement path within the imaging range, multiple detection frames are set corresponding to each of the multiple locations, and they do not overlap with each other. The person attribute determination device according to claim 1, characterized in that the person image extraction means focuses on the centroid of the person image and extracts the person image in which the centroid is included in each of the plurality of detection frames as the person image to be determined for the provisional attribute.

6. The person attribute determination device according to claim 5, characterized in that the person image extraction means extracts the person image whose center of gravity is closest to the center of the detection frame.

7. A method for determining a person's attributes performed in a person's attribute determination device comprising an imaging means, a person's image extraction means, a provisional attribute determination means, and a final attribute determination means, The steps include: capturing an image of a predetermined imaging range that includes the movement path of a person whose attributes are to be determined, using the imaging means; The steps include: extracting a person image corresponding to the same person from images captured by the imaging means at multiple locations along the movement path using the person image extraction means; The steps include: determining the provisional attributes of a person using each of the multiple person images extracted by the person image extraction means, The steps include: assigning weights to the multiple provisional attributes determined by the provisional attribute determination means using weight values ​​corresponding to each of the multiple locations, thereby ultimately determining the attributes of a person using the final attribute determination means; A person attribute determination method comprising, wherein each of the multiple locations is a location corresponding to each of a set of pre-set postures when the same person moves along the movement path.

Images