Posture estimation device, posture estimation method, and program

Abstract

The pose estimation device (2000) detects keypoints from a target image (10) in which one or more people are captured, and calculates one or more selection factors based on the keypoints. The selection factors include a density of people in the target image (10), a resolution, or both. The pose estimation device (2000) selects one of a plurality of predefined keypoint association algorithms based on the selection factors. The pose estimation device (2000) performs keypoint association on the keypoints using the selected algorithm to divide the keypoints into one or more keypoint groups, each of which includes keypoints of the same person. For each keypoint group, the pose estimation device (2000) estimates a pose of the person corresponding to the keypoint group based on the keypoints included in the keypoint group.

Description

[Technical field]

[0001] This disclosure relates generally to techniques for analyzing images to estimate the pose of a person captured in the image. [Background technology]

[0002] There are various types of analyses that can be performed on images of one or more people. One of these analyses is pose estimation, which estimates the pose of each person captured in the image. A person's pose can be estimated based on key points, such as body joints, detected in the image.

[0003] When multiple people are in an image, pose estimation involves a process called "key-point association" that divides the keypoints into groups, each of which contains keypoints that belong to the same person. Patent document 1 discloses one algorithm for keypoint association. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2021 / 014530 Summary of the Invention [Problem to be solved by the invention]

[0005] There are multiple algorithms for keypoint association, and no algorithm is optimal for all situations. It is an object of the present disclosure to provide a novel technique for accurately estimating the pose of a person captured in an image. [Means for solving the problem]

[0006] The pose estimation apparatus provided by the present disclosure includes at least one memory configured to store instructions and at least one processor configured to execute the instructions to: obtain a target image in which one or more people are captured, detect keypoints from the target image, calculate one or more selection factors based on the keypoints, the one or more selection factors including a density of people in the target image, a resolution, or both, select an algorithm for keypoint association from predefined algorithms for keypoint association based on the one or more selection factors, perform keypoint association on the keypoints using the selected algorithm to divide the keypoints into one or more keypoint groups, each of which includes the keypoints of the same person, and estimate, for each keypoint group, a pose of the person corresponding to the keypoint group based on the keypoints included in the keypoint group.

[0007] The present disclosure provides a pose estimation method, executed by one or more computers, that includes the steps of: acquiring a target image in which one or more people are captured, detecting keypoints from the target image, calculating one or more selection factors based on the keypoints, the one or more selection factors including a density of people in the target image, a resolution, or both, selecting an algorithm for keypoint association from predefined algorithms for keypoint association based on the one or more selection factors, performing keypoint association on the keypoints using the selected algorithm to divide the keypoints into one or more keypoint groups, each of which includes the keypoints of the same person, and estimating, for each keypoint group, a pose of the person corresponding to the keypoint group based on the keypoints included in the keypoint group.

[0008] The present disclosure provides a non-transitory computer-readable storage medium storing a program that causes one or more computers to execute the steps of: acquiring a target image in which one or more people are captured, detecting keypoints from the target image, calculating one or more selection factors based on the keypoints, the one or more selection factors including a density of people in the target image, a resolution, or both, selecting an algorithm for keypoint association from predefined algorithms for keypoint association based on the one or more selection factors, performing keypoint association on the keypoints using the selected algorithm to divide the keypoints into one or more keypoint groups, each of which includes the keypoints of the same person, and estimating, for each keypoint group, a pose of the person corresponding to the keypoint group based on the keypoints included in the keypoint group. Effect of the Invention

[0009] The present disclosure provides a novel technique for accurately estimating the pose of a person in an image. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating an overview of a posture estimation device according to a first embodiment. [Diagram 2] 1 is a block diagram showing an example of a functional configuration of a posture estimation device according to a first embodiment. [Diagram 3] FIG. 2 is a block diagram showing an example of a hardware configuration of the posture estimation device according to the first embodiment. [Figure 4] 5 is a flowchart illustrating an exemplary flow of processing executed by the posture estimation device of the first embodiment. [Diagram 5] 10 is a flowchart illustrating an example of a method for calculating a density of people in a target image. [Figure 6] FIG. 13 is a diagram showing an example of steps 202 to 206. [Figure 7] FIG. 1 illustrates training of a midpoint model and an orientation map model. [Figure 8] FIG. 1 illustrates training of an orientation map model and a position map model. [Figure 9] FIG. 1 illustrates training of a midpoint model, an orientation map model, and a position map model. [Figure 10] 1 is a flowchart illustrating a first exemplary flow of a process for selecting a keypoint association algorithm. [Figure 11] 11 is a flowchart illustrating a second exemplary flow of a process for selecting a keypoint association algorithm. [Figure 12] 13 is a flowchart illustrating a third exemplary flow of a process for selecting a keypoint association algorithm. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The embodiments of the present disclosure will be described below with reference to the drawings. In each drawing, the same elements are given the same reference numerals, and duplicated descriptions will be omitted as necessary. In addition, unless otherwise specified, the predetermined information (e.g., the predetermined value or the predetermined threshold value) is stored in advance in a storage device accessible by a computer that uses the information.

[0012] EMBODIMENT 1 <Summary> Fig. 1 shows an overview of posture estimation apparatus 2000 according to embodiment 1. Note that the overview shown in Fig. 1 shows an example of the operation of posture estimation apparatus 2000 in order to make posture estimation apparatus 2000 easier to understand, and does not limit or narrow the range of operations that posture estimation apparatus 2000 can perform.

[0013] The pose estimation device 2000 acquires a target image 10 in which one or more people are captured, and estimates the pose of each person. To this end, the pose estimation device 2000 detects keypoints from the target image 10, and performs keypoint association on the detected keypoints. The keypoints may indicate characteristic points of a person's body, such as joints. The keypoint association is a process of dividing keypoints into groups such that each group contains keypoints that belong to the same person. The pose of each person can be estimated based on the keypoints identified as belonging to that person by the keypoint association.

[0014] There are multiple algorithms for keypoint association, and which algorithm is suitable for estimating the pose of a person captured in the image depends on the image being analyzed. Hereinafter, an algorithm for keypoint association is referred to as a "keypoint association algorithm." Thus, the pose estimation device 2000 calculates a factor related to the target image 10 to select a keypoint association algorithm for the target image 10 from multiple predefined keypoint association algorithms. Hereinafter, this factor is referred to as a "selection factor." The selection factor may include the density of people in the target image 10, the resolution, or both.

[0015] The pose estimation device 2000 obtains groups of keypoints (hereinafter, keypoint groups) by executing the selected keypoint association algorithm on the keypoints detected from the target image 10, and each keypoint group includes keypoints that are estimated to belong to the same person. Then, for each keypoint, the pose estimation device 2000 identifies the pose of the person corresponding to the keypoint group based on the keypoints included in the keypoint group.

[0016] <Examples of effects> There are various keypoint association algorithms, and no algorithm is optimal for all situations. According to the pose estimation device 2000, the keypoint association algorithm applied to the target image 10 is not fixed, but is selected from a plurality of predefined keypoint association algorithms based on a selection factor. The selection factor may include the density, resolution, or both of the people captured in the target image 10. In this way, the keypoint association algorithm applied to the target image 10 is appropriately selected based on the density, resolution, or both of the people captured in the target image 10. Therefore, the pose of the person in the target image 10 can be accurately estimated.

[0017] The posture estimation device 2000 will now be described in detail.

[0018] <Example of functional configuration> FIG. 2 is a block diagram showing an example of a functional configuration of the posture estimation device 2000 of the first embodiment. The posture estimation device 2000 includes an acquisition unit 2020, a keypoint detection unit 2040, an algorithm selection unit 2060, a keypoint association unit 2080, and an estimation unit 2100. The acquisition unit 2020 acquires a target image 10. The keypoint detection unit 2040 detects keypoints from the target image 10. The algorithm selection unit 2060 calculates one or more selection factors and selects a keypoint association algorithm from among predefined ones based on the calculated selection factors. The keypoint association unit 2080 executes the selected keypoint association algorithm on the detected keypoints, thereby generating keypoint groups. For each keypoint group, the estimation unit 2100 estimates a posture of a person corresponding to the keypoint group based on the keypoints in the keypoint group.

[0019] <Example of hardware configuration> Posture estimation apparatus 2000 may be realized by one or more computers. Each of the one or more computers may be a dedicated computer manufactured for realizing posture estimation apparatus 2000, or may be a general-purpose computer such as a personal computer (PC), a server machine, or a mobile device.

[0020] Posture estimation device 2000 may be realized by installing an application on a computer. The application is realized by a program for causing a computer to function as posture estimation device 2000. In other words, the program implements the functional units of posture estimation device 2000.

[0021] Fig. 3 is a block diagram showing an example of a hardware configuration of a computer 1000 that realizes the posture estimation apparatus 2000 of embodiment 1. In Fig. 3, the computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output interface 1100, and a network interface 1120.

[0022] The bus 1020 is a data transmission path for the processor 1040, the memory 1060, the storage device 1080, the input / output interface 1100, and the network interface 1120 to transmit and receive data to and from each other. The processor 1040 is a processor such as a central processing unit (CPU), a graphics processing unit (GPU), or a field-programmable gate array (FPGA). The memory 1060 is a main storage element such as a random access memory (RAM) or a read only memory (ROM). The storage device 1080 is an auxiliary storage element such as a hard disk, a solid state drive (SSD), or a memory card. The input / output interface 1100 is an interface between the computer 1000 and peripheral devices such as a keyboard, a mouse, and a display device. The network interface 1120 is an interface between the computer 1000 and a network. The network may be a local area network (LAN) or a wide area network (WAN).

[0023] The hardware configuration of computer 1000 is not limited to that shown in Fig. 3. For example, as described above, posture estimation device 2000 may be realized by multiple computers. In this case, the computers may be connected to each other via a network.

[0024] <Processing flow> FIG. 4 is a flowchart showing an exemplary flow of a process executed by the posture estimation device 2000 of the first embodiment. The acquisition unit 2020 acquires a target image 10 (S102). The keypoint detection unit 2040 detects keypoints from the target image 10 (S104). The algorithm selection unit 2060 calculates one or more selection factors (S106). The algorithm selection unit 2060 selects a keypoint association algorithm from predefined ones based on the calculated selection factors (S108). The keypoint association unit 2080 executes the selected algorithm on the detected keypoints to generate keypoint groups (S110). The estimation unit 2100 estimates a posture of a person for each keypoint group (S112).

[0025] <Acquisition of target image 10: S102> The acquisition unit 2020 acquires the target image 10. There are various methods for acquiring the target image 10. In some aspects, the target image 10 is stored in advance in a storage device so that the posture estimation device 2000 can acquire it. In this case, the acquisition unit 2020 may access the storage device to acquire the target image. In other embodiments, the target image 10 may be transmitted by another computer, such as a camera that generates the target image 10. In this case, the acquisition unit 2020 may acquire the target image 10 by receiving it.

[0026] In some embodiments, the target image 10 may be one of a sequence of images, such as video frames forming a video. In this case, the pose estimation device 2000 may obtain all or a part of the sequence of images as the target image 10 and estimate the pose of each person for each target image 10.

[0027] <Keypoint detection: S104> The key point detection unit 2040 detects key points from the target image 10 (S104). There are various methods for detecting key points of a person from an image, and the key point detection unit 2040 may use one of these methods to detect key points from the target image 10. The key points of a person may be one or more features of the person's body, such as joints at the neck, shoulders, elbows, wrists, hips, knees, ankles, etc.

[0028] In some embodiments, the keypoint detector 2040 is configured to receive an image as an input, and further includes a machine learning based model (e.g., a neural network) that is pre-trained to detect keypoints from the input image in response to the image input, hereinafter referred to as a "keypoint detection model."

[0029] The keypoint detection model may take a target image 10 as an input, extract features from the target image 10, detect one or more keypoints from the target image 10 based on the extracted features, and identify a class of each keypoint (e.g., neck, right shoulder, etc.) based on the extracted features. In this case, the keypoint detection model may include a first model pre-trained to extract features from the target image 10 and a second model pre-trained to detect and classify keypoints based on the features extracted from the target image 10. Each of the first model and the second model may be configured as a machine learning-based model such as a neural network. Note that there are various types of machine learning models that can detect keypoints from an input image and further classify them, and the keypoint detection model may be configured as one of such models.

[0030] <Calculation of selection factors: S106> To select a suitable keypoint association algorithm for the target image 10, the algorithm selection unit 2060 calculates selection factors based on the detected keypoints (S106). As mentioned above, the selection factors may include the density of people in the target image 10, the resolution of people in the target image 10, or both. An example of how these factors are calculated will now be described.

[0031] "Density of people" When using the person density as a selection factor, the algorithm selection unit 2060 calculates the person density in the target image 10 based on the keypoints detected from the target image 10. The person density can be measured using the right shoulder and left shoulder keypoints.

[0032] 5 is a flowchart showing an example of a method for calculating the density of people in the target image 10. The algorithm selection unit 2060 extracts key points representing the left shoulder or right shoulder from all detected key points (S202). Then, for each left shoulder key point, the algorithm selection unit 2060 searches for the closest right shoulder key point and links them (S204). This step may link several right shoulder key points to multiple left shoulder key points.

[0033] For each MP point, the algorithm selection unit 2060 extracts the longest link and the shortest link that have the MP point as one end, and removes the longest link when the length of the longest link exceeds a predetermined multiple (e.g., twice) of the length of the shortest link (S206). This step may cause some MP points to become non-MP points (i.e., right shoulder key points linked to only a single left shoulder key point) as a result of removing the longest link.

[0034] 6 shows an example of the case of steps 202 to 206. In this example, three right shoulders 22-1 to 22-3 and five left shoulders 24-1 to 24-5 are detected in step S202. Next, in step S204, the left shoulders 24-1 to 24-5 are linked to the right shoulders 22-1, 22-2, 22-2, 22-3, and 22-3, respectively. In this case, the right shoulders 22-2 and 22-3 are MP points.

[0035] In step S206, it is determined that the length of the longest link between the right shoulder 22-3 and the left shoulder 24-5 exceeds a predetermined multiple of the length of the shortest link between the right shoulder 22-3 and the left shoulder 24-4. As a result, the link between the right shoulder 22-3 and the left shoulder 24-5 is removed. As a result of this removal, the right shoulder 22-3 becomes a non-MP point.

[0036] Step S206 is repeated until the number of MP points converges (for example, becomes constant). Hereinafter, this number of MP points is referred to as "NMP". Specifically, the algorithm selection unit 2060 determines whether or not NMP has converged (S208). If NMP has not converged (S208: NO), step S206 is executed again. On the other hand, if NMP has converged (S208: YES), the algorithm selection unit 2060 sets NMP as the density of people in the target image 10. Conceptually, the larger the NMP, the more densely people are packed in the target image 10.

[0037] In another aspect, the algorithm selection unit 2060 may calculate the density of people in the target image 10 based on NMP. For example, a function that outputs a value proportional to an input value is defined in advance to calculate the density. In this case, the algorithm selection unit 2060 may input NMP to this function, obtain an output value proportional to NMP, and use this output value as the density of people in the target image 10.

[0038] "Resolution of People" When using the resolution of the person as the selection factor, the algorithm selection unit 2060 may calculate the resolution of the person based on the above-mentioned NMP. Specifically, the algorithm selection unit 2060 extracts links whose length is less than a threshold (e.g., 1 / 25 of the width of the target image 10) that can be defined based on one of the dimensions of the target image from the MP points remaining after step S208 in Fig. 5. The links extracted here are called "SL links."

[0039] The algorithm selection unit 2060 may calculate a value called RSL as follows:

number

[0040] Conceptually, the larger the RSL, the lower the resolution of the person in the target image 10. Therefore, in some embodiments, the algorithm selection unit 2060 may calculate the resolution of the person in the target image 10 as a value that increases as the RSL decreases. For example, a function that outputs a value proportional to the inverse of an input value in advance to calculate the resolution. In this case, the algorithm selection unit 2060 may input the RSL to this function to obtain an output value proportional to 1 / RSL, and use this output value as the resolution of the person in the target image 10. Note that this function may be defined to output a maximum value when 0 is given as an input.

[0041] In another aspect, the algorithm selection unit 2060 may use RSL as a selection factor representing the resolution of the person in the target image 10. In this case, as described below, when determining whether the resolution of the person in the target image 10 is below a threshold, the algorithm selection unit 2060 may determine that the resolution of the person in the target image 10 is below the threshold if RSL is greater than the threshold.

[0042] <Keypoint Association Algorithm> The predefined keypoint association algorithms may include two or more of: 1) a midpoint algorithm, 2) a direction map algorithm, and 3) a position map algorithm, each of which is described below.

[0043] <<Midpoint Algorithm>> The midpoint algorithm detects a midpoint from a target image 10 to perform keypoint association. A midpoint is a point located in the middle of two keypoints. Details of the midpoint algorithm are disclosed in Patent Document 1.

[0044] The midpoint algorithm may be implemented using a machine learning model such as a neural network. Hereinafter, this machine learning model will be referred to as a "midpoint model." The midpoint model may be configured to receive a target image 10 as input, and may be pre-trained to output a midpoint in the target image 10 in response to input data.

[0045] The midpoint algorithm may input the target image 10 into a midpoint model to obtain midpoints in the target image 10. The midpoint algorithm then divides the keypoints into keypoint groups based on the midpoints.

[0046] <<Orientation Map Algorithm>> The orientation map algorithm generates an orientation map of the target image 10 and divides the keypoints into keypoint groups using the orientation map for keypoint association. The orientation map is a feature map extracted from the target image 10 and has the same size as the target image 10. The orientation map indicates unit vectors for each pixel of a person's region in the target image 10. A unit vector corresponding to a pixel in a person region points from that pixel to a predefined reference point for that person region. The reference point for a person region may be a specific keypoint of the person (e.g., a neck keypoint) that corresponds to the person region.

[0047] More specifically, the orientation map may include a set of two feature maps called an H orientation map and a V orientation map. In the H orientation map, pixels in the person region indicate the horizontal component (i.e., the x component) of the corresponding unit vector, while in the V orientation map, pixels in the person region indicate the vertical component (i.e., the y component) of the corresponding unit vector.

[0048] The orientation map algorithm may be implemented using a machine learning model such as a neural network. Hereinafter, this machine learning model is referred to as a "orientation map model." The orientation map model may be configured to receive a target image 10 as an input, and may be pre-trained to output an orientation map of the target image 10 in response to receiving input data. The orientation map algorithm may input the target image 10 to the orientation map model to obtain the orientation map. The orientation map algorithm then uses the orientation map to divide the keypoints into keypoint groups.

[0049] The orientation map algorithm uses the orientation map to calculate a score for each keypoint-person pair detected in the target image 10, and identifies which keypoint belongs to which person based on the calculated scores. Then, for each keypoint, the orientation map algorithm identifies that keypoint as belonging to the person corresponding to the maximum score for that keypoint.

[0050] Suppose the pair of keypoint K1 and person P1 has a score S1, the pair of keypoint K1 and person P2 has a score S2, and the pair of keypoint K1 and person P3 has a score S3. The maximum score of S1, S2, and S3 is S2. In this case, the pair of keypoint K1 and person P2 has the maximum score, so the orientation map algorithm identifies keypoint K1 as belonging to person P2.

[0051] The score of a keypoint-person pair can be calculated as the product of three factors OB, RoD, and D (i.e., S=OB*RoD*D). These three factors are calculated as follows: The Orientation Map algorithm generates one or more intermediate points on the line between the keypoint and the reference point of the person. For each intermediate point, the Orientation Map algorithm determines whether the intermediate point is located in a person region or not. The Orientation Map algorithm calculates the factor OB as what percentage of the intermediate points are located in the person region. For example, if two out of three intermediate points are located in the person region, the factor OB is 2 / 3.

[0052] Then, for each intermediate point located in the person region, the orientation map algorithm obtains the unit vector corresponding to the intermediate point from the orientation map. The coordinates of the intermediate point on the target image 10 are (x1, y1). In this case, the unit vector corresponding to the intermediate point is located at (x1, y1) in the orientation map. The orientation map algorithm also obtains the unit vectors corresponding to the key points.

[0053] The Direction Map algorithm calculates the variation of the directions of the obtained unit vectors as a factor RoD, which represents the maximum difference between two of those unit vectors.

[0054] For factor D, the orientation map algorithm calculates the distance between the person's key points and the reference points.

[0055] <<Location Map Algorithm>> The position map algorithm calculates a position map of the target image 10 and divides the keypoints into keypoint groups using the position map for keypoint association. The position map is a feature map extracted from the target image 10 and has the same size as the target image 10. In the position map, the pixels of the person region indicate the relative position of the person region. The relative position of the person region is the position of a reference point (e.g., neck keypoint) with respect to a base position of the target image 10. The base position of the target image 10 can be its origin (e.g., the top left corner).

[0056] More specifically, the position map may include two feature maps, referred to as an H position map and a V position map. In an H position map, a pixel in a person region indicates the horizontal position of the reference point of that person region relative to a base position (e.g., the left edge of the target image 10), while in a V position map, a pixel in a person region indicates the vertical position of the reference point of that person region relative to a base position (e.g., the top edge of the target image 10).

[0057] Let the width of the target image 10 be W and the height be H, and let the absolute coordinates of the reference point of the person area in the target image 10 be (x1, y1). In this case, the relative position of the person area is (x1 / W, y1 / W). Therefore, the pixel of this person area indicates x1 / W in the H position map and y1 / H in the V position map.

[0058] The location map algorithm may be implemented using a machine learning model such as a neural network. Hereinafter, this machine learning model is referred to as a "location map model." The location map model may be configured to receive a target image 10 as an input, and may be pre-trained to output a location map of the target image 10 in response to input of input data. The location map algorithm may input the target image 10 to the location map model to obtain the location map. The location map algorithm then uses the location map to divide the keypoints into keypoint groups.

[0059] For each keypoint detected in the target image 10, the location map algorithm calculates its distance from each person detected in the target image 10. The distance is calculated using the relative positions of the keypoints and people taken from the location map.

[0060] Specifically, the position map algorithm obtains pixel values ​​from pixels in the position map whose coordinates on the position map are the same as those of the keypoint on the target image 10, and uses the obtained values ​​as the relative position of the keypoint. In the target image 10, the coordinates of keypoint K1 are (x1, y1). The pixel at (x1, y1) in the H position map indicates x2, and the pixel at (x1, y1) in the V position map indicates y2. In this case, the position map algorithm obtains the pixel value x2 from the pixel at (x1, y1) in the H position map as the x coordinate of the relative position of keypoint K1. The position map algorithm also obtains the pixel value y2 of the pixel at (x1, y1) in the V position map as the y coordinate of the relative position of keypoint K1. As a result, the relative position of keypoint K1 is identified as (x2, y2).

[0061] For the relative position of a person, the position map algorithm obtains pixel values ​​from pixels in the person region corresponding to the person, and uses the obtained values ​​as the relative position of the person. Assume that the pixels in the person region of person P1 show a value x3 in the H position map, and the pixels in the person region of person P1 show a value y3 in the V position map. In this case, the relative position of person P1 is specified as (x3, y3). As mentioned above, when the relative position of a keypoint is shown as (x2, y2), the position map algorithm calculates the distance between (x2, y2) and (x3, y3) as the distance between the keypoint and person P1.

[0062] After calculating the distance of each person to the keypoint, the location map algorithm identifies the person with the shortest distance from the keypoint. The location map then identifies the keypoint as belonging to the identified person.

[0063] <Model configuration> <<About the Orientation Map Model>> An orientation map is a feature map that describes the geometric relationship between a reference point (such as a neck keypoint) and any other pixel in the entire body region of a person. To have a good quality orientation map, the orientation map model preferably has a good understanding of the person's body context (i.e., how different body parts are connected). A midpoint defined in the middle of two paired keypoints can help the orientation map model to better understand the connections between different body parts, and thus improve the quality of the orientation map.

[0064] Based on the above insight, the direction map is preferably configured to generate the direction map using keypoints and midpoints detected from the target image 10. Thus, when the midpoint algorithm and the direction map model algorithm are employed as the predefined keypoint association algorithm, the direction map model can be configured to take as input not only the target image 10 but also the output of the midpoint model. In this case, the midpoint model and the direction map model can be trained together using the same training data. Note that the midpoint model may function as a keypoint detection model to detect keypoints from the target image 10.

[0065] 7 illustrates the training of a midpoint model and a direction map model. In this example, the midpoint model takes a target image 10 as input and detects keypoints and midpoints from the target image 10. The direction map model is connected to the midpoint model so that the direction map model can take the output of the midpoint (i.e., keypoints and midpoints) as input.

[0066] The training data for training the model shown in Fig. 7 includes a set of input images in which one or more people are captured, and ground truth data. The ground truth data indicates keypoints and midpoints detected from the corresponding input images, and a direction map generated from the corresponding input images. The models are trained by calculating a loss that represents the degree of dissimilarity between their outputs (i.e., the keypoints and midpoints detected by the midpoints, and the direction map generated by the direction map model) and the ground truth data, and further updating the trainable parameters of the models based on the calculated loss.

[0067] <<About the location map model>> For any pixel in the person's body region excluding the reference point, its value in the person's two-orientation map (denoted vx and vy, respectively) is the X and Y components of the unit vector from the pixel to the reference point. Therefore, vx and vy satisfy the condition "vx^2 + vy^2 = 1", which means that the sum of the squares of vx and vy is a constant value for all pixels in the person's body region excluding the reference point. On the other hand, the position map is defined as filling the person's body region with constant values ​​corresponding to the person's position in the image. Thus, the squaring of the orientation map can help improve the quality of the position map by converging the values ​​of all pixels in the person's body region to a constant value. Note that the squaring of the orientation map includes the squared H-orientation map, where each pixel in the H-orientation map indicates the square of the value indicated by the corresponding pixel, and the squared V-orientation map, where each pixel in the V-orientation map indicates the square of the value indicated by the corresponding pixel.

[0068] Based on the above insight, it is preferable to configure the location map model to generate a location map using the square of the direction map. Thus, when the direction map model algorithm and the location map algorithm are employed as the predefined keypoint association algorithm, the location map model can be configured to take as input not only the target image 10 but also the square of the output of the direction map model. In this case, the direction map model and the location map can be trained together using the same training data as each other.

[0069] 8 illustrates training of the orientation map model and the position map model. In this example, between the orientation map model and the position map model there is a component for computing the square of the output of the orientation map model. This component is configured to take the output of the orientation map model, compute the square of this output, and provide the result of the computation to the position map model.

[0070] The training data for training the model shown in Fig. 8 includes a set of input images in which one or more people are captured, and ground truth data. The ground truth data indicates an orientation map and a position map generated from the corresponding input images. The models are trained by calculating a loss that represents the degree of dissimilarity between their outputs (i.e., the orientation map generated by the orientation map model and the position map generated by the position map model) and the ground truth data, and further updating the trainable parameters of the models based on the calculated loss.

[0071] By combining the configurations shown in Figures 7 and 8, the midpoint model, the orientation map model, and the location map model can be trained collectively when these models are adopted as predefined keypoint association algorithms. Figure 9 shows the training of the midpoint model, the orientation map model, and the location map model.

[0072] The training data for training the model shown in Fig. 9 includes a set of input images in which one or more people are captured, and ground truth data. The ground truth data indicates keypoints and midpoints detected from the corresponding input images, and a direction map and a position map generated from the corresponding input images. The models are trained by calculating a loss that represents the degree of dissimilarity between their outputs (i.e., the keypoints and midpoints detected by the midpoints, the direction map generated by the direction map model, and the position map generated by the position map model) and the ground truth data, and further updating the trainable parameters of the models based on the calculated loss.

[0073] <Selection of keypoint association algorithm: S108> The algorithm selection unit 2060 selects (S108) a keypoint association algorithm suitable for the target image 10 based on the selection factor. In some embodiments, the algorithm selection unit 2060 may determine the keypoint association algorithm based on whether the selection factor is greater than a predefined threshold.

[0074] 10 is a flow chart showing a first exemplary flow of a process for selecting a keypoint association algorithm. In this example, the predefined keypoint algorithms include a midpoint algorithm, a direction map algorithm, and a position map algorithm. Also in this example, the selection factors include the density of people in the target image 10 and the resolution of people in the target image 10.

[0075] Specifically, the algorithm selection unit 2060 determines whether the resolution of people in the target image 10 is smaller than a threshold ThR (S302). When the resolution is smaller than the threshold ThR (S302: YES), the algorithm selection unit 2060 selects the midpoint algorithm as the keypoint association algorithm to be applied to the target image 10 (S304). On the other hand, when the resolution is equal to or greater than the threshold ThR (S302: NO), the algorithm selection unit 2060 determines whether the density of people in the target image 10 is smaller than a threshold ThD (S306).

[0076] When the density is smaller than the threshold ThD (S306: YES), the algorithm selection unit 2060 selects the direction map algorithm as the keypoint association algorithm to be applied to the target image 10 (S308). On the other hand, when the density is equal to or larger than the threshold ThD (S306: NO), the algorithm selection unit 2060 selects the position map algorithm as the keypoint association algorithm to be applied to the target image 10 (S310).

[0077] 11 is a flow chart showing a second exemplary flow of a process for selecting a keypoint association algorithm. In this example, the predefined keypoint algorithms include a midpoint algorithm and a direction map algorithm. Also in this example, the resolution of the person in the target image 10 is used as a selection factor.

[0078] Specifically, the algorithm selection unit 2060 determines whether the resolution of the person in the target image 10 is smaller than a threshold ThR (S402). If the resolution is smaller than the threshold ThR (S402: YES), the algorithm selection unit 2060 selects the midpoint algorithm as the keypoint association algorithm to be applied to the target image 10 (S404). On the other hand, if the resolution is equal to or greater than the threshold ThR (S402: NO), the algorithm selection unit 2060 selects the orientation map algorithm as the keypoint association algorithm to be applied to the target image 10 (S406).

[0079] 11, a position map algorithm may be adopted as one of the predefined keypoint association algorithms instead of the orientation map algorithm. In this case, when the resolution is equal to or greater than the threshold ThR, the algorithm selection unit 2060 selects the position map algorithm as the keypoint association algorithm to be applied to the target image 10.

[0080] 12 is a flow chart showing a third exemplary flow of a process for selecting a keypoint association algorithm. In this example, the predefined keypoint algorithms include a direction map algorithm and a location algorithm. Also in this example, the density of people in the target image 10 is used as a selection factor.

[0081] Specifically, the algorithm selection unit 2060 determines whether the density of people in the target image 10 is less than a threshold ThD (S502). If the density is less than the threshold ThD (S502: YES), the algorithm selection unit 2060 selects the direction map algorithm as the keypoint association algorithm to be applied to the target image 10 (S504). On the other hand, if the density is equal to or greater than the threshold ThD (S502: NO), the algorithm selection unit 2060 selects the position map algorithm as the keypoint association algorithm to be applied to the target image 10 (S506).

[0082] 12, a midpoint algorithm may be adopted as one of the predefined keypoint association algorithms instead of the orientation map algorithm. In this case, when the density is equal to or greater than the threshold ThD, the algorithm selection unit 2060 selects the midpoint algorithm as the keypoint association algorithm to be applied to the target image 10.

[0083] <Output from posture estimation device 2000> The posture estimation device 2000 may be configured to output information (called output information) indicating the result of posture estimation. For example, the output information may include an identifier of the target image 10 (e.g., a frame number) and, for each key point group, an identifier of an estimated posture of the key point group and a set of key point information for each key point in the key point group. The estimated posture identifier indicates what type of posture the person corresponding to the key point group is assuming. The key point information indicates the type (e.g., neck, right shoulder, etc.) and position (e.g., coordinates) of the key point.

[0084] There are various ways to output the output information, and in some aspects the output information may be stored in a storage device, displayed on a display device, or transmitted to another computer, such as a PC or smartphone of a user of the pose estimation apparatus 2000.

[0085] The program can be stored and provided to the computer using various types of non-transitory computer readable media. The non-transitory computer readable media includes various types of tangible storage media. Examples of the non-transitory computer readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs, CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs). The program may also be provided to the computer by various types of transitory computer readable media. Examples of the transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer readable media can provide the program to the computer via a wired communication path such as an electric wire and an optical fiber, or via a wireless communication path.

[0086] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-mentioned embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present invention.

[0087] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes. <Additional Notes> (Appendix 1) A posture estimation device, comprising: at least one memory configured to store instructions; acquiring a target image including one or more people; Detecting key points from the target image; calculating one or more selection factors based on the keypoints, the one or more selection factors including a density of people in the target image, a resolution, or both; selecting an algorithm for keypoint association from predefined algorithms for keypoint association based on the one or more selection factors; and performing keypoint association on the keypoints using the selected algorithm to divide the keypoints into one or more keypoint groups, each group including keypoints from the same person; For each of the keypoint groups, estimating a pose of the person corresponding to the keypoint group based on the keypoints included in the keypoint group; and at least one processor configured to execute the instructions to perform the above steps. (Appendix 2) The types of key points include a right shoulder and a left shoulder, The calculation of the density of people in the target image includes: for each left shoulder key point, linking the left shoulder key point to its nearest right shoulder key point; For each key point of the right shoulder linked to a plurality of key points of the left shoulder, deleting the longest link having the key point of the right shoulder when the length of the longest link exceeds a predetermined multiple of the length of the shortest link having the key point of the right shoulder; and calculating the density of people in the target image based on a number of right shoulder key points linked with a number of left shoulder key points. (Appendix 3) The types of key points include a right shoulder and a left shoulder, The calculation of the resolution of the person in the target image comprises: for each left shoulder key point, linking the left shoulder key point to its nearest right shoulder key point; For each key point of the right shoulder linked to a plurality of key points of the left shoulder, deleting the longest link having the key point of the right shoulder when the length of the longest link exceeds a predetermined multiple of the length of the shortest link having the key point of the right shoulder; and calculating the resolution of the person in the target image based on a number of the links remaining after the removal and a number of the links whose length is less than a threshold defined based on one of the dimensions of the target image. (Appendix 4) 4. The pose estimation apparatus of claim 1, wherein the predefined algorithm comprises a midpoint algorithm, a direction map algorithm, a position map algorithm, or two or three of them. (Appendix 5) The selection algorithm for keypoint association is determining whether the selection factor is less than a selection factor threshold; selecting a first algorithm for keypoint association when the selection factor is less than the selection factor threshold; selecting a second algorithm for keypoint association when the selection factor is greater than or equal to the selection factor threshold. (Appendix 6) The selection algorithm for keypoint association is determining whether the resolution of the person in the target image is less than the resolution threshold; selecting a midpoint algorithm when the resolution of the person in the target image is less than the resolution threshold; and selecting an orientation map algorithm or a position map algorithm when the resolution of the person in the target image is greater than or equal to the resolution threshold. (Appendix 7) The selection algorithm for keypoint association is determining whether the density of people in the target image is less than a density threshold; selecting a midpoint algorithm or a direction map algorithm when the density of people in the target image is less than the density threshold; and selecting a position map algorithm when the density of people in the target image is greater than or equal to the density threshold. (Appendix 8) 1. A method for pose estimation implemented by one or more computers, comprising: acquiring an image of a target in which one or more people are captured; Detecting key points from the target image; calculating one or more selection factors based on the keypoints, the one or more selection factors including a density of people in the target image, a resolution, or both; selecting an algorithm for keypoint association from predefined algorithms for keypoint association based on the one or more selection factors; performing keypoint association on the keypoints using the selected algorithm to divide the keypoints into one or more keypoint groups, each group containing keypoints from the same person; and for each of the keypoint groups, estimating a pose of the person corresponding to the keypoint group based on the keypoints included in the keypoint group. (Appendix 9) The types of key points include a right shoulder and a left shoulder, The calculation of the density of people in the target image includes: for each left shoulder keypoint, linking the left shoulder keypoint to its closest right shoulder keypoint; for each right shoulder key point linked to a plurality of left shoulder key points, deleting the longest link having the right shoulder key point when the length of the longest link exceeds a predetermined multiple of the length of the shortest link having the right shoulder key point; and calculating the density of people in the target image based on a number of right shoulder key points linked with a number of left shoulder key points. (Appendix 10) The types of key points include a right shoulder and a left shoulder, The calculation of the resolution of the person in the target image comprises: for each left shoulder keypoint, linking the left shoulder keypoint to its closest right shoulder keypoint; for each right shoulder key point linked to a plurality of left shoulder key points, deleting the longest link having the right shoulder key point when the length of the longest link exceeds a predetermined multiple of the length of the shortest link having the right shoulder key point; calculating the resolution of people in the target image based on a number of the links remaining after the removal and a number of the links that have a length less than a threshold defined based on one of the dimensions of the target image. (Appendix 11) 11. The pose estimation method of any one of claims 8 to 10, wherein the predefined algorithm comprises a midpoint algorithm, an orientation map algorithm, a position map algorithm, or two or three thereof. (Appendix 12) The selection algorithm for keypoint association is determining whether the selection factor is less than a selection factor threshold; selecting a first algorithm for keypoint association when the selection factor is less than the selection factor threshold; selecting a second algorithm for keypoint association when the selection factor is greater than or equal to the selection factor threshold. (Appendix 13) The selection algorithm for keypoint association is determining whether the resolution of the person in the target image is less than the resolution threshold; selecting a midpoint algorithm when the resolution of the person in the target image is less than the resolution threshold; selecting an orientation map algorithm or a position map algorithm when the resolution of the person in the target image is greater than or equal to the resolution threshold. (Appendix 14) The selection algorithm for keypoint association is determining whether the density of people in the target image is less than a density threshold; selecting a midpoint algorithm or a direction map algorithm when the density of people in the target image is less than the density threshold; selecting a position map algorithm when the density of people in the target image is greater than or equal to the density threshold. (Appendix 15) A non-transitory computer-readable storage medium storing a program, the program being capable of executing, on one or more computers: acquiring an image of a target in which one or more people are captured; Detecting key points from the target image; calculating one or more selection factors based on the keypoints, the one or more selection factors including a density of people in the target image, a resolution, or both; selecting an algorithm for keypoint association from predefined algorithms for keypoint association based on the one or more selection factors; performing keypoint association on the keypoints using the selected algorithm to divide the keypoints into one or more keypoint groups, each group containing keypoints from the same person; and for each of the keypoint groups, estimating a pose of the person corresponding to the keypoint group based on the keypoints included in the keypoint group. (Appendix 16) The types of key points include a right shoulder and a left shoulder, The calculation of the density of people in the target image includes: for each left shoulder key point, linking the left shoulder key point to its nearest right shoulder key point; For each key point of the right shoulder linked to a plurality of key points of the left shoulder, deleting the longest link having the key point of the right shoulder when the length of the longest link exceeds a predetermined multiple of the length of the shortest link having the key point of the right shoulder; and calculating the density of people in the target image based on a number of right shoulder key points linked with a number of left shoulder key points. (Appendix 17) The types of key points include a right shoulder and a left shoulder, The calculation of the resolution of the person in the target image comprises: for each left shoulder key point, linking the left shoulder key point to its nearest right shoulder key point; For each key point of the right shoulder linked to a plurality of key points of the left shoulder, deleting the longest link having the key point of the right shoulder when the length of the longest link exceeds a predetermined multiple of the length of the shortest link having the key point of the right shoulder; and calculating the resolution of the people in the target image based on the number of links remaining after the removal and the number of links that are less than a threshold length defined based on one of the dimensions of the target image. (Appendix 18) 18. The storage medium of any one of claims 15 to 17, wherein the predefined algorithm includes a midpoint algorithm, a direction map algorithm, a position map algorithm, or two or three thereof. (Appendix 19) The selection algorithm for keypoint association is determining whether the selection factor is less than a selection factor threshold; selecting a first algorithm for keypoint association when the selection factor is less than the selection factor threshold; and selecting a second algorithm for keypoint association when the selection factor is greater than or equal to the selection factor threshold. (Appendix 20) The selection algorithm for keypoint association is determining whether the resolution of the person in the target image is less than the resolution threshold; selecting a midpoint algorithm when the resolution of the person in the target image is less than the resolution threshold; and selecting an orientation map algorithm or a position map algorithm when the resolution of the person in the target image is greater than or equal to the resolution threshold. (Appendix 21) The selection algorithm for keypoint association is determining whether the density of people in the target image is less than a density threshold; selecting a midpoint algorithm or a direction map algorithm when the density of people in the target image is less than the density threshold; and selecting a position map algorithm when the density of people in the target image is greater than or equal to the density threshold. [Explanation of symbols]

[0088] 10 Target image 22 Right shoulder key point 24 Left shoulder key point 1000 Computers 1020 Bus 1040 Processor 1060 Memory 1080 storage device 1100 Input / Output Interface 1120 Network Interface 2000 Posture estimation device 2020 Acquisition Department 2040 Keypoint Detector 2060 Algorithm Selection Section 2080 Keypoint Association 2100 Estimation Department

Claims

1. A posture estimation device, comprising: at least one memory configured to store instructions; acquiring a target image including one or more people; Detecting key points from the target image; calculating one or more selection factors based on the keypoints, the one or more selection factors including a density of people in the target image, a resolution, or both; selecting an algorithm for keypoint association from predefined algorithms for keypoint association based on the one or more selection factors; and performing keypoint association on the keypoints using the selected algorithm to divide the keypoints into one or more keypoint groups, each group including keypoints from the same person; For each of the keypoint groups, estimating a pose of the person corresponding to the keypoint group based on the keypoints included in the keypoint group; and at least one processor configured to execute the instructions to perform the steps of:

2. The types of key points include a right shoulder and a left shoulder, The calculation of the density of people in the target image includes: for each left shoulder key point, linking the left shoulder key point to its nearest right shoulder key point; For each key point of the right shoulder linked to a plurality of key points of the left shoulder, deleting the longest link having the key point of the right shoulder when the length of the longest link exceeds a predetermined multiple of the length of the shortest link having the key point of the right shoulder; and calculating the density of people in the target image based on a number of right shoulder key points linked with a number of left shoulder key points.

3. The types of key points include a right shoulder and a left shoulder, The calculation of the resolution of the person in the target image comprises: for each left shoulder key point, linking the left shoulder key point to its nearest right shoulder key point; For each key point of the right shoulder linked to a plurality of key points of the left shoulder, deleting the longest link having the key point of the right shoulder when the length of the longest link exceeds a predetermined multiple of the length of the shortest link having the key point of the right shoulder; and calculating the resolution of the person in the target image based on a number of the links remaining after the removal and a number of the links whose length is less than a threshold defined based on one of the dimensions of the target image.

4. The pose estimation apparatus of claim 1 , wherein the predefined algorithm comprises a midpoint algorithm, a direction map algorithm, a position map algorithm, or two or three of them.

5. The selection algorithm for keypoint association is determining whether the selection factor is less than a selection factor threshold; selecting a first algorithm for keypoint association when the selection factor is less than the selection factor threshold; and selecting a second algorithm for keypoint association when the selection factor is greater than or equal to the threshold selection factor.

6. The selection algorithm for keypoint association is determining whether the resolution of the person in the target image is less than the resolution threshold; selecting a midpoint algorithm when the resolution of the person in the target image is less than the resolution threshold; and selecting an orientation map algorithm or a position map algorithm when the resolution of the person in the target image is greater than or equal to the resolution threshold.

7. The selection algorithm for keypoint association is determining whether the density of people in the target image is less than a density threshold; selecting a midpoint algorithm or a direction map algorithm when the density of people in the target image is less than the density threshold; and selecting a position map algorithm when the density of people in the target image is greater than or equal to the density threshold.

8. 1. A method for pose estimation executed by one or more computers, comprising: acquiring an image of a target in which one or more people are captured; Detecting key points from the target image; calculating one or more selection factors based on the keypoints, the one or more selection factors including a density of people in the target image, a resolution, or both; selecting an algorithm for keypoint association from predefined algorithms for keypoint association based on the one or more selection factors; performing keypoint association on the keypoints using the selected algorithm to divide the keypoints into one or more keypoint groups, each group including keypoints from the same person; and for each of the keypoint groups, estimating a pose of the person corresponding to the keypoint group based on the keypoints included in the keypoint group.

9. The types of key points include a right shoulder and a left shoulder, The calculation of the density of people in the target image includes: for each left shoulder keypoint, linking the left shoulder keypoint to its closest right shoulder keypoint; for each right shoulder key point linked to a plurality of left shoulder key points, deleting the longest link having the right shoulder key point when the length of the longest link exceeds a predetermined multiple of the length of the shortest link having the right shoulder key point; and calculating the density of people in the target image based on a number of right shoulder key points linked with a number of left shoulder key points.

10. The types of key points include a right shoulder and a left shoulder, The calculation of the resolution of the person in the target image comprises: for each left shoulder keypoint, linking the left shoulder keypoint to its closest right shoulder keypoint; for each right shoulder key point linked to a plurality of left shoulder key points, deleting the longest link having the right shoulder key point when the length of the longest link exceeds a predetermined multiple of the length of the shortest link having the right shoulder key point; and calculating the resolution of the people in the target image based on a number of the links remaining after the removal and a number of the links whose length is less than a threshold defined based on one of the dimensions of the target image.

11. The pose estimation method according to claim 8 , wherein the predefined algorithm comprises a midpoint algorithm, an orientation map algorithm, a position map algorithm, or two or three of them.

12. The selection algorithm for keypoint association is determining whether the selection factor is less than a selection factor threshold; selecting a first algorithm for keypoint association when the selection factor is less than the threshold selection factor; A method according to claim 8 , further comprising the step of: selecting a second algorithm for keypoint association when the selection factor is greater than or equal to the threshold selection factor.

13. The selection algorithm for keypoint association is determining whether the resolution of the person in the target image is less than the resolution threshold; selecting a midpoint algorithm when the resolution of the person in the target image is less than the resolution threshold; and selecting an orientation map algorithm or a position map algorithm when the resolution of people in the target image is greater than or equal to the resolution threshold.

14. The selection algorithm for keypoint association is determining whether the density of people in the target image is less than a density threshold; selecting a midpoint algorithm or a direction map algorithm when the density of people in the target image is less than the density threshold; and selecting a position map algorithm when the density of people in the target image is greater than or equal to the density threshold.

15. A non-transitory computer-readable storage medium storing a program, the program being capable of executing, on one or more computers: acquiring an image of a target in which one or more people are captured; Detecting key points from the target image; calculating one or more selection factors based on the keypoints, the one or more selection factors including a density of people in the target image, a resolution, or both; selecting an algorithm for keypoint association from predefined algorithms for keypoint association based on the one or more selection factors; performing keypoint association on the keypoints using the selected algorithm to divide the keypoints into one or more keypoint groups, each group including keypoints from the same person; and for each of the keypoint groups, estimating a pose of the person corresponding to the keypoint group based on the keypoints included in the keypoint group.

16. The types of key points include a right shoulder and a left shoulder, The calculation of the density of people in the target image includes: for each left shoulder key point, linking the left shoulder key point to its nearest right shoulder key point; For each key point of the right shoulder linked to a plurality of key points of the left shoulder, deleting the longest link having the key point of the right shoulder when the length of the longest link exceeds a predetermined multiple of the length of the shortest link having the key point of the right shoulder; and calculating the density of people in the target image based on a number of right shoulder key points linked with a number of left shoulder key points.

17. The types of key points include a right shoulder and a left shoulder, The calculation of the resolution of the person in the target image comprises: for each left shoulder key point, linking the left shoulder key point to its nearest right shoulder key point; For each key point of the right shoulder linked to a plurality of key points of the left shoulder, deleting the longest link having the key point of the right shoulder when the length of the longest link exceeds a predetermined multiple of the length of the shortest link having the key point of the right shoulder; Calculating the resolution of the person in the target image based on the number of links remaining after the removal and the number of links that are less than a threshold length defined based on one of the dimensions of the target image.

18. 18. The storage medium of claim 15, wherein the predefined algorithm comprises a midpoint algorithm, a direction map algorithm, a position map algorithm, or two or three of them.

19. The selection algorithm for keypoint association is determining whether the selection factor is less than a selection factor threshold; selecting a first algorithm for keypoint association when the selection factor is less than the threshold selection factor; and selecting a second algorithm for keypoint association when the selection factor is greater than or equal to the threshold selection factor.

20. The selection algorithm for keypoint association is determining whether the resolution of the person in the target image is less than the resolution threshold; selecting a midpoint algorithm when the resolution of the person in the target image is less than the resolution threshold; and selecting an orientation map algorithm or a position map algorithm when the resolution of people in the target image is greater than or equal to the resolution threshold.

21. The selection algorithm for keypoint association is determining whether the density of people in the target image is less than a density threshold; selecting a midpoint algorithm or a direction map algorithm when the density of people in the target image is less than the density threshold; and selecting a location map algorithm when the density of people in the target image is greater than or equal to the density threshold.

Images