Image processing device, image processing method, and program

Abstract

To provide an image processing apparatus, an image processing method, and a program that correctly count the number of detection targets either in a case where shielding objects shield partly or entirely the detection targets or in a case where the shielding objects are present without shielding the detection target.SOLUTION: An image processing apparatus 200 includes a first detection unit that detects a person in an input image; a second detection unit that detects a specific object in the input image; and a specification unit that specifies a combination of the detected person and the detected object indicating an identical person.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a technique for detecting a specific object from an image. [Background technology]

[0002] In recent years, systems have been proposed that perform image analysis processing such as estimating the number of people and people flow for images captured by imaging devices such as security cameras. Patent Document 1 prepares human models for warm weather, cold weather, and rainy weather, and counts the number of people captured in an input image by comparing these human models with the change area of ​​the input image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2015-528614 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present invention is to accurately count detection targets in both cases where a shielding object shields a part or the whole of the detection targets, and where a shielding object exists without shielding the detection targets. [Means for solving the problem]

[0005] The present invention is characterized by having a first detection means for detecting a person in an input image, a second detection means for detecting a specific object in the input image, and an identification means for identifying a combination of the detected person and the detected object that indicates the same person. Effect of the Invention

[0006] The detection targets can be counted accurately in both cases where the detection targets are partially or entirely blocked by an obstruction, and where the detection targets are present but not blocked by an obstruction. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 illustrates a problem in counting the number of detection targets by extending the prior art. [Diagram 2] FIG. 2 illustrates an example of a hardware configuration of an image processing apparatus. [Diagram 3] FIG. 2 is a diagram illustrating an example of a functional configuration of the image processing apparatus according to the embodiment. [Figure 4] 4 is a flowchart showing a flow of image processing by the image processing device. [Diagram 5] FIG. 13 is a diagram illustrating an example of division of an input image. [Figure 6] 11A and 11B are diagrams illustrating examples of small images and head density distributions corresponding to the small images. [Figure 7] FIG. 13 is a diagram showing an example of associating a head with an umbrella based on distance. [Figure 8] FIG. 13 is a diagram showing an example of associating a head with an umbrella based on a vector field map. [Figure 9] FIG. 13 is a diagram showing an example in which a head and an umbrella are associated with each other based on midpoints. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Technology that can estimate the number of people and the flow of people from images makes it possible to detect congestion in public spaces and grasp the flow of people when crowded, and is expected to help alleviate congestion during events and provide appropriate evacuation guidance during disasters. Here, in the case of a method that estimates the number of detection targets (human heads in this case) based on a density map, the number of detection targets can be counted accurately even in a highly crowded area as long as the detection targets are visible. However, this method has the problem that the estimation accuracy deteriorates when the detection targets are obstructed by an obstruction (an umbrella in this case). For example, when counting the number of people, on a rainy day, umbrellas can partially or completely hide people's heads, degrading the estimation accuracy of the density map and, as a result, the estimation accuracy of the number of people also deteriorates.

[0009] A simple way to avoid the above problem is to train an estimator that estimates the density maps of both the detection target and the occluder. This method can count the number of detection targets well when the detection target is either not accompanied by an occluder or is completely occluded by an occluder. For example, Fig. 1(a) is an input image showing a situation where two heads are completely occluded by an umbrella and one head is not occluded. When this image is input to the estimator described above, the estimation results of a head density map (Fig. 1(b)) that estimates the human head and an umbrella density map (Fig. 1(c)) that estimates the position of the umbrella are output. If the estimator described above works ideally, the sum of the values ​​of each pixel in the head density map in Fig. 1(b) is about 1, and the sum of the values ​​of each pixel in the umbrella density map in Fig. 1(c) is about 2, which is about 3 in total. This means that there are about three people in Fig. 1(a).

[0010] However, this simple method has a poor estimation accuracy when the detection target appears with an obstruction, but is partially or completely hidden by the obstruction. For example, Fig. 1(d) is an input image showing a situation where two people are holding umbrellas, but their heads are not partially or completely covered by the umbrella. When this image is input to the estimator described above, the estimation results of a head density map (Fig. 1(e)) that estimates the heads of people and an umbrella density map (Fig. 1(f)) that estimates the position of the umbrella are output. If the estimator described above works ideally, the sum of the values ​​of each pixel in the head density map in Fig. 1(e) is about 3, and the sum of the values ​​of each pixel in the umbrella density map in Fig. 1(f) is about 2, totaling about 5. This means that there are about 5 people in Fig. 1(d). Since there are 3 people in Fig. 1(d), the number of people is mistakenly counted higher than the actual number. In other words, the method of comparing a human model with a change area can count the number of people with high accuracy when the degree of congestion is low, but when the degree of congestion is high and people overlap, it becomes difficult to compare the human model with the change area, and the accuracy of the counting deteriorates. In response to such a problem, an image processing device is provided that can accurately count the detection target in both cases where an obstruction obstructs part or all of the detection target, and where an obstruction does not obstruct the detection target but exists.

[0011] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the configurations shown in the drawings.

[0012] First Embodiment 2 shows an example of the hardware configuration of the image processing device 200 according to the present embodiment. The image processing device 200 has, as a hardware configuration, a CPU 201, a RAM 202, a ROM 203, a storage device 204, a GPU 205, an input unit 206, an output unit 207, and an I / F unit 208, which are connected to each other via a system bus 209. The CPU 201 reads and executes an OS and other programs stored in the ROM 203 and the storage device 204 using the RAM 202 as a work memory, controls each component connected to the system bus 209, and performs calculations and logical judgments for various processes. Note that the processing executed by the CPU 101 or the GPU 205 includes the image processing of the embodiment. The storage device 204 is an external memory, and stores programs to be processed by the image processing device 200. The GPU (graphics processing unit) 205 executes calculations such as learning processing and image recognition processing. It is not necessary to use a single GPU for the arithmetic processing, and one or more CPUs, ASICs (Application Specific Integrated Circuits), FPGAs (Field Programmable Gate Arrays), DSPs (Digital Signal Processors), etc. may be used. The input unit 206 is a human interface device or the like, and performs processing related to input of information, etc. Specifically, it is a touch panel, a keyboard, a mouse, or a robot controller. The output unit 207 is a display or the like, and presents the processing results of the image processing device 200 to the user. The display device may be of any type, such as a liquid crystal display device, a projector, or an LED indicator. The I / F unit 208 connects a camera or the like and inputs a captured image to the image processing device 200. The I / F unit 208 is a wired interface such as a universal serial bus, Ethernet, or optical cable, or a wireless interface such as Wi-Fi or Bluetooth.

[0013] 3 shows an example of the functional configuration of an image processing device 200 according to this embodiment. The image processing device 200 has, as its functional configuration, an image acquisition unit 301, an area division unit 302, a first detection unit 303, a second detection unit 304, an identification unit 305, and a determination unit 306.

[0014] The image acquisition unit 301 acquires an input image. The input image acquired by the image acquisition unit 301 may be an image captured by a security camera or the like, an image recorded in a storage device such as a hard disk, or an image received via a network such as the Internet. In the image processing device 200 of this embodiment, the input image acquired by the image acquisition unit 301 is used as a target for image analysis. The input image from the image acquisition unit 301 is sent to the region division unit 302.

[0015] The region division unit 302 divides the image acquired by the image acquisition unit 301 into predetermined small regions. Next, each small region is resized to a predetermined size to generate a small image. The resized size is adjusted to match the image size to be input to a trained model used in the subsequent detection process, for example.

[0016] The first detection unit 303 estimates the position of a detection target (e.g., a person's head) for each small image divided by the region division unit 302. Here, the detection target is a person, and a first trained model for detecting a person's head from an image is used. The first trained model outputs a likelihood for each region of the input image, and indicates a high likelihood at a position in the input image where the person's head is likely to be. A position indicating a likelihood equal to or greater than a predetermined threshold is the position where the person's head is located.

[0017] The second detection unit 304 estimates the position of an obstruction (e.g., an umbrella) for each small image divided by the region division unit 302. Here, a second trained model that detects the position of a specific object (obstruction) from an input image is used.

[0018] The identification unit 305 associates the detection target with the obstructing object for each small image divided by the region division unit 302, based on the detection target position estimated by the first detection unit 303 and the obstructing object position estimated by the second detection unit 304. As a result, the images are divided into a group of only the detection target, a group of the detection target and the obstructing object corresponding to each other, and a group of only the obstructing object.

[0019] The determination unit 306 determines the number of detection targets in the input image. For each small image divided by the region division unit 302, the number of groups obtained by the identification unit 305 is counted. The number of groups in all small images is summed up to obtain the number of detection targets in the input image.

[0020] An example of the process flow of the image processing device 200 according to this embodiment will be described with reference to Fig. 4. Hereinafter, an example will be described in which the detection target is a person's head and the obstruction is an umbrella, but the image analysis process is not limited to this. For example, the detection target may be a person's eye and the obstruction may be sunglasses.

[0021] In S401, the image acquisition unit 301 acquires an input image. In S402, the region division unit 302 divides the input image acquired by the image acquisition unit 301 into N small regions according to a predetermined division method. FIG. 5 is an example of region division. Each rectangle in FIG. 5 represents a small region obtained by dividing the input image 500. In FIG. 5, a division method is adopted in which the ratio between the size of each small region and the size of the human body reflected in each small region is approximately constant. Next, the region division unit 302 resizes each small region to a predetermined size to make it a small image. Since the subsequent processing can be performed independently for each small region, there is an advantage that the overall processing can be speeded up by performing parallel processing. However, it is not essential to divide the image into small regions, and it is also possible to treat the entire input image as one small region. In addition, resizing is not essential, and it is also possible to treat the small region as it is as a small image.

[0022] In S403, the first detection unit 303 detects a person in the input image. The first detection unit 303 uses a first trained model for detecting a person's head from an image. An estimation process of the person's head position is performed for the small images divided by the region division unit 302. Any known method can be used for estimating the head position. A method of estimating the head position via density distribution will be described below.

[0023] In the method of estimating the head position via density distribution, first, the density distribution of the head is estimated from a small image, and then the head position is estimated from the density distribution of the head. Each of these will be described in detail. The density distribution from a small image is estimated by inputting the small image in advance to a density estimator (first trained model) that inputs an image and outputs the density distribution of the head. Here, the density distribution of the head represents a location in an input image where the head is estimated to be present. An example is shown in FIG. 6. In FIG. 6, a density distribution 603 is calculated in the output 602 of the density estimator corresponding to the head position of a person 601 in an input image 600. The density estimator is trained in advance based on a known machine learning method such as support vector regression or deep learning. The head position can be estimated from the density distribution of the head by various methods. For example, the coordinates that are maximum or equal to or greater than a reference value in the density distribution can be regarded as the head position. Alternatively, a head position estimator that inputs a density distribution and outputs a head position can be trained in advance based on a known machine learning method such as deep learning and used.

[0024] The head position estimation method has been described above. The head position estimation method is not limited to the above. For example, a method of estimating the head position using a head detector may be used. The head detector must be trained based on a known machine learning method such as a support vector machine or deep learning so as to input an image and output the location of the head position. The location of the head position is expressed in the form of a rectangle or an ellipse.

[0025] In S404, the second detection unit 304 detects a specific object (obstruction) in the input image. A second trained model is used to detect the position of the specific object (obstruction) from the input image. The second detection unit 304 performs an estimation process of the position of the obstruction (here, an umbrella) for all of the small images divided by the region division unit 302. This estimation process can be realized by replacing the person's head with an umbrella in the method described in S403. For example, it is possible to define a density distribution with the top of the umbrella as the representative point and center, and estimate the position of the umbrella via the density distribution of the umbrella.

[0026] In S405, the identification unit 305 identifies combinations of detected people and obstructing objects that indicate the same person. For all small images divided by the area division unit 302, a process of matching head positions with umbrella positions is performed. The matching process is a process of matching a person's head with an umbrella that is estimated to be held by the person on a one-to-one basis. As a result of the matching, the images are divided into a group where the detected object and obstructing object correspond one-to-one, a group where there is only the detected object, and a group where there is only the obstructing object. Various methods can be applied to the matching.

[0027] The first matching method is to define a cost (first score) based on the distance between each head and each umbrella, and match them so that the overall cost is minimized. Note that the combination may be determined when the cost becomes smaller than a predetermined value. Existing optimization methods such as minimum cost flow and Hungarian matching can be used for matching. Here, an example of matching using Hungarian matching is explained.

[0028] Assume that the positions of three heads A, B, and C are obtained in S403, and the positions of three umbrellas a, b, and c are obtained in S404. FIG. 7(a) shows the mapping of the positions of the heads and the umbrellas on a two-dimensional coordinate system. FIG. 7(b) shows an example of a cost matrix. Each component of this cost matrix is ​​the square of the distance between the position of the head and the position of the umbrella. However, in order to prevent matching of heads and umbrellas that are far away, three dummies 1, i.e., x1, x2, and x3, which are the same as the number of heads A, B, and C, are added in the column direction. In addition, three dummies 2, i.e., X1, X2, and X3, which are the same as the number of umbrellas a, b, and c, are added in the row direction. Furthermore, the distance that is the threshold for prohibiting matching of the heads and umbrellas is set to r, the distance between the dummies and any point is set to a value r1 that is greater than r, and the distance between the heads and umbrellas that is greater than r is set to r2 that is even greater than r1. That is, r <r1<r2となるようにする。

[0029] Figure 7(c) shows the results of Hungarian matching based on this dummy matrix to obtain associations between heads and umbrellas. The bold frames in Figure 7(c) represent matched head and umbrella pairs. In this example, head B matches with umbrella b, and head C matches with umbrella a. Matched heads and umbrellas are considered to be one group. Heads that match with dummies, such as head A and umbrella c, are treated as independent groups. The remaining dummies are simply ignored, as they match with other dummies. The groups obtained as a result of Hungarian matching are shown as ellipses in Figure 7(d).

[0030] The cost definition is not limited to the above. For example, a cost weighted by the positional relationship between the umbrella and the head may be defined. For example, since an umbrella usually appears on top of the head, a penalty may be added to the cost for pairs in which the umbrella appears in a specific direction (for example, below, depending on the camera installation position) relative to the head.

[0031] The second matching method uses a vector field estimator that takes an input image as input and estimates a vector field map. This vector field map is a two-dimensional map in which each pixel is a two-dimensional vector that indicates the connection between the head and the umbrella. The direction of the vector at a pixel located between a person's head position and the position of the umbrella held by that person is the same as the direction from the head position to the umbrella position. By using such a vector field map, a person's head can be matched to the umbrella held by that person.

[0032] The vector field estimator is trained in advance using a pair of an input image and a correct vector field map created from correct data, using a known method such as deep learning. The correct vector field map can be created, for example, by the following procedure. First, a list of pairs of a person's head position and the position of the umbrella held by that person is prepared as correct data required for creating the correct vector field map. Next, the correct vector field map is initialized as a two-dimensional vector field filled with zero vectors and has the same size as the input image. Next, for each pair of head position and umbrella position in the correct data, a unit vector in the direction from the head position to the umbrella position is repeatedly added to the pixels of the correct vector field map located on the line segment connecting the head position and the umbrella position. For training the vector field estimator, for example, a method based on deep learning disclosed in Cao, Zhe, et al. “Realtime multi-person 2d pose estimation using part affinity fields.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. can be used.

[0033] An example is shown in FIG. 8. FIG. 8(a) is a diagram in which the head position obtained in S403 and the umbrella position obtained in S404 are mapped on a two-dimensional coordinate system. FIG. 8(b) is a diagram showing a vector field obtained by inputting an input image into a vector field estimator. The vector field estimator outputs a vector field that suggests that head C and umbrella a, and head B and umbrella b correspond to each other. Next, a score (second score) based on the vector field map is calculated for all pairs (combinations) between the head obtained in S403 and the umbrella obtained in S404. The score can be defined, for example, as the sum of the inner products of the vector with the head position as the start point and the umbrella position as the end point, and the vector of the corresponding pixel in the vector field map, for pixels on a line segment connecting the head position and the umbrella position. It can be interpreted that the larger this score is, the higher the probability that the head and the umbrella correspond to each other. FIG. 8(c) shows an example of the score. The definition of the score is not limited to this, and any definition can be used. For example, a term may be added to the above score definition that gives a larger value the shorter the distance between the head and the umbrella.

[0034] By associating the heads with the umbrellas so that the sum of the scores is maximized overall, the association between the heads and the umbrellas can be obtained. As an example of association, a method using a minimum cost flow will be described. First, each of the heads obtained in S403 and each of the umbrellas obtained in S404 are regarded as a node. Then, if the score between a certain head and an umbrella is equal to or greater than a predetermined threshold, that is, if matching between the head and the umbrella is permitted, an edge with a capacity of 1 and a cost of (-1) x the above score is extended from the head node to the umbrella node. A start node is added, and edges with a capacity of 1 and a cost of 0 are extended from the start node to all head nodes. A terminal node is added, and edges with a capacity of 1 and a cost of 0 are extended from all umbrella nodes to the terminal node. By calculating the minimum cost flow from the start node to the terminal node in the network generated in this way, the association between the heads and the umbrellas can be obtained so that the sum of the scores is maximized. The association method is not limited to this method, and existing optimization methods such as Hungarian matching can be used.

[0035] The third matching method is to use a midpoint detection model (estimator) that detects the "midpoint" that corresponds to the midpoint of the line segment connecting the head and umbrella that are to be matched. The estimation process of this estimator can be realized by replacing the method described in S403 with the midpoint instead of the human head. For example, it is possible to define a density distribution centered on the midpoint and estimate the position of the midpoint via the density distribution of the midpoint.

[0036] An example is shown in Figure 9. Figure 9(a) is a diagram in which the head position obtained in S403, the umbrella position obtained in S404, and the midpoint position obtained by the midpoint estimator are mapped onto a two-dimensional coordinate system.

[0037] It is expected that the midpoints will correspond to one head and one umbrella. Therefore, this correspondence can be achieved by solving a 3D matching problem that associates the head, umbrella, and midpoint. 3D matching can be solved in various ways. For example, the matching between the head and midpoint can be solved by using existing optimization methods such as minimum cost flow or Hungarian matching, and then the matching between the midpoint and umbrella can be solved by using a similar optimization method. In this case, the cost (third score) is designed to be smaller as the distance between the head and midpoint and the distance between the midpoint and umbrella are closer, and as the distance from the line segment connecting the head and umbrella to the midpoint is smaller. In other words, the combination is determined based on the third score obtained according to the distance from each line segment connecting the detected person and the detected object to each midpoint. The 3D matching problem can also be solved by greedily assigning the closest head and umbrella to each midpoint. In Figure 9(b), the groups obtained as a result of matching are shown as ellipses. By including an intermediate point in matching the head and umbrella, the probability of correctly matching the head and umbrella that are far apart is increased.

[0038] Here, the identification unit 305 may estimate only the number of groups without explicitly grouping the heads and umbrellas. For example, the number of heads obtained in S403 is K, the number of umbrellas obtained in S404 is L, and the number of midpoints obtained by the midpoint estimator is M. Ideally, if one head and one umbrella correspond to one midpoint, it can be approximated that there are M groups consisting of heads, umbrellas, and midpoints, KM groups consisting of heads alone, and LM groups consisting of umbrellas alone. Therefore, it can be estimated that there are M+(KM)+(LM) groups.

[0039] In S406, the determination unit 306 obtains an estimated number of people in the input image by summing up the number of groups determined in S405 across all small images. The number of people in the input image may be determined by counting the number of identified combinations and the number of detected people or detected objects not included in the identified combinations.

[0040] In S407, the image processing device 200 determines whether to end the image analysis process. If the image analysis process is not to be ended, the process moves to S401. The process may be ended when a user instructs the process to end, or may be ended after the process has been executed for a certain period of time. In addition, the determination unit may output an instruction to end the process when a certain number of people have been counted.

[0041] As described above, according to the image processing device 200 of the first embodiment, the number of detection targets can be accurately counted in both cases where an obstruction obstructs part or all of the detection targets, and where an obstruction exists without obstructing the detection targets.

[0042] Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and variations are possible within the scope of the gist of the present invention described in the claims.

[0043] A program for implementing one or more functions in the signal processing according to the present invention can be supplied to a system or device via a network or a storage medium, and can be implemented by being read and executed by one or more processors of a computer in the system or device. It can also be implemented by a circuit (e.g., ASIC) that implements one or more functions.

[0044] The above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these. In other words, the present invention can be implemented in various forms without departing from its technical concept or main characteristics. [Explanation of symbols]

[0045] 200 Image processing device 301 Image Acquisition Unit 302 Area division part 303 First detection unit 304 Second detection unit 305 Specific section 306 Decision Section

Claims

1. a first detection means for detecting a person in an input image; a second detection means for detecting a specific object in the input image; and a specifying means for specifying a combination of the detected person and the detected object that indicates the same person, The image processing device is characterized in that the identification means identifies the combination in which a second score calculated based on a vector field map indicating a positional relationship between a person and the specific object is greater than a predetermined value.

2. The image processing device according to claim 1 , wherein the second score is calculated based on the value of a vector associated with a line segment connecting a position of the detected person and a position of the detected object in the vector field map.

3. The image processing device according to claim 1 or 2, characterized in that the vector field map is obtained by learning a model that estimates the likelihood of a connection between a person and the specific object based on a ground-truth vector having a direction from the specific object to the person for a combination of the person and the specific object.

4. A first detection means for detecting a person in an input image; a second detection means for detecting a specific object in the input image; and a specifying means for specifying a combination of the detected person and the detected object that indicates the same person, the identification means identifies the combination in which a first score calculated based on a distance between the detected person and the detected object is smaller than a predetermined value; An image processing device characterized in that the first score is calculated for each combination of a dummy generated for each of the detected person and the detected object, and the detected person and the detected object.

5. The image processing device described in Claim 4, characterized in that the first score is made larger when the object is located in a predetermined direction of the person based on the positions of the detected person and the detected object.

6. A first detection means for detecting a person in an input image; a second detection means for detecting a specific object in the input image; A determination means for determining a combination of the detected person and the detected object that indicates the same person; a third detection means for detecting an intermediate point in the input image based on a model that outputs an intermediate point between a person and the specific object, The image processing device according to claim 1, wherein the identification means identifies the combination based on a positional relationship between the detected person, the detected object, and the detected midpoint.

7. The image processing device according to claim 6, wherein the identification means identifies the combination based on a third score obtained according to a distance from a line segment connecting the detected person and the detected object to the midpoint.

8. An image processing device as described in any one of claims 1 to 7, further comprising a determination means for determining the number of people in the input image based on the detected people, the detected objects, and the identified combination.

9. The determining means counts each of the identified combinations and the detected persons or the detected objects not included in the identified combinations as one person, The image processing apparatus according to claim 8 , wherein the number of people in the input image is determined.

10. The image processing device according to claim 1 , wherein the first detection means detects a person by inputting the input image into a trained model that detects a position of a person's head.

11. The image processing device according to claim 1 , wherein the second detection means detects the specific object by inputting the input image to a trained model for detecting the specific object.

12. A program for causing a computer to function as each of the means included in the image processing device according to any one of claims 1 to 11.

13. A first detection step in which a first detection means detects a person in an input image; a second detection step in which a second detection means detects a specific object in the input image; and a specifying step of specifying a combination of the detected person and the detected object that indicates the same person by using a specifying means; An image processing method characterized in that, in the identification process, the combination is identified such that a second score calculated based on a vector field map indicating a positional relationship between a person and the specific object is greater than a predetermined value.

14. A first detection step in which a first detection means detects a person in an input image; a second detection step in which a second detection means detects a specific object in the input image; and a specifying step of specifying a combination of the detected person and the detected object that indicates the same person by using a specifying means; In the identifying step, the combination is identified such that a first score calculated based on a distance between the detected person and the detected object is smaller than a predetermined value; An image processing method characterized in that the first score is calculated for each combination of a dummy generated for each of the detected person and the detected object, and the detected person and the detected object.

15. A first detection step in which a first detection means detects a person in an input image; a second detection step in which a second detection means detects a specific object in the input image; an identifying step of identifying a combination of the detected person and the detected object that indicates the same person by an identifying means; a third detection step in which a third detection means detects an intermediate point in the input image based on a model that outputs an intermediate point between a person and the specific object; an image processing method comprising: identifying the combination based on the respective positional relationships between the detected person, the detected object, and the detected intermediate point in the identifying step;

Images