Information processing apparatus, information processing method, and program

Abstract

To accurately estimate the number of objects to be measured from an image.SOLUTION: An information processing apparatus acquires, from an input image, likelihood information representing distribution of likelihoods indicating a probability that specific objects are present, and on the basis of, the likelihood information, when the number of likelihoods included in a subregion of the input image is equal to or more than a first predetermined value, determines the subregion as a region for which a threshold is adjusted to be lower. The information processing apparatus excludes the likelihoods less than the adjusted threshold, of the likelihoods included in the determined subregion, from ones to be counted and counts the number of likelihoods for every subregion to estimate the number of the specific objects.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a technique for detecting a specific object from an image.

Background Art

[0002] Patent Document 1 discloses a technique related to setting a threshold value of likelihood for face region detection corresponding to a shooting scene. For example, when estimating the number of objects to be measured using only likelihood, since all likelihoods with low probability are included in the counting, by using a threshold value, likelihoods with low probability are excluded from the counting target.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As described above, when excluding likelihoods with low probability using a threshold value, it is possible to reduce false detection in which an object that is not a measurement target is erroneously counted. However, when a specific object is correctly detected (correctly detected) but its likelihood is low, the correctly detected likelihood may be excluded as false detection, resulting in a decrease in the likelihood of correct detection.

[0005] Therefore, an object of the present invention is to accurately estimate the number of objects to be measured from an image.

Means for Solving the Problems

[0006] The information processing apparatus of the present invention calculates a likelihood indicating the probability that a specific object exists from an input image including the position of the specific object, the An acquisition means for acquiring likelihood information representing a likelihood distribution, and based on the likelihood information, when it is determined that the likelihood included in a partial region of the input image is equal to or greater than a predetermined value, a determination means for determining as a region to adjust a threshold value for the partial region to be lower, and excluding from a counting target likelihoods less than the adjusted threshold value among the likelihoods included in the partial region determined by the determination means, and estimating the number of the specific objects by counting the likelihoods for each partial region.

Advantages of the Invention

[0007] According to the present invention, the number of objects to be measured can be accurately estimated from an image.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments will be described with reference to the drawings. Note that the configurations shown in the following embodiments are merely examples and are not limited to the illustrated configurations. <First Embodiment> There are an increasing number of cases where images captured by surveillance cameras or images stored in a storage device after being captured are analyzed and utilized. For example, there is an image analysis system that uses, as objects to be counted, specific objects such as human bodies or heads existing in an image, estimates the likelihood indicating the probability of being a specific object, and estimates the approximate number of objects to be counted included in the image based on the likelihood. In this embodiment, when counting target objects in an image, by changing the threshold for counting target objects for each region, undetected or misdetected target objects are suppressed, and the detection accuracy is improved. FIG. 1 is a diagram showing an example of the system configuration to which the information processing apparatus 100 of this embodiment is applied. The system in this embodiment includes an information processing apparatus 100, an imaging apparatus 110, a recording apparatus 120, and a display 130. The information processing apparatus 100, the imaging apparatus 110, and the recording apparatus 120 are interconnected via a network 140.

[0010] The network 140 is realized by, for example, a plurality of routers, switches, cables, etc. compliant with a communication standard such as ETHERNET (registered trademark). Note that the network 140 may be realized by the Internet, a wired LAN (Local Area Network), a wireless LAN (Wireless Lan), a WAN (Wide Area Network), or the like.

[0011] The imaging device 110 is a device that captures an object such as a subject. In the present embodiment, a surveillance camera is cited as an example of the imaging device 110, and it is assumed that a specific object such as a human body or a head shown in the image captured by the surveillance camera is the object to be estimated and measured in the present embodiment (hereinafter referred to as the measurement target object). The imaging device 110 associates the data of the image acquired by imaging, the information of the imaging date and time when the image was captured, and the identification information which is the information for identifying the imaging device, and transmits them to external devices such as the information processing device 100 and the recording device 120 via the network 140. Hereinafter, the data of the image captured by the imaging device 110 will be simply referred to as an image, and the imaging date and time and the identification information of the image will be referred to as associated information. In the system according to the present embodiment, although the imaging device 110 is one, it may be plural. That is, a plurality of imaging devices 110 may be connected to the information processing device 100 and the recording device 120 via the network 140. In this case, the information processing device 100 and the recording device 120 use the identification information included in the associated information of the image transmitted from the imaging device to determine which imaging device among the plurality of imaging devices captured the transmitted image.

[0012] The recording device 120 records by associating the image captured by the imaging device 110 with the associated information. Then, in accordance with a request from the information processing device 100, the recording device 120 transmits the recorded information (image and associated information) to the information processing device 100. Hereinafter, the image transmitted from the imaging device 110 or the recording device 120 and input to the information processing device 100 will be appropriately referred to as an input image.

[0013] The information processing apparatus 100 estimates the likelihood indicating the probability of a specific object from an input image sent from the imaging apparatus 110 or the recording apparatus 120, acquires likelihood information representing the distribution of the likelihood, and estimates the number of specific objects based on the likelihood. Further, the information processing apparatus 100 generates a partial region for adjusting the likelihood threshold based on the likelihood information. Furthermore, when there are a certain number or more of high likelihoods equal to or higher than a predetermined value within the partial region for threshold adjustment, the information processing apparatus 100 determines that the partial region is a region where the threshold is adjusted lower, and determines the likelihoods less than the threshold determined for the partial region as likelihoods outside the measurement target of the specific object. Then, the information processing apparatus 100 excludes the determined likelihoods and counts the number of specific objects. Note that the predetermined value as a comparison target for determining whether the likelihood is high may be a predetermined value (for example, 0.02, etc.), or a value relatively adjusted from the predetermined value according to the estimation result (for example, a value relatively adjusted to 0.02 or more according to the estimation result, etc.) may be used. Details of the configuration and information processing of the information processing apparatus 100 in the present embodiment will be described later.

[0014] The display 130 is configured by an LCD (Liquid Crystal Display) or the like, and displays the result of information processing by the information processing apparatus 100, the image captured by the imaging apparatus 110, and the like. The display 130 is connected to the information processing apparatus 100 via a display cable conforming to a communication standard such as HDMI (registered trademark) (High Definition Multimedia Interface). Note that at least one or all of the display 130, the information processing apparatus 100, and the recording apparatus 120 may be provided in a single housing. Also, the result of information processing by the information processing apparatus 100 and the captured image by the imaging apparatus 110 may be displayed not only on the display 130 but also on a display included in an external device such as the following, for example. For example, it may be displayed on a display included in a mobile device such as a smartphone or a tablet terminal connected via the network 140.

[0015] Next, FIG. 2 is a diagram showing an example of the functional blocks of the information processing apparatus 100 according to the present embodiment. Hereinafter, with reference to the functional blocks shown in FIG. 2, information processing in the information processing apparatus 100 according to the present embodiment will be described. In the following description, for example, an example will be given in which each functional unit shown in the functional blocks of FIG. 2 is realized by a personal computer executing an information processing program according to the present embodiment. That is, each functional unit in FIG. 2 is assumed to be realized by a CPU (Central Processing Unit) of a personal computer or the like executing a computer program stored in a ROM (Read Only Memory) or the like. The hardware configuration and the like of a personal computer or the like that realizes each function related to the information processing apparatus 100 of the present embodiment will be described later with reference to FIG. 10.

[0016] The communication unit 200 communicates with the imaging device 110 and the recording device 120 via the I / F 1040 of FIG. 10, which will be described later. The communication unit 200 receives input images such as moving images and still images captured by the imaging device 110, or moving images and still images recorded in the recording device 120, and their related information. Note that a moving image is composed of a plurality of frame images having temporal continuity (or relevance). The communication unit 200 acquires at least one image of the plurality of frame images constituting the moving image or a still image as an image to be processed. Further, the communication unit 200 transmits a control command for controlling the imaging device to the imaging device 110 or transmits a control command for controlling the recording device to the recording device 120. Note that the control command for the imaging device includes, for example, a command for instructing the imaging device to capture an image. Also, the control command for the recording device includes a command for instructing to read an image from the recording device.

[0017] The output control unit 201 outputs the input image, a setting screen for setting information related to the information processing according to the present embodiment, or information indicating the result of the information processing by the information processing apparatus 100 to an external device, or displays it on the display 130. Note that the external device that is the output destination of the information by the output control unit 201 includes other information processing apparatuses (not shown) and the recording device 120. The operation reception unit 202 receives operations performed by the user via an input device (not shown) such as a keyboard, a mouse, or a touch panel.

[0018] The setting unit 203 sets a plurality of partial regions (first partial regions) that are the targets of the estimation process for estimating the number of specific objects in the input image. The first partial region is a counting target region when the number of specific objects estimated from the image by the estimation unit 204 described later is added up and counted. For example, it is a region set by the setting unit 203 based on the position specified on the input image by the user via the operation reception unit 202. Specifically, it is a partial image (batch) obtained by dividing the input image for input to the regression-based estimation method described later. In the present embodiment, for example, it is assumed that the entire region of the input image is set as a predetermined region. For example, if there is a region where no specific object exists, a part of the input image may be set as the predetermined region.

[0019] The setting unit 203 also sets a plurality of partial regions that are the targets of the specific object estimation process by the estimation unit 204 described later so as to cover the predetermined region. At this time, the setting unit 203 sets the individual sizes and positions of the plurality of partial regions based on the size that a specific object can take at each of a plurality of different points on the input image and the information of each position. Details of the plurality of partial regions set for the predetermined region of the input image and examples of setting the sizes and positions of those partial regions will be described later. Note that the setting unit 203 may set, for example, a region designated by the user via the operation reception unit 202 as a partial region that is the target of the process for estimating a specific object. Also, the setting unit 203 sets a threshold value as a comparison target when determining whether to include the likelihood, which is the estimation result of the specific object estimated by the estimation unit 204 described later, in the likelihood to be counted. In the present embodiment, the process of estimating the number of specific objects includes two processes: a process of estimating the likelihood indicating the number of specific objects included in the input image (object estimation process), and a process of estimating the number of specific objects included in the input image by comparing the estimated likelihood with the threshold value (number estimation process). The threshold value mentioned here is a threshold value for determining whether to validate the estimation result of the specific object in the latter number estimation process. Specifically, the threshold value is set to a predetermined value such as 0.05, and the likelihood less than (or less than or equal to) the threshold value is excluded from the counting target during the number estimation process. The details of the likelihood threshold value will be described later.

[0020] The estimation unit 204 executes an estimation process for estimating the number of specific objects in each of the plurality of partial regions (first partial regions) set by the setting unit 203 for a predetermined region of the input image. In the present embodiment, as a method (object estimation process) for the estimation unit 204 to estimate the number of specific objects, for example, an example using a regression-based estimation method is assumed. In the regression-based estimation method, a small image of a certain fixed size s is input, and a regressor (pre-trained recognition model) that outputs the number of specific objects existing in the small image is used to estimate the number of specific objects in a plurality of partial regions within a predetermined region of the input image. When training the regressor, a large number of small images of a fixed size s with known positions of specific objects are prepared, and the regressor is trained based on a machine learning method using the target small images as training data. At this time, in order to improve the estimation accuracy of the number of specific objects, it is desirable that the ratio between the size of the small image (fixed size s) as the training data and the size of the specific objects existing in the small image is substantially constant. Then, the estimation unit 204 uses, as small images, those obtained by resizing the images of each of the plurality of partial regions set in the predetermined region of the input image to the fixed size s, and inputs the small images to the regressor, thereby obtaining, as the output from the regressor, "the position and likelihood (estimated value) of the specific objects within the partial region". Then, the estimation unit 204 obtains, as the number of specific objects (referred to as the estimated number) within the partial region, the value obtained by summing up the likelihoods of the specific objects estimated within the partial region (number estimation process). Note that, for each likelihood, by comparing it with a threshold value, an estimation result that becomes noise can be excluded. In the present embodiment, the total value of the likelihoods excluding the estimation results that become noise is output as the final number estimation result. In the following description, a person is taken as an example of the specific object, but it is not limited thereto, and the specific object may be, for example, a part of the human body such as a person's head, upper body, arm, or leg, or another object such as a car or a license plate.

[0021] The recording unit 205 stores data such as information and images related to the information processing by the information processing apparatus 100. For example, the recording unit 205 records the number of specific objects obtained by the estimation process by the estimation unit 204 for each of a plurality of partial regions set in a predetermined region of the input image. The recording unit 205 also records likelihood information, which is information obtained by the estimation process of the estimation unit 204. In the present embodiment, the likelihood information is assumed to be held by the recording unit 205 of the information processing apparatus 100, but it is not limited thereto. For example, an external device (for example, the recording device 120) connected to the information processing apparatus 100 via the network 140 may hold the likelihood information. When the likelihood information is held by the external device, the communication unit 200 of the information processing apparatus 100 may transmit a command for requesting the likelihood information to the external device and acquire the likelihood information transmitted from the external device according to the command.

[0022] The adjustment unit 206 acquires the likelihood value and the position of the likelihood, which are the likelihood information of the estimation result by the estimation unit 204. Further, based on the likelihood information by the estimation unit 204, the adjustment unit 206 generates a target partial region (second partial region) for threshold adjustment by making it the same as the partial region input to the regressor of the regression-based estimation method or by connecting a plurality of partial regions. The second partial region may be an area common to the first partial region, or may be generated by integrating the first partial regions including approximately the same likelihood based on the likelihood information. Furthermore, the adjustment unit 206 compares the total likelihood of each of the second partial regions with a first predetermined value, and determines, for the partial region having a likelihood greater than the first predetermined value, as an area where no threshold for the number estimation process is set. For example, when the total value of the likelihoods included in a certain partial region is 5.4 and the first predetermined value is 5, since 5.4 > 5, the said partial region becomes a target region for changing the threshold. Note that, for the partial region having a likelihood smaller than the first predetermined value, it is determined as an area where a process of comparing each of the likelihoods included in the partial region with the threshold is performed during the number estimation process. When there are a certain number or more of high likelihoods of a second predetermined value or more within a partial region, the adjustment unit 206 may perform a threshold adjustment process for lowering the threshold for the likelihood within the partial region, or a threshold adjustment process for eliminating (not setting) the threshold for the likelihood within the partial region. For example, when each of the likelihoods included in a certain partial region is (0.8, 0.8, 0.7, 0.5, 0.3, 0.01), each of the likelihoods is compared with a second predetermined value (for example, 0.1). When the number of likelihoods greater than the second predetermined value is a certain number (for example, 4 or more), the said partial region is determined as an area where the threshold is adjusted. In the partial region of this specific example, there are five likelihoods greater than 0.1, and since the number of likelihoods satisfying the condition is 5 > 4, it is determined as an area where the threshold is adjusted. Then, the adjustment unit 206 determines the likelihoods less than the threshold as likelihoods outside the measurement target of the specific object, and excludes the determined likelihoods from the likelihoods estimated by the estimation unit 204 to calculate the number of specific objects. Details of these processes in the adjustment unit 206 will be described later.

[0023] As described above, the information indicating the estimated number of specific objects (e.g., the number of people), which is the sum of likelihoods estimated by the estimation unit 204 and adjusted by the adjustment unit 206, is output to an external device by the output control unit 201. In addition to the information on the estimated number of specific objects, the output control unit 201 causes a display 130, which is an example of an external device, to display information indicating the total processing time in the adjustment unit 206, information indicating the total frame processing time, and the like.

[0024] As described above, the information processing apparatus 100 according to the present embodiment estimates a specific object (such as a human body or a head), which is an object to be measured, existing in an input image, and obtains a likelihood indicating the probability that the estimated specific object exists. Further, based on the likelihood, the information processing apparatus 100 creates a second partial region, and when the likelihood of the second partial region is equal to or greater than a first predetermined value, adjusts the likelihood threshold in the partial region to be lower or eliminated. Then, the information processing apparatus 100 determines that the likelihood less than the threshold is not a counting target of the specific object, excludes the likelihood from the counting, and estimates the number of specific objects by counting the likelihood equal to or greater than the threshold.

[0025] Next, with reference to FIG. 3, the process of setting a plurality of partial regions for a predetermined region of the input image will be described. FIG. 3 is a diagram showing an example of a plurality of partial regions set by the setting unit 203, with the image captured by the imaging device 110 as the input image and the entire input image as the predetermined region.

[0026] As shown in FIG. 3, the setting unit 203 sets a plurality of partial regions 301a, 301b, and 301c for the image 300 input from the imaging device 110. In the present embodiment, the setting unit 203 sets a plurality of partial regions for the image 300 based on the size and position information of the person reflected at each of a plurality of different points on the image 300. The size of each partial region in the image captured by the imaging device 110 depends on the size of a specific object reflected in the captured image. In the example of FIG. 3, a human body (person) is cited as the specific object, and the size of the partial region depends on the size of the human body that appears with different sizes depending on the position in the captured image. For example, when the imaging device 110 is arranged at a position overlooking a horizontal plane such as a floor surface, that is, when the imaging device 110 is installed such that the optical axis of the lens faces below the horizontal axis, the human body appears large in the lower part of the captured image and small in the upper part. Therefore, the size of each of the plurality of partial regions for the image 300 is made larger for the partial region 301a arranged in the lower part of the image than for the partial regions 301b and 301c arranged in the upper part thereof. Similarly, the partial region 301b arranged in the middle part of the image is made to have a larger size than the partial region 301c arranged in the upper part thereof. Note that as the angle formed by the lens optical axis of the imaging device 110 and the horizontal axis approaches 90 degrees, the size of the human body included in the captured image approaches substantially uniform regardless of the position in the captured image, and the difference in the size of the partial regions between the upper and lower parts of the captured image becomes smaller. In the example of FIG. 3, each rectangular region surrounded by a broken line is a partial region set by the setting unit 203. As the calculation parameters for determining the size of the partial region as described above, imaging parameters such as the installation position and installation angle of the imaging device 110 and the zoom ratio may be used, or the size of a human body or the like specified by the user from the captured image may be used.

[0027] For example, when setting the size of a partial area based on a user's specification, the setting unit 203 forms, via the output control unit 201, a GUI (GRAPHICAL USER INTERFACE) for the user to specify the size of the partial area and the like on a display or the like. The user uses the operation reception unit 202 to specify, through the GUI, information on the size and position of the person reflected at each of a plurality of different points within a predetermined area of the input image. For example, when the user performs an operation of specifying the average size of the person reflected at the upper, middle, and lower positions respectively on the image displayed on the GUI, the operation reception unit 202 receives the user instruction. Then, the information on the position of the person based on the instructions for the upper, middle, and lower parts received by the operation reception unit 202 and the information on the average size of the person reflected at that position are recorded by the recording unit 205. Then, the setting unit 203 acquires the information on the position of the person in the upper, middle, and lower parts of the image and the information on the average size of the person at those positions recorded in the recording unit 205.

[0028] Also, the information on the size and position of the person reflected at each of a plurality of different points may be obtained by performing image analysis on an image previously captured by the imaging device 110. For example, the setting unit 203 executes a process of detecting a person using pattern matching or the like on the image previously captured by the imaging device 110, and the recording unit 205 records information associating the position of the person detected from the image and the size of the person at that position. The setting unit 203 acquires, from the recording unit 205, the information on the position and size of the person at each of a plurality of different points detected from the image and recorded in the recording unit 205 as described above.

[0029] Then, the setting unit 203 estimates the size information f(x, y) of a person reflected at an arbitrary position on the image based on the positions of the persons corresponding to a plurality of different points on the image obtained by the above-described user's designation or image analysis, and the sizes of the persons reflected at those positions. The size information f(x, y) of a person at an arbitrary position on the image indicates the average size of the person reflected at the arbitrary position coordinates (x, y) on the image. Assume that the size information f(x, y) can be represented by, for example, x indicating the x coordinate on the image, y indicating the y coordinate on the image, and one or more parameters. For example, assume f(x, y)=ax + by + c. In this example, the unknown parameters are a, b, and c. At this time, the setting unit 203 can obtain the unknown parameters by statistical processing such as the least squares method using the information on the positions and sizes of the persons at a plurality of different points on the image read from the recording unit 205.

[0030] Then, the setting unit 203 sets a plurality of partial regions for the image (a predetermined region of the input image) based on the size information f(x, y) which is the size of a person at an arbitrary position on the image. In the example shown in FIG. 3, as the size of a person at an arbitrary position on the image 300, the size of the person is substantially the same in the horizontal direction of the image 300, and the size of the person becomes smaller from the lower part to the upper part in the vertical direction of the image 300. In the case of the example of FIG. 3, first, the setting unit 203 sets a plurality of partial regions 301a along the lower end of the image 300. At this time, the setting unit 203 sets the partial region 301a such that the ratio of the size of the partial region 301a (referred to as size a) to the size of the person indicated by the size information f(x, y) at the coordinate of the lower end in the partial region 301a is substantially the same as the ratio r corresponding to the learning data.

[0031] Next, the setting unit 203 sets a plurality of partial regions 301b along the upper ends of the plurality of partial regions 301a. At this time, the setting unit 203 sets the partial region 301b such that the ratio of the size of the partial region 301b (referred to as size b) to the size of the person indicated by the size information f(x, y) at the coordinate of the lower end in the partial region 301b is substantially the same as the ratio r corresponding to the learning data. Furthermore, the setting unit 203 sets a plurality of partial regions 301c along the upper ends of the plurality of partial regions 301b. At this time, the setting unit 203 sets the partial regions 301c such that the ratio of the size of the partial region 301c (referred to as size c) to the size of the person indicated by the size information f(x, y) at the coordinate of the lower end within the partial region 301c is substantially the same as the ratio r corresponding to the learning data.

[0032] In this way, the setting unit 203 of the present embodiment sets partial regions for the image such that the ratio of the size of the partial region to the size of a specific object such as a person in the partial region is substantially the same as the ratio r of the size of the small image, which is the learning data, to the size of the specific object shown in the small image. That is, in the present embodiment, by setting partial regions for the image so as to approach the environment of the learning data, the estimation accuracy of the number of specific objects included in the partial regions can be further improved. In the above description with reference to FIG. 3, the partial regions were set in the order from the lower end to the upper end of the image, but the present invention is not limited to this, and the partial regions may be set from other positions.

[0033] Subsequently, with reference to the flowchart shown in FIG. 4, the flow of information processing executed in the information processing apparatus 100 of the present embodiment will be described. By executing the processing of FIG. 4, the information processing apparatus 100 can suppress the likelihood of correctly detected likelihoods decreasing while reducing false detections of objects that are not the measurement target based on the likelihood information that is the result of performing estimation processing on the input image. That is, in the present embodiment, it is possible to suppress the likelihood of a correct detection, which has a high possibility of being a correct estimation result, from not being counted while reducing an estimation result with a low likelihood that is highly likely to be a false detection (suppressing the amount of decrease in the likelihood of correct detection).

[0034] Note that the process shown in FIG. 4 may be executed according to a user instruction in a setting mode in which a likelihood threshold is set for an image captured by the imaging device 110, for example. Further, the process of FIG. 4 may be automatically executed when the setting of the likelihood threshold is effective for an image captured by the imaging device 110, for example. Note that the process of FIG. 4 is realized by the CPU executing a computer program stored in the ROM and is executed by the functional blocks shown in FIG. 2. In the following flowchart, "S" of the reference numerals used represents a processing step.

[0035] First, in S401, the estimation unit 204 acquires likelihood information including the position and likelihood of a specific object based on a learned model for estimating the presence of the specific object in the input image. That is, an estimation process for estimating the above-described specific object (such as a person or a head) and the number thereof is executed. By inputting each of a plurality of divided partial regions (first partial regions) of the input image into the learned model, the likelihood of the specific object included in each partial region is acquired. Note that the likelihood takes a value between 0 (non-existent) and 1 (existent). Here, as the specific object, the position and likelihood of the head of a person are acquired as likelihood information.

[0036] Next, in S402, the adjustment unit 206 determines whether the likelihood threshold setting by the setting unit 203 is effective. If the adjustment unit 206 determines that the likelihood threshold setting is effective, the process proceeds to S404. On the other hand, if the adjustment unit 206 determines that the likelihood threshold setting is ineffective, the process proceeds to S410.

[0037] When proceeding to S403, the adjustment unit 206 acquires the information of the estimation result obtained by the estimation unit 204, that is, the likelihood information representing the distribution of likelihoods indicating the probability of the presence of the specific object. Here, the adjustment unit 206 acquires, as the likelihood information, for example, the position information in the image and the likelihood values corresponding to the position information. Next, in S404, the adjustment unit 206 generates a partial region for threshold adjustment (second partial region) based on the likelihood information obtained from the estimation unit 204 in S403. Details of the process for creating the partial region for threshold adjustment based on the likelihood information will be described later.

[0038] Next, the adjustment unit 206 repeats the loop process from S405 to S408 for each of all the partial regions for threshold adjustment created in S404. In S405 to S408, it is determined whether there are a certain number or more of likelihoods equal to or greater than a predetermined value within the partial region for threshold adjustment. When there are a certain number or more, a process is performed to determine the region to be adjusted so as to lower the threshold value of the partial region or not set the threshold value.

[0039] That is, in the loop process from S405 to S408, in S406, the adjustment unit 206 determines whether there are a certain number or more of high likelihoods equal to or greater than a predetermined value within the partial region for threshold adjustment. When there are a certain number or more of high likelihoods within the partial region for threshold adjustment, the adjustment unit 206 determines that the likelihood of the object being the object to be counted is high, that is, the likelihood of a correct detection exists, and proceeds with the process to S407. On the other hand, when there are not a certain number or more of likelihoods equal to or greater than a predetermined value within the partial region for threshold adjustment, the adjustment unit 206 determines that the likelihood of the object to be counted existing is low, that is, the likelihood of a false detection exists, and proceeds with the process to S408.

[0040] When proceeding to S407, the adjustment unit 206 adjusts the likelihood threshold value set by the setting unit 203 to be lower for the partial region for threshold adjustment. When proceeding to S407, the adjustment unit 206 may also adjust the likelihood threshold value to 0 for the partial region for threshold adjustment, that is, adjust it so as not to set the threshold value. Then, when the loop process from S405 to S408 is performed for each of all the partial regions for threshold adjustment created in S404, the adjustment unit 206 proceeds with the process to S409.

[0041] When proceeding to S409, the adjustment unit 206 updates so that the likelihood less than the threshold value adjusted for each partial region for threshold adjustment becomes 0. As a result, the likelihood less than the threshold value becomes out of the count target. After this S409, the process of the information processing apparatus 100 proceeds to S410 by the estimation unit 204.

[0042] When proceeding to S410, the estimation unit 204 estimates the number of specific objects based on the likelihood after being updated in S409. That is, in the estimation unit 204, by counting the likelihood after being updated in S409, a process of calculating the number of specific objects (estimated number of persons) is performed.

[0043] Next, with reference to FIGS. 5(A) to 5(C), the creation process of the partial region for threshold adjustment performed in the adjustment unit 206 of the present embodiment will be described. FIG. 5(A) is a diagram showing an example in which a partial region input to the regressor during the estimation process of the estimation unit 204 is created as a partial region for threshold adjustment. In the image 500A of FIG. 5(A), the partial region for threshold adjustment indicated by the hatched portion is a region where the likelihood threshold value is not lowered, and the partial region that is not the hatched portion is a region where the likelihood threshold value is adjusted to be low.

[0044] FIG. 5(B) is a diagram showing an example in which groups are created for each partial region where a plurality of likelihoods equal to or greater than a predetermined value are densely distributed, and a partial region for threshold adjustment is created in units of groups. The adjustment unit 206 determines the groups based on the distance between the likelihoods that are high and equal to or greater than the predetermined value. In the image 500B of FIG. 5(B), the partial region for threshold adjustment indicated by the hatched portion is a region where the threshold value is not lowered, and the region that is not the hatched portion is a region that is grouped such that the likelihood threshold value is adjusted to be low.

[0045] FIG. 5(C) is a diagram showing an example in which a partial region for threshold adjustment is created in units of blocks including likelihoods equal to or greater than a predetermined value. In the image 500C of FIG. 5(C), the partial region for threshold adjustment indicated by the hatched portion is a region where the threshold value is not lowered, and the region that is not the hatched portion is a region created in units of blocks. The block unit may be a set of likelihoods where pixels are continuous, or a set of likelihoods existing in pixels within a certain range centered on a likelihood that is high and equal to or greater than a predetermined value.

[0046] Note that the partial region for threshold adjustment may target the entire screen. Also, in the case of a video, if the estimation result changes between frames, the adjustment unit 206 may recreate the partial region for threshold adjustment by executing the creation of the partial region for threshold adjustment again. Also, the adjustment unit 206 may use the threshold location of the partial region for threshold adjustment where there is no likelihood greater than or equal to a predetermined value as a region to be set high.

[0047] As described above, in the first embodiment, based on the likelihood information estimated from the input image, the threshold of the partial region where there is a high possibility of positive detection is adjusted downward. As a result, it is possible to suppress the decrease in the likelihood of positive detection while reducing the estimated value of low likelihood that is likely to be a false detection. Therefore, according to this embodiment, it is possible to improve the counting accuracy of the estimated specific object.

[0048] An example of specific values and the like will be described. For example, assume that the specific object is the head of a human body, the predetermined value used to determine that the likelihood is high is 0.02, there are many likelihoods of 0.02 or more that constitute the head of a positive detection, while on the other hand, assume an analysis result where false detections, which are aggregates of low likelihoods, are very few. In the case of this example, if there are a certain number or more (for example, 100 or more) of likelihoods of 0.02 or more in the image, the likelihood threshold is set to a low value (for example, 0.010), while if it is less than 100, for example, the likelihood threshold is set to a value higher than 0.01 (for example, 0.015). Thereby, it is possible to prevent the likelihood of positive detection from being reduced while reducing the estimated value of low likelihood that is likely to be a false detection.

[0049] As another example, assume an analysis result in which the likelihood constituting the positive detection includes a high likelihood of, for example, 0.03 or more and less than 0.10, and hardly includes a likelihood exceeding 0.03 as the likelihood of false detection. In the case of this example, if there are a certain number or more (for example, 100 or more) of likelihoods of 0.03 or more in the image, the likelihood threshold is set to, for example, 0 (the same as no threshold), while if there are, for example, 10 to 100, the likelihood threshold is set to a low value (for example, 0.010). This makes it possible to suppress the decrease in the likelihood of positive detection while reducing the estimated value of the likelihood with a high possibility of false detection.

[0050] <Second Embodiment> Next, as a second embodiment, an example of a threshold adjustment process for determining a threshold for each partial region for threshold adjustment according to the number of high likelihoods equal to or greater than a predetermined value will be described. In the first embodiment described above, an example in which the threshold is lowered or the threshold is eliminated depending on whether or not there are a certain number or more of likelihoods equal to or greater than a predetermined value was described. In contrast, in the second embodiment, a threshold adjustment process for stepwise adjusting the likelihood threshold for each partial region for threshold adjustment according to the number of high likelihoods equal to or greater than a predetermined value will be described. Note that the system configuration and the functional blocks of the information processing apparatus 100 in the second embodiment are the same as those described with reference to FIGS. 1 and 2 in the first embodiment, and thus the description thereof will be omitted.

[0051] FIG. 6 is a diagram showing an example of a histogram (frequency distribution table) of likelihood values held by each pixel corresponding to a positive detection for one person when the specific object is, for example, a person. Here, the likelihood value 0 is not included in the likelihood value held by each pixel corresponding to the positive detection. Therefore, the likelihood value 0 is not included in the number of likelihoods in FIG. 6. For example, when the predetermined value for determining that the likelihood is high is 0.02 or more, in the example of FIG. 6, 17 likelihood values are targeted. In the second embodiment, the threshold for the partial region for threshold adjustment of interest is determined according to the number of such high likelihoods.

[0052] FIG. 7 is a diagram showing, as Table 700, an example of determining a threshold value for each partial region for threshold adjustment according to the number of high likelihoods equal to or greater than a predetermined value. From the example of FIG. 7, when the number of likelihoods equal to or greater than the predetermined value included in the partial region for threshold adjustment is 0, the adjustment unit 206 considers that there is a high possibility of no positive detection, and performs threshold adjustment to set the highest likelihood threshold value. Further, when the number of high likelihoods equal to or greater than the predetermined value is 1 to 100, the adjustment unit 206 sets the threshold value to 0.01, when the number of high likelihoods is 101 to 200, the adjustment unit 206 sets the threshold value to 0.005, and when the number is 201 or more, the adjustment unit 206 performs threshold adjustment to set no threshold value (threshold value 0).

[0053] FIG. 8 is a diagram showing an example of the result of performing threshold adjustment processing for stepwise determining the likelihood threshold value for each partial region for threshold adjustment according to the number of high likelihoods equal to or greater than the predetermined value described in FIG. 7. Based on the table of FIG. 7, in the image 800 of FIG. 8, the threshold value is 0.02 in the region indicated by the hatched portion, whereas the region with a threshold value of 0.01 indicates a region where the number of likelihoods equal to or greater than the predetermined value is 1 to 100. Also, in the image 800 of FIG. 8, the region with a threshold value of 0.005 indicates a region where the number of likelihoods equal to or greater than the predetermined value is 101 to 200, and the region with no threshold value indicates a region where the number of likelihoods equal to or greater than the predetermined value is 200 or more.

[0054] Note that the processing according to the second embodiment may be executed, for example, when the user enables the likelihood threshold value setting and selects automatic adjustment. Also, the automatic adjustment of the likelihood threshold value setting may be executed as internal processing as processing for improving detection accuracy. In the case of a video, when the estimation result changes between frames, the partial region for threshold adjustment may be automatically recreated.

[0055] In the case of the second embodiment, based on the likelihood information in the image, the threshold value is gradually adjusted downward according to the number of positive detections in the partial region where there is a high possibility of the likelihood of positive detection. As a result, the larger the number of high likelihoods equal to or greater than a predetermined value, that is, the larger the number of positive detections, the larger the cumulative value of the low likelihoods of the positive detections. According to the second embodiment, by determining the likelihood threshold value stepwise in this way, it is possible to prevent the decrease amount of the likelihood of positive detection from increasing as the number of positive detections increases. Thereby, in the second embodiment, a locally suitable likelihood threshold value can be set, and it is possible to improve the counting accuracy of the estimated specific object.

[0056] <Third Embodiment> Next, as a third embodiment, an example will be described in which, when a user designates a partial region for adjusting the likelihood threshold value using a GUI for setting the likelihood threshold value and the like, the likelihood threshold value is set for each partial region for adjusting the threshold value. Note that the system configuration and the functional block diagram of the information processing apparatus 100 in the third embodiment are the same as those described with reference to FIGS. 1 and 2 of the first embodiment, and thus the description thereof will be omitted.

[0057] FIG. 9 is a diagram showing an example of a GUI screen that the setting unit 203 causes to be displayed on the display 130 or the like via the output control unit 201 in the third embodiment. The adjustment unit 206 sets the likelihood threshold value in the partial region for adjusting the threshold value based on the information set by the user via this GUI screen.

[0058] The display screen 900 in FIG. 9 is a diagram showing an example of a GUI for setting a likelihood threshold for the purpose of removing false detections with low likelihood. In the initial state, and when the user selects the checkbox 901 on the display screen 900 and further presses the OK button 911, the adjustment unit 206 does not perform the likelihood threshold setting. Also, when the user selects the checkbox 902 on the display screen 900 and further presses the OK button 911, the adjustment unit 206 executes the likelihood threshold setting. Note that when the user selects the checkbox 902, it is assumed that the checkbox 903 is selected in the initial state. Also, when the user selects the checkbox 903 and further presses the OK button 911, the setting unit 203 automatically determines and sets the likelihood threshold assuming that the user has instructed automatic determination. On the other hand, when the user selects the checkbox 904 and further presses the OK button 911, the setting unit 203 enables the selection of the radio button 905 or the radio button 906. And when the user selects the radio button 905, the adjustment unit 206 enables the specification of the likelihood threshold for the entire screen. As a result, it becomes possible to exclude likelihoods less than an arbitrary threshold from the counting target for the entire screen. Also, when the user selects the radio button 905, the setting unit 203 enables the pressing of the area setting button 913. And when the area has not been set and the user presses the area setting button 913 and sets the area, the adjustment unit 206 can set the likelihood threshold for all the set areas. Note that the display areas 907 to 910 show the thresholds (for example, 0.005 to 0.01) set for areas 1 to 4, respectively. And when the user presses the OK button 911, the setting is completed. On the other hand, when the cancel button is pressed, the adjustment unit 206 does not reflect the set content.

[0059] As described above, in the case of the third embodiment, the user can specify an area and set a likelihood threshold. Also, in this embodiment, the user can specify a likelihood threshold suitable for each situation, such as an area where false detections are likely to occur or an area where a significant decrease in the estimated number of positive detections due to threshold setting is expected. According to the third embodiment, the user can specify to set a high threshold for an area where false detections are likely to occur or to set a low threshold to suppress the decrease in the likelihood of positive detections in an area where specific objects are concentrated, and as a result, it is possible to improve the counting accuracy.

[0060] <Hardware Configuration> FIG. 10 is a diagram showing an example of a hardware configuration capable of realizing each function of the information processing apparatus 100 according to the first to third embodiments described above. Here, the hardware configuration of the information processing apparatus 100 will be described, but the recording apparatus 120 and the imaging apparatus 110 may also be realized by a similar hardware configuration.

[0061] The information processing apparatus 100 in the present embodiment includes at least a CPU 1000, a RAM 1010, a ROM 1020, an HDD 1030, and an I / F 1040. The CPU 1000 is a central processing unit that controls the information processing apparatus 100 overall. The RAM 1010 temporarily stores computer programs executed by the CPU 1000. Also, the RAM 1010 provides a work area used when the CPU 1000 executes processing. Further, the RAM 1010 functions as, for example, a frame memory or a buffer memory. The ROM 1020 stores programs for the CPU 1000 to control the information processing apparatus 100 and the like. The HDD 1030 is a storage device that records image data and the like. The information processing program according to the present embodiment is stored in the ROM 1020 or the HDD 1030, expanded in the RAM 1010, and executed by the CPU 1000, whereby each process in each functional unit shown in FIG. 2 described above is realized. The I / F 1040 communicates with an external device via the network 140 in accordance with TCP / IP, HTTP, or the like.

[0062] In the description of the above-described embodiments, an example in which the CPU 1000 executes processing has been described. However, at least a part or all of the processing of the CPU 1000 may be performed by dedicated hardware. For example, the processing of displaying a GUI or image data on the display 130 may be executed by a GPU (GRAPHICS PROCESSING UNIT). Further, the processing of reading the program code from the ROM 1020 and expanding it in the RAM 1010 may be executed by a DMA (DIRECT MEMORY ACCESS) that functions as a transfer device. Further, each part of the information processing apparatus 100 may be realized by the hardware shown in FIG. 10, or may be realized by software.

[0063] <Other Embodiments> Although each of the above embodiments has been described, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist thereof. One or more functions of the information processing apparatus 100 according to each of the above-described embodiments may be provided in another apparatus. For example, one or more functions of the information processing apparatus 100 according to each of the embodiments may be provided in the imaging apparatus 110. Note that each of the above-described embodiments may be arbitrarily combined and implemented. The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and causing one or more processors in a computer of the system or apparatus to read and execute the program. Further, it can also be realized by a circuit (for example, ASIC) that realizes one or more functions. Each of the above-described embodiments is merely an example of implementation of the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from its technical idea or its main features.

Description of Reference Numerals

[0064] 100: Information processing apparatus, 110: Imaging apparatus, 120: Recording apparatus, 130: Display, 200: Communication unit, 201: Output control unit, 202: Operation reception unit, 203: Setting unit, 204: Estimation unit, 205: Recording unit, 206: Adjustment unit

Claims

1. An acquisition means for acquiring likelihood information representing a distribution of likelihoods, including the likelihood indicating the probability of the presence of a specific object and the position of the specific object, from an input image; A determination means for determining, as an area to be adjusted, an area where, when it is determined based on the likelihood information that the likelihood included in a partial area of the input image is equal to or greater than a predetermined value, the threshold value for the partial area is adjusted to be lower; An estimation means for estimating the number of the specific objects by excluding, from objects to be counted, likelihoods less than the adjusted threshold value among the likelihoods included in the partial area determined by the determination means, and counting the likelihoods for each partial area; An information processing apparatus, characterized by comprising the above.

2. The information processing apparatus according to claim 1, wherein the determination means determines, as an area to be adjusted, an area where the threshold value for the partial area is adjusted to be lower when the number of likelihoods greater than a second predetermined value among the likelihoods included in the partial area is equal to or greater than a certain number.

3. Further comprising a generation means for generating a plurality of partial areas from the input image; The information processing apparatus according to claim 1 or 2, wherein the determination means determines whether to adjust the threshold value for the generated partial areas.

4. The acquisition means acquires the likelihood information as a result of inputting the input image into a learned model for estimating the position of the specific object; The information processing apparatus according to claim 3, wherein the learned model outputs the likelihood information for each of the plurality of partial areas generated by the generation means.

5. Further comprising a setting means for setting a predetermined threshold value for each of the plurality of generated partial areas; The information processing apparatus according to claim 3 or 4, wherein the determination means determines, as an area to be adjusted, an area where, when there are a certain number or more of likelihoods equal to or greater than a predetermined value within the partial area, the partial area is adjusted to a threshold value lower than the threshold value set by the setting means.

6. The generation means generates a region of interest from the input image based on the likelihood information, The determination means determines a region for adjusting the threshold value based on the likelihood information included in the region of interest, and the information processing apparatus according to any one of claims 3 to 5.

7. The generation means generates the region of interest as a partial region in which the likelihood is greater than a predetermined number based on the likelihood information, The determination means determines the region of interest as a region where the threshold value is adjusted lower, and the information processing apparatus according to claim 6.

8. The generation means generates a region included in a predetermined range from a likelihood equal to or greater than a predetermined value included in the partial region as a region of interest based on the likelihood information, The determination means determines the region of interest as a region where the threshold value is adjusted lower, and the information processing apparatus according to claim 6.

9. The generation means updates the partial region if the likelihood information changes between frames of the video, and the information processing apparatus according to any one of claims 3 to 8.

10. The generation means generates the partial region according to an instruction from the user, and the information processing apparatus according to any one of claims 3 to 8.

11. The determination means determines the adjusted threshold value according to the number of likelihoods equal to or greater than a predetermined value included in each partial region, and the information processing apparatus according to any one of claims 1 to 8.

12. The determination means adjusts the threshold value of the partial region in which there is no likelihood equal to or greater than the predetermined value to be higher than the threshold value of the partial region in which there is a likelihood equal to or greater than the predetermined value, and the information processing apparatus according to any one of claims 1 to 9.

13. The information processing apparatus according to claim 12, wherein the determination means sets the threshold value for each of the partial regions in accordance with an instruction for automatic determination from a user or an instruction for a threshold value from the user.

14. An information processing method executed by an information processing apparatus, an acquisition step of acquiring likelihood information representing a distribution of likelihoods, including likelihoods indicating probabilities that a specific object exists and positions of the specific object, from an input image; a determination step of determining, as a region where a threshold value for a partial region included in the input image is adjusted to be lower when it is determined that the likelihood included in the partial region is equal to or greater than a predetermined value based on the likelihood information; an estimation step of estimating the number of the specific objects by excluding, from objects to be counted, likelihoods less than the adjusted threshold value among the likelihoods included in the partial region determined in the determination step and counting the likelihoods for each of the partial regions; characterized by including the above.

15. A program for causing a computer to function as the information processing apparatus according to any one of claims 1 to 13.

Images