Image processing apparatus, image processing method, and program

Abstract

To achieve analysis of an area in an image in a more suitable mode.SOLUTION: A condition setting unit 202 performs settings related to the size of an object to be detected included in an image to a predetermined position of the image. An area setting unit 203 divides the image into a plurality of partial areas based on the settings performed by the condition setting unit 202. An area selection unit 204 selects part of the series of partial areas based on the settings performed by the condition setting unit 202. An analysis unit 205 executes predetermined analysis on the selected part of the areas.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates to a technique for detecting a specific object from an image.

Background Art

[0002] In recent years, a system has been proposed that captures a predetermined area with an imaging device and analyzes the captured image to measure the number of people in the image. Such a system is expected to be used for congestion resolution during events and evacuation guidance during disasters by detecting congestion in public spaces and grasping the flow of people during congestion. As a method for measuring the number of people in such an image, a method (Non-Patent Document 1) has been proposed that estimates the number of people reflected in a predetermined area of the image using a recognition model obtained by machine learning. Hereinafter, this method is also referred to as the "regression-based number estimation method". In addition, the area created by the regression-based number estimation method is also referred to as the "regression area". An example of a method for counting the number of people using this measurement method is disclosed in Patent Document 1.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Regression-based population estimation methods can estimate the number of people even when they are present in high density or appear small in the image. However, to perform more accurate analysis, it is sometimes desirable to set a size for the human body and then define the regression region according to the size of the human body captured as a subject in the image. In contrast, if feedback of the analysis results of the regression region is provided to verify the accuracy of the setting of the human body size, the load on the analysis of that regression region increases, and as a result, the execution of that analysis may take longer.

[0006] In view of the above-mentioned problems, the present invention aims to enable the analysis of regions in an image in a more suitable manner. [Means for solving the problem]

[0007] The image processing apparatus according to the present invention comprises: setting means for setting the size and position of at least a portion of a human body captured as a subject in an image; dividing means for setting regression regions by dividing the image into a plurality of regions based on the settings by the setting means; selection means for selecting a portion of the set regression regions based on the settings by the setting means; and analysis means for performing a predetermined analysis on the selected portion of the region. [Effects of the Invention]

[0008] According to the present invention, it becomes possible to perform analysis of regions in an image in a more preferred manner. [Brief explanation of the drawing]

[0009] [Figure 1] This diagram shows an example of the hardware configuration of an information processing device. [Figure 2] This is a functional block diagram showing an example of the functional configuration of an image processing device. [Figure 3] This is a flowchart illustrating an example of processing performed by an image processing device. [Figure 4] This figure illustrates an example of feedback based on analysis results. [Figure 5] This diagram shows an example where multiple settings exist. [Modes for carrying out the invention]

[0010] Preferred embodiments of this disclosure will be described in detail below with reference to the attached drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0011] <Hardware Configuration> Referring to Figure 1, an example of the hardware configuration of an information processing device 100 applicable as an image processing device according to one embodiment of the present disclosure will be described. The information processing device 100 according to this embodiment includes a CPU (Central Processing Unit) 101, RAM (Random Access Memory) 103, and ROM (Read Only Memory) 102. The information processing device 100 also includes an auxiliary storage device 104 and a network interface 107. The information processing device 100 may also include at least one of an input device 105 and an output device 106. The CPU 101, RAM 102, ROM 103, auxiliary storage device 104, input device 105, output device 106, and network interface 107 are each interconnected via a bus 108.

[0012] The CPU 101 is a central processing unit that controls various operations of the information processing device 100. For example, the CPU 101 may control the operation of the entire information processing device 100. RAM102 is the main memory of the CPU101 and is used as a work area or temporary storage area for deploying various programs. ROM103 stores programs (such as BIOS) that the CPU101 uses to control the operation of the information processing unit 100.

[0013] The auxiliary storage device 104 stores programs such as the OS (Operating System), which is basic software, and various applications, as well as various data. The auxiliary storage device 104 can be realized by, for example, a non-volatile memory typified by an HDD (Hard Disk Drive) or an SDD (Solid State Drive).

[0014] The network I / F 107 is an interface for connecting to a predetermined network (e.g., LAN, Internet, etc.) and communicating with external devices via the network.

[0015] The input device 105 is a device for receiving instructions from the user. The input device 105 can be realized by, for example, a pointing device such as a mouse, an operation device such as a keyboard, or a touch panel. The output device 106 is a device for presenting various information to the user. The output device 106 can be realized by, for example, a display device that presents information to the user by displaying various display information, screens, etc., such as a display.

[0016] Note that the above-described configuration is merely an example and does not necessarily limit the hardware configuration of the information processing device 100. As a specific example, some of the components (e.g., at least one of the input device 105 and the output device 106) of the series of components of the information processing device 100 described above may be externally attached to the information processing device 100. Also, the functional configuration described later with reference to FIG. 2 and the processing described later with reference to FIG. 3 are realized by the desired information processing device 100 expanding and executing the programs stored in the ROM 103 and the auxiliary storage device 104 in the RAM 102.

[0017] <Functional Configuration> Referring to FIG. 2, an example of the functional configuration of the image processing apparatus according to the present embodiment will be described. The image processing apparatus 200 according to the present embodiment includes an image acquisition unit 201, a condition setting unit 202, a region setting unit 203, a region selection unit 204, an analysis unit 205, and an output control unit 206.

[0018] The image acquisition unit 201 acquires image data to be analyzed. As a specific example, the image acquisition unit 201 may acquire image data corresponding to an imaging result by an individual imaging device such as a CMOS sensor or a CCD sensor. As another example, the image acquisition unit 201 may acquire image data stored in a predetermined storage area such as the auxiliary storage device 104, the ROM 103, and the RAM 102 shown in FIG. 1.

[0019] The condition setting unit 202 performs settings related to the analysis by the analysis unit 205 described later on the image corresponding to the image data acquired by the image acquisition unit 201.

[0020] As a specific example, the condition setting unit 202 may perform settings related to the estimation of the number of human bodies imaged as subjects in the image (hereinafter also referred to as "number of persons"), such as the size of the human body (the range of the size of the object to be detected as the detection target) and the position of the human body (the range to detect the detection target in the image). In this case, for example, settings related to the size of the human body and the position of the human body may be performed according to an instruction from the user. Specifically, the condition setting unit 202 presents the target image to the user via a predetermined output device, and accepts the designation of the size of the human body at a desired position in the image from the user via a predetermined input device, thereby performing settings related to the size of the human body and the position of the human body. Note that since the size of the human body to be detected appears differently depending on the position in the image, the detection accuracy can be improved by making the detection size different according to the position in the image. In this case, the condition setting unit 202 may accept the specification of the size of the human body at each of the multiple locations in the image. This makes it possible for the condition setting unit 202 to estimate the average size of the human body at any location in the image by interpolation based on the information specified by the user.

[0021] As another example, the condition setting unit 202 may use various methods such as known pattern recognition and machine learning to detect human bodies in the image, and based on the results of the detection, it may acquire information about a set of human body frames indicating the location of human bodies. The detection of human bodies in the image can be achieved by identifying the location of pre-set parts, such as the entire human body or a part of the human body (e.g., the face). As a concrete example, let's assume that s is the size of the human body frame at coordinates (x,y) in an image, and that s is represented by x, y, and one or more unknown parameters. More specifically, let's define the unknown parameters as a, b, and c, and assume that s = ax + by + c holds true. Under these assumptions, by using a set of human body frames obtained from a predetermined set of training images, it is possible to estimate the size of the human body frame by applying statistical processing such as the least squares method.

[0022] The methods exemplified above may be used to set the size and position of a human body captured as a subject in an image. In this case, for example, the estimated size of the human body using the method may be used as the initial value to set the size and position of the human body. Furthermore, the parameters may be dynamically adjusted according to the time of day and the circumstances at the time. Of course, the methods described above are merely examples, and the method is not particularly limited as long as it is possible to set the size and position of the human body captured as a subject in the image.

[0023] The region setting unit 203 sets at least a portion of the target image as the regression region. As a specific example, the region setting unit 203 may divide the target image into multiple regions (in other words, partial regions) and set at least a portion of these multiple regions as the regression region. In this case, the region setting unit 203 may set the regression region according to the settings made by the condition setting unit 202. As a specific example, the region setting unit 203 may set the regression region such that the ratio between the size of the regression region and the size of the human body approximately matches a predetermined ratio, according to the settings made by the condition setting unit 202 regarding the size and position of the human body. For convenience, the region setting unit 203 will be assumed to set the regression region by dividing the image into multiple regions using the lower left edge of the image as the reference position, but this does not necessarily limit the method of setting the regression region. As a specific example, the regression region may be set by dividing the image using a predetermined position other than the lower left edge of the image as the reference position. As another example, the regression region may be set based on the position of the human body according to the size and position of the human body set by the condition setting unit 202. As yet another example, the regression region may be set not limited to the size and position of the human body, but by dividing the target image into multiple regions, each having a predetermined size.

[0024] The region selection unit 204 selects some of the regression regions from a series of regression regions set by the region setting unit 203. In this case, the region selection unit 204 may select regression regions according to the settings made by the condition setting unit 202. As a specific example, the region selection unit 204 may select regression regions that include the human body from the series of regression regions according to the human body position set by the condition setting unit 202. In other words, the region selection unit 204 may select regression regions that include the position where the human body size has been set by the condition setting unit 202. Alternatively, the region selection unit 204 may select regression regions located near the human body position.

[0025] The analysis unit 205 performs a predetermined analysis on the regression region selected by the region selection unit 204. For example, the analysis unit 205 may estimate the number of people in the selected regression region by using a known regression-based population estimation method.

[0026] Here, as a more specific example, we will describe a method using a regressor that takes a small image of a predetermined size as input and outputs the number of human bodies captured in that small image. In this case, for example, a regressor is constructed based on known machine learning methods such as support vector machines or deep learning, using multiple small images with a known number of people as training data. Then, when estimating the number of people, the region setting unit 203 sets the regression regions by dividing the target image into multiple regions for the unit in which the estimation is performed. The analysis unit 205 then resizes the size of each of the set regression regions as needed and uses them as input to the regressor to estimate the number of people in each regression region. An example of regression-based number estimation is disclosed in Non-Patent Literature 1.

[0027] Furthermore, the estimated number of people may not necessarily be an integer; it may be a real number. In this case, for example, the estimated real number may be rounded to an integer based on predetermined conditions such as rounding, and then treated as the number of people, or the estimated real number may be treated as the number of people as is. Furthermore, by utilizing the settings for the size and position of the human body by the condition setting unit 202, regression regions of different sizes may be set within the image.

[0028] The output control unit 206 outputs information corresponding to the results of the analysis by the analysis unit 205 to a predetermined output destination. For example, the output control unit 206 may target images acquired by the image acquisition unit 201 and output information corresponding to the number of people estimated by the analysis unit 205 to a predetermined output destination. In this case, for example, the output control unit 206 may output the number of people estimated in the regression region set in the image by the region setting unit 203 to a predetermined output destination. As another example, the output control unit 206 may output the number of people estimated for each set regression region so that the number of people estimated for each position in the image can be identified.

[0029] As the output destination, for example, a display unit 250 such as a display that presents the target information to the user by displaying it as image or other display information may be applied. Alternatively, as another example, another device that presents various information to the user via a desired output unit such as the display unit 250 may be applied as the output destination. In this case, the output control unit 206 may output data to the other device so that the other device can present information to the user via a desired output unit according to the results of the analysis by the analysis unit 205.

[0030] It should be noted that the above-described configuration is merely an example, and the functional configuration of the image processing apparatus 200 according to this embodiment is not necessarily limited to the example shown in Figure 2, as long as it is possible to realize the processing of each of the above-described components. For example, the processing of each component shown in Figure 2 may be realized through the cooperation of multiple devices. As a specific example, some of the components of the image processing device 200 shown in Figure 2 may be provided in other devices different from the image processing device 200. Also, the load related to the execution of processing of at least some of the components of the image processing device 200 shown in Figure 2 may be distributed among multiple devices.

[0031] <Processing> Referring to Figure 3, an example of the processing performed by the image processing apparatus according to this embodiment will be described.

[0032] In S301, the image acquisition unit 201 acquires image data to be analyzed. In S302, the condition setting unit 202 performs settings related to the analysis of the image corresponding to the image data acquired by the image acquisition unit 201. Here, the condition setting unit 202 performs settings related to the size of the human body and the position of the human body, which are to be used as the target for estimating the number of people in the image.

[0033] In S303, the region setting unit 203 divides the target image into multiple regions based on the settings for the size and position of the human body in S302, and sets at least some of these regions as regression regions. In S304, the region selection unit 204 selects some of the series of regression regions set in S303 as the target of analysis, based on the settings regarding the size and position of the human body in S302. In S305, the analysis unit 205 performs a predetermined analysis on the regression region selected in S304. Here, the analysis unit 205 estimates the number of people in the selected regression region by using a known regression-based population estimation method.

[0034] In S306, the output control unit 206 outputs information corresponding to the analysis results (estimated number of people) in S305 to a predetermined output destination. Here, the output control unit 206 displays information corresponding to the estimated number of people in the target regression region on the display unit 250. Furthermore, there are no particular limitations on the manner in which the information corresponding to the estimation results is displayed. As a specific example, the output control unit 206 may display the results of the person estimation as a numerical value. As another example, the output control unit 206 may display information such as marks or icons corresponding to the number of people detected, indicating whether or not people were detected from the target regression area.

[0035] Furthermore, the output control unit 206 may store the information corresponding to the acquired analysis results in a predetermined memory area. By applying such control, the output control unit 206 can compare the newly acquired information corresponding to the analysis results with information corresponding to previously acquired analysis results and output information corresponding to the results of that comparison. Note that the information output as the current analysis result may be limited to information corresponding to the partial analysis results targeting the regression region selected in S304.

[0036] As described above, by outputting information corresponding to the results of the analysis, the user can check this information and give instructions to the image processing device 200 regarding the size and position of the human body in the image (for example, instructions to change the settings).

[0037] In S307, the image processing device 200 determines whether or not the settings regarding the size of the human body and the position of the human body have been changed. If the image processing device 200 determines in S307 that the settings related to the size or position of the human body have been changed, it proceeds to S303. In this case, the processes from S303 to S306 are executed again based on the changed settings. Furthermore, if the image processing device 200 determines in S307 that the settings regarding the size of the human body and the position of the human body have not been changed, it proceeds to S308.

[0038] In S308, the image processing device 200 checks whether there are any unanalyzed regression regions (in other words, regression regions that were not the subject of analysis in S305) among the series of regression regions set in S303. If the image processing device 200 determines in S308 that there are unanalyzed regression regions, it proceeds to S305. In this case, in S305, the unanalyzed regression regions are selected again, and analysis is performed on those regression regions in S306. Then, if the image processing device 200 determines in S308 that there are no unanalyzed regression regions, it terminates the series of processes shown in Figure 3.

[0039] It should be noted that the processing described above is merely an example and does not necessarily limit the processing of the image processing apparatus according to this embodiment. For example, the above shows an example in which a series of regression regions are set and then analysis processing is performed on those regression regions, but the order in which each process is executed is not particularly limited as long as analysis processing is performed on the desired regression regions. As a specific example, after setting and analyzing some regression regions, the process of setting and analyzing other regression regions may be executed sequentially.

[0040] <Examples> Next, with reference to Figures 4 and 5, a specific example of the image processing apparatus according to this embodiment will be described.

[0041] First, let's explain the example shown in Figure 4. Figure 4 is a diagram illustrating an example of feedback based on analysis results. It shows an example of feedback based on the estimated number of people, given the specified size and position of the human body.

[0042] First, let's explain Figure 4(a). Image 400 schematically shows an image corresponding to the imaging result from a desired imaging device. Multiple human bodies 401 are imaged as subjects in Image 400. Marker 402 schematically shows information corresponding to the setting of the size and position of the human bodies 401 based on user instructions. In other words, in the example shown in Figure 4(a), the size and position of the human bodies 401 in Image 400 are set according to the position and size of the marker 402.

[0043] Next, Figure 4(b) will be explained. Figure 4(b) schematically shows the result of setting the regression region for image 400. Specifically, in the example shown in Figure 4(b), image 400 is divided into multiple regions based on the size and position of the human body 401 as specified by marker 402 in Figure 4(a), and these regions are set as regression regions 403. At this time, the regression region 403 is set so that the ratio between the size of the regression region 403 and the size of the human body 401 approximately matches a predetermined ratio. Furthermore, in the example shown in Figure 4(b), some of the set regression regions 403 have been selected, and analysis has been performed on these selected regression regions. For example, regression region 405 represents the regression region selected from the set of regression regions 403 according to the size and position of the human body 401. Display information 404 schematically shows the information displayed according to the results of the analysis on regression region 405. In this way, by providing feedback on the displayed information 404, the user can check the displayed information 404 and, if necessary, give instructions regarding the size and position of the human body 401 in the image 400.

[0044] Next, Figure 4(c) will be explained. Figure 4(c) schematically shows the result when the size and position of the human body 401 are not changed after the feedback exemplified in Figure 4(b), and the selection and addition of regression regions are performed sequentially. For example, region 406 schematically shows the regression region that is set as a result of the sequential selection and addition of regression regions (in other words, the regression region set by the sum of a series of sequentially selected regression regions).

[0045] As described above, by partially selecting and analyzing a portion of the region divided from the image based on the size and position of the human body captured as a subject in the image, it is possible to suppress the increase in the load associated with the analysis. In other words, with the image processing device according to this embodiment, it is possible to more quickly provide feedback on the analysis results of the regression region, which allows for verification of the accuracy of the setting of the human body size.

[0046] Next, we will explain the example shown in Figure 5. Figure 5 shows an example where there are multiple settings regarding the size and position of the human body. For example, markers 500 and 501 schematically represent information corresponding to the settings for the size and position of the human body, respectively.

[0047] As shown in the example in Figure 5, if there are multiple settings related to the size and position of the human body, the region setting unit 203 sets the regression region by determining unknown parameters based on the multiple settings. Furthermore, the region selection unit 204 selects some regression regions from a set of regression regions based on the multiple settings related to the size and position of the human body described above. For example, in the example shown in Figure 5, the region selection unit 204 selects regression region 502 according to the setting corresponding to marker 500 and regression region 503 according to the setting corresponding to marker 501. Then, the analysis unit 205 performs analysis on the selected regression regions 502 and 503.

[0048] Furthermore, 504 schematically represents a region in the target image for which no regression region was set (in other words, a region excluded from the analysis). For example, in the example shown in Figure 5, region 504 is presented with information indicating that it is excluded from the analysis, such as displaying it in a different manner from other regions. Hereafter, for convenience, this region labeled 504 will also be referred to as the "region excluded from analysis." Such areas 504 that are not included in the analysis may become apparent, for example, when a regression region is set by determining unknown parameters based on multiple settings related to the size and position of the human body, as these are areas where a regression region was not set.

[0049] In light of this situation, in the example shown in Figure 5, the output control unit 206 presents the area 504 that is not subject to analysis in such a way that it can be distinguished from other areas (for example, a set regression area). As a specific example, the output control unit 206 may present the area 504 that is not subject to analysis in a way that makes it distinguishable from other areas by displaying information (for example, a marker, etc.) indicating that it is an area not subject to analysis. As another example, the output control unit 206 may present the area 504 that is not subject to analysis in a way that makes it distinguishable from other areas by controlling the display manner (for example, color, etc.) of the area 504 that is not subject to analysis. Through such feedback, the user can recognize areas in the image that are excluded from the analysis.

[0050] <Other Embodiments> The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or recording medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0051] Although preferred embodiments of the present invention have been described in detail above, the present invention is not limited to these specific embodiments. In the embodiments described above, humans were used as an example of the object of estimation. On the other hand, by replacing the part described as "humans" with any object such as animals other than humans, vehicles such as automobiles and bicycles, and microorganisms, the configuration and processing of the embodiments described above can be used to estimate the number of any specific object. As described above, according to the embodiment described, it becomes possible to quickly check the results of the analysis by partially selecting a region from the regression region of the regression-based population estimation method based on settings for human body size and position. This allows the user to set appropriate human body size and position while checking the analysis results. [Explanation of symbols]

[0052] 200 Image Processing Devices 202 Condition Setting Section 203 Area setting section 204 Area Selection Section 205 Analysis Department 206 Output Control Unit

Claims

1. a setting means for setting a size of a detection target included in an image with respect to a predetermined position of the image; a dividing means for dividing the image into a plurality of partial regions based on the setting by the setting means; a selection means for selecting a part of the series of partial regions based on the setting by the setting means; an analysis means for performing a predetermined analysis on the selected partial region; An image processing device comprising:

2. The image processing apparatus according to claim 1 , further comprising an output unit that outputs information corresponding to the result of the analysis to a predetermined output destination.

3. the selection means selects, from the series of partial regions, a region that includes a position in the image that corresponds to the setting made by the setting means; 3. The image processing device according to claim 1.

4. the selection means sequentially adds regions that have not been subjected to the analysis from the series of partial regions; 3. The image processing device according to claim 1.

5. When the setting by the setting means has not been changed, the selection means selects an area that has not been subjected to the analysis by the analysis means from among the series of partial areas in accordance with a position in the image according to the setting by the setting means.

3. The image processing device according to claim 1.

6. the dividing means divides the image into the plurality of partial regions based on two or more settings made by the setting means, the selection means selects a part of the series of partial regions based on two or more settings made by the setting means.

3. The image processing device according to claim 1.

7. the output means displays the number of detection targets included in the selected area on a predetermined display device to present the number to the user. The image processing device according to claim 2 .

8. the output means displays information indicating that an area that is not set as a partial area or an area that is excluded from the analysis target in association with the area is an area that is not the analysis target; The image processing device according to claim 7 .

9. When the setting by the setting means is changed, information according to the past analysis results by the analysis means is stored; The output means outputs information corresponding to the stored past analysis results to a predetermined output destination. The image processing device according to claim 2 .

10. a setting means for setting the size of an object to be detected in an image in correspondence with a predetermined position; an output unit that outputs a result of a predetermined analysis of a partial region of the image based on the size set by the setting unit; An image processing device comprising:

11. a setting means for setting the image analysis; a dividing means for dividing the image into a plurality of regions based on the setting by the setting means to set partial regions; a selection means for selecting a part of the set series of partial regions based on the setting by the setting means; an analysis means for performing a predetermined analysis on the selected partial region; an output means for outputting information according to the result of the analysis to a predetermined output destination; An image processing device comprising:

12. An image processing method executed by an image processing device, a setting step of setting a size of a detection target included in the image at a predetermined position on the image; a dividing step of dividing the image into a plurality of regions based on the setting in the setting step to set partial regions; a selection step of selecting a part of the set series of partial regions based on the setting in the setting step; an analysis step of performing a predetermined analysis on the selected partial region; An image processing method comprising:

13. An image processing method executed by an image processing device, a setting step of setting a size of an object to be detected in an image in correspondence with a predetermined position; an output step of outputting a result of a predetermined analysis of the partial region of the image based on the size set in the setting step; An image processing method comprising:

14. An image processing method executed by an image processing device, a setting step for setting up image analysis; a dividing step of dividing the image into a plurality of regions based on the setting in the setting step to set partial regions; a selection step of selecting a part of the set series of partial regions based on the setting in the setting step; an analysis step of performing a predetermined analysis on the selected partial region; an output step of outputting information according to the result of the analysis to a predetermined output destination; An image processing method comprising:

15. A program for causing a computer to function as each of the means of the image processing device according to any one of claims 1 to 11.

Images