Information processing device, information processing method, and program

Abstract

To visually and intuitively set detection objects to be used to calculate parameters, in an information processing device that calculates parameters for estimating the size of a detection object corresponding to each position in an image based on positions and sizes of at least three detection objects.SOLUTION: An information processing device calculates parameters for estimating size of a detection object corresponding to each position in an image, based on positions and sizes of at least three detection objects in the image. The information processing device displays, on the image, a polygon whose vertices are positions of the at least three detection object candidates in the image. Based on a user input, the information processing device determines the at least three detection object candidates as the at least three detection objects. The information processing device calculates the parameters for estimating the size of the detection object corresponding to each position in the image based on positions and sizes of the determined at least three detection objects.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a technique for detecting an object from an image. [Background technology]

[0002] In general, many image processing devices that perform detection processing require the size of a detection target. Patent Document 1 discloses a technology that sets the size of one pixel of an image in the real world based on the results of image analysis processing. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6261815 Summary of the Invention [Problem to be solved by the invention]

[0004] When estimating the size of a detection target corresponding to a position on an image, there is a technique for estimating the relationship between the positions and sizes of three or more detection targets. However, there are cases where the estimation of the relationship between the positions and sizes of the detection targets in the entire image is inappropriate, such as when the detection targets used here are densely packed. The technique described in Patent Document 1 does not indicate which of the detection targets detected by image analysis is used to calculate the parameters, so it is difficult for a user to visually and intuitively determine whether the case is inappropriate.

[0005] The present invention aims to provide an information processing device that calculates parameters for estimating the size of a detection target corresponding to each position in an image based on the positions and sizes of at least three detection targets, and to set the detection targets used to calculate the parameters in a visually and intuitively easy-to-understand manner. [Means for solving the problem]

[0006] In order to achieve the object of the present invention, for example, an information processing device according to one embodiment has the following configuration: That is, the information processing device is characterized by having a first display means for displaying on an image a polygon having vertices at the positions of at least three detection target candidates in the image, a second acquisition and determination means for acquiring a user input for finalizing the at least three detection target candidates as the at least three detection targets based on the user input, and a calculation means for calculating parameters for estimating the size of the detection target corresponding to each position in the image based on the positions and sizes of the at least three detection targets determined. Effect of the Invention

[0007] The detection target used to calculate the parameters can be set in a visually and intuitively understandable manner. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a diagram showing the hardware configuration of the information processing device according to the first embodiment. [Diagram 2] FIG. 2 is a diagram showing the functional configuration of the information processing device according to the first embodiment. [Diagram 3] 4 is a flowchart showing information processing performed by the information processing device according to the first embodiment. [Figure 4] FIG. 2 is a view showing an image group used by the information processing device according to the first embodiment. [Diagram 5] 4A to 4C are diagrams showing auxiliary figures displayed by the information processing device according to the first embodiment. [Figure 6] 6A to 6C are diagrams showing displays according to the evaluation of appropriateness by the information processing device according to the first embodiment. [Figure 7] 6 is a diagram showing evaluation of appropriateness according to an overlapping region in the information processing device according to the first embodiment. [Figure 8] 11A to 11C are views showing an image group used by an information processing device according to a second embodiment. [Figure 9] 13A and 13B are diagrams showing detection lines and partial regions set by an information processing device according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0010] [Embodiment 1] The information processing device according to this embodiment calculates parameters for estimating the size of the detection target corresponding to each position in the image based on the positions and sizes of at least three detection targets in the image. To this end, the information processing device 100 according to this embodiment acquires a user input specifying three or more candidates for the detection targets in the image, and displays a polygon having each of the specified candidates for the detection targets as vertices. Next, the information processing device 100 acquires a user input for confirming the specified candidates for the detection targets as the detection targets to be used in the calculation of the above-mentioned parameters. Hereinafter, when simply written as "candidates", it refers to at least three candidates for the detection targets to be used in such parameter setting.

[0011] Hereinafter, an information processing device 100 used in the people counting system according to this embodiment will be described. Here, the people counting system is a system that detects detection targets in an image by image analysis and counts the number of detected detection targets. The detection targets are described as human bodies, but are not particularly limited as long as they are detectable, such as animals or vehicles, and detection processing of multiple types of detection targets may be performed simultaneously. In the people counting system according to this embodiment, the size of the human body taking into account the position (perspective) on the image is used as a parameter in the detection processing. Hereinafter, the detection processing may be performed on the entire image, or may be performed only on a partial area in the image. Hereinafter, the term "analysis" will be described as referring to the processing of counting the number of people described above.

[0012] FIG. 1 is a block diagram showing an example of a hardware configuration of an information processing device 100. The information processing device 100 includes a processor 101, a memory 102, a network interface (I / F) 103, a display device 104, and an input device 105. The processor 101 is, for example, a CPU, and performs processing by each functional unit of the information processing device 100 shown in FIG. 2. The memory 102 is, for example, a ROM and a RAM, and stores data or programs used by the processor 101. The display device 104 is, for example, a liquid crystal display, and displays the results of processing by the processor 101 and presents them to a user. The input device 105 is, for example, a mouse and a keyboard, a touch panel, or a button, and acquires user input. The I / F 103 is an interface when the information processing device 100 connects to a network. In this embodiment, the processor 101 executes processing based on a program stored in the memory 102, thereby realizing functions by each functional unit of the information processing device 100.

[0013] 2 is a block diagram showing an example of a functional configuration of the information processing device 100 according to this embodiment. The information processing device 100 includes an image acquisition unit 201, a display unit 202, a calculation unit 203, a setting unit 204, and an analysis unit 205.

[0014] The image acquisition unit 201 acquires one or more images to be processed. The image acquisition unit 201 may acquire images stored in the memory 102, may acquire images via the I / F 103, or may acquire images using an imaging unit (not shown). The image acquisition unit 201 can acquire a set of images that meet specific conditions (for example, images taken by the same camera at predetermined time intervals) using information about the images such as the image capture date and time.

[0015] The display unit 202 displays the image acquired by the image acquisition unit 201. Next, the display unit 202 acquires a user input for designating a candidate on the acquired image. For example, the display unit 202 can display a preview image for designating the candidate. Although a detailed description will be given later with reference to Figs. 4 to 7, the display unit 202 may display a designator used for designating a candidate superimposed on the image. In this case, the display unit 202 may acquire a user input for operating a designator for designating a detection target by a click operation using a mouse cursor or by a touch operation on a touch panel. Here, the detection target on which the designator arranged by the user operation is superimposed is the detection target designated by the user input. This process is not particularly limited as long as it acquires a user input for designating a candidate, and for example, the display unit 202 may acquire designation of coordinates in the image as a user input. The display unit 202 acquires designation for a predetermined number of candidates equal to or greater than three (for example, 3, 4, 5, or more), but may be configured to allow the user to designate any number of candidates before performing a candidate confirmation operation.

[0016] Furthermore, the display unit 202 may obtain user input for designating candidates across multiple images. For example, the display unit 202 can obtain multiple images that are consecutive in time and obtain user input for one or more of them. For this purpose, the display unit 202 may be capable of switching between and displaying multiple images used to designate candidates, or may display them simultaneously. When candidates are designated for multiple images, an image that displays an auxiliary figure (described later) may be a representative image of those multiple images (e.g., an image at the most recent time), or may be an image obtained by combining multiple images (e.g., an image that simultaneously displays designators for each candidate).

[0017] The display unit 202 displays an auxiliary figure (polygon) on the image, with the positions of each of the designated candidates as vertices. The user can check this auxiliary figure and determine whether the designated candidates are appropriate. For example, when three candidates are designated, the display unit 202 can display a triangle with the positions of each of the candidates as vertices, superimposed on the image. Here, the center position of the designator is used as the position of each candidate, but any detected position, such as the detected position of the head, may be used, or a position obtained from within a range with a predetermined margin above and below the detected position may be used. In addition, each side of the auxiliary figure is basically a straight line, but may be a curved line, such as a parabola.

[0018] The display unit 202 may generate evaluation information on the suitability of each designated candidate as a candidate according to the position of each candidate, and display the auxiliary figure in a manner according to the generated evaluation information. The display unit 202 according to this embodiment generates evaluation information on the suitability of each candidate as a candidate according to the area of ​​the auxiliary figure determined according to the position of each candidate. The display unit 202 may display the auxiliary figure in a darker color as the area increases, or may display the auxiliary figure in different colors depending on whether the area exceeds a predetermined threshold.

[0019] 4 to 6 are diagrams for explaining display by the display unit 202 according to this embodiment. FIG. 4 shows an example of a preview image displayed by the display unit 202. The image group 400 is a plurality of images presented to the user for specifying a candidate. A human body 402 is shown in the image 401, and this is detected by the detection process. The image group 400 may include images that do not show the detection target.

[0020] FIG. 5 is a diagram showing an example of a GUI displayed by the display unit 202 to allow the user to specify candidates. In the setting screen 500, the user uses a specifier 502 to specify a candidate human body from among the human bodies shown in the image 501. In this case, in order to specify a candidate, the specifier is arranged so that the position and size match those of the upper body of the human body. However, for example, the specifier may be aligned with the whole body so that the height matches, or the specifier may be aligned with the head so that the head width matches. The auxiliary figure 503 is a polygon with each of the specified candidates as vertices, and is a triangle in this case. In addition, when a confirmation button 505 on the setting screen 500 is pressed, the candidate specified at that time is set as the human body to be used for calculating parameters for estimating the size of the detection target. In addition, the setting screen 500 displays an image change button 504 for changing the image to be displayed. When a user specifies a candidate from a different image, the user can switch the image to be displayed by pressing the image change button 504. In this case, the specifier arranged before changing the image to be displayed is erased, and a new candidate is specified in the newly displayed image. On the other hand, the designators placed on the images may remain even after the images are changed, and auxiliary graphic 503 may be formed based on all the designators placed across the multiple images.

[0021] Moreover, the setting screen 506 is a screen that appears when the confirm button 505 is not pressed on the setting screen 500, and a candidate is reselected after the image change button 504 is pressed. On the setting screen 506, three designators 502 different from those on the setting screen 500 are designated, and an auxiliary figure 503 having a larger area is displayed. When changing the image to be displayed, the display may be changed to the next image (for example, the next image that is consecutive in time) every time the image change button 504 is pressed. When changing the image to be displayed, the display may be changed to the previous image or the next image in response to pressing a "previous image" button or a "next image" button as displayed on the setting screen 506, respectively.

[0022] FIG. 6 is a diagram showing an example of a GUI that displays an auxiliary figure according to evaluation information of suitability calculated from a designated candidate. In this case, considering that if the designated candidates are crowded together, the influence of individual differences becomes large with respect to the size difference depending on the position, the greater the area of ​​a polygon having each designated candidate as a vertex, the higher the suitability is set. In FIG. 6, the auxiliary figure is displayed in a shading display according to its area (so that the larger the area, the darker the color). Setting screens 601 to 603 display auxiliary figures 603 to 605, respectively, and the display colors of auxiliary figures 605, 604, and 603 are darker in order according to their areas. The display unit 202 may perform a visually easy-to-understand display by displaying a color palette 606 indicating the relationship between the display color and the degree of suitability on the setting screen. The display color may be changed by changing the fill color of the auxiliary figure or the color of the frame, and the location is not particularly limited as long as the corresponding auxiliary figure can be distinguished. When changing the color of the frame line, the display unit 202 may unify the colors of all the sides of one auxiliary figure, or may display each side in a different color. For example, the display unit 202 may display the sides of the auxiliary figure in a color according to their length, and prompt the user to reset candidates for forming short sides. The color according to the length of the side may be displayed in a shade according to the length, similar to the color according to the area of ​​the auxiliary figure, or may be displayed in a different color depending on whether or not the length exceeds a predetermined threshold.

[0023] Here, the description will be given on the assumption that the evaluation information of appropriateness is displayed by color. However, it is not particularly limited to this as long as the user can grasp whether it is appropriate or not. The display unit 202 may display the evaluation of the appropriateness of the auxiliary graphic by, for example, a message, or may display it by blinking display (for example, when the area is higher than the threshold value, the auxiliary graphic blinks). Here, for example, a first threshold value t1 and a second threshold value t2 (t1 < t2) are set, and the evaluation of the appropriateness of the auxiliary graphic is considered high when the area S is larger than t2, medium when S is higher than t1 and equal to or less than t2, and low when S is equal to or less than t1. In this case, the display unit 202 may perform corresponding character display of "excellent", "acceptable", or "unacceptable" according to the evaluation of appropriateness, may perform "excellent" display when t2 < S and it is considered appropriate as a candidate, or may perform "unacceptable" display when it is determined that resetting should be prompted when S ≤ t1. The value of the threshold used here may be set in advance, or may vary according to the setting, such as a predetermined ratio of the area of the region where the analysis process is performed.

[0024] The display unit 202 may also display a warning (for example, at the timing when the confirm button 505 is pressed) according to the position of the designated candidate. For example, the display unit 202 may display a warning to prompt re-setting when the area of ​​the auxiliary figure is smaller than a predetermined threshold. In addition, when the positions of the designated candidates are aligned in a straight line, the range in which the size of the human body can be estimated by the calculated parameters is limited, so the display unit 202 may display a warning to prompt re-setting. In addition, when the designated candidates are concentrated near one straight line (aligned nearly in a straight line), the display unit 202 may display a warning to prompt re-setting in consideration of the decrease in accuracy of the analysis process in the vertical direction to the line. Here, when the positions of the designated candidates are included in a partial area having a predetermined width centered on a certain straight line, the designated candidates are treated as being concentrated near one straight line. Note that, in this embodiment, the display unit 202 displays a warning to prompt re-setting when the evaluation of the appropriateness is low, but the display of the confirm button may be changed so that the pressing operation is disabled while the evaluation of the appropriateness is low.

[0025] FIG. 7 is a diagram showing an example of an index displaying evaluation information of the appropriateness of a designated candidate. In this example, an area 702 to be analyzed is set in an image, the area of ​​the area 702 is set as S1, the area of ​​the auxiliary figure is set as S2, and the area of ​​the area where the area 702 and the auxiliary figure overlap is set as S3. Here, the display unit 202 may evaluate the appropriateness of the auxiliary figure as "not acceptable" when S2 is equal to or less than a predetermined percentage (X% or less) of S1, "passable" when S2 is greater than X% of S1 and equal to or less than S1, and "good" when S2 exceeds S1. Here, X is a value greater than 0 and equal to or less than 100, which is set according to a desired accuracy. When performing the evaluation in this way, the evaluation method can be set before setting the position of the area 702.

[0026] For example, the display unit 202 may evaluate the appropriateness of the auxiliary graphic as excellent when S3 and S1 match, acceptable when 0 < S3 < S1, and unacceptable otherwise. When performing such an evaluation, the evaluation can be carried out in consideration of the setting of the position of the region 702.

[0027] Also, in FIG. 7, the region 704 is the region of the inner edge of the image, and the region 705 is the region where the auxiliary graphic 702 and the region 704 overlap and has an area of S5. In imaging using a wide-angle lens, the inner edge of the captured image such as the region 704 is likely to be distorted. Therefore, it is considered that the relationship between the size and the position of the human body is also distorted as it approaches the periphery of the image, so the human body at such a position is likely to be inappropriate as a candidate. From such a perspective, the display unit 202 may calculate evaluation information on the appropriateness of the auxiliary graphic according to the degree of overlap of the region 705 with the region 702. In this example, the display unit 202 evaluates as excellent when S4 is 0, acceptable when it is greater than 0 and less than or equal to the threshold (0 < S4 ≤ Y), and unacceptable when Y < S4. Here, Y is the degree used for the determination of the above-mentioned degree of overlap, and may be a fixed value or a value set according to the area of the region 704 (for example, in proportion). Here, the region of the inner edge portion may be a region having a predetermined width (for example, a width of 1 / 10 of the width of the entire image) from the edge of the image, or may be a region excluding a figure such as an ellipse that is substantially inscribed in the image from the entire image as shown in FIG. 7.

[0028] The calculation unit 203 calculates parameters for estimating the size of the human body corresponding to each position in the image based on the determined position and size of the human body used for the calculation of the parameters. Here, the calculation unit 203 calculates the parameters in the function for calculating the size s of the human body with respect to the position (x, y) of the human body in the image. For example, the calculation unit 203 can calculate the parameters a, b, and c assuming the relationship between the position and the size is s = ax + by + c. In this case, since there are three unknown parameters, at least three positions (x i , y i ) and size s of the human body ii is required. The parameter calculation process can be performed in the same manner as a general process in which one or more unknown parameters that associate coordinates in an image with the size of a detection target are calculated using statistical processing such as the least squares method. When parameters are calculated in the same manner for a detection target other than a human body, the size of the detection target may be acquired from an image by a detection process, or may be calculated using the ratio to the size of a human body. The size ratio between a human body and other detection targets may be set in advance depending on the type of the detection target, may be configured to be set by the user, or may be automatically estimated during the detection process.

[0029] The setting unit 204 performs settings necessary for analysis processing. The setting unit 204 according to this embodiment refers to, edits, and registers settings used for human body detection, tracking, and counting the number of people. In addition, the setting unit 204 may perform a part or all of the processing performed by the display unit 202 and the calculation unit 203 described above. The analysis unit 205 performs analysis processing on the image acquired by the image acquisition unit 201. The analysis processing will be described in detail in the second embodiment.

[0030] 3 is a flowchart showing an example of processing performed by the information processing device 100 according to this embodiment. In S301, the image acquisition unit 201 acquires one or more images for preview. In S302, the display unit 202 displays one of the images acquired in S301, and displays a designator and an auxiliary figure superimposed on the image. Here, it is assumed that a predetermined number (for example, three) of human bodies are selected as candidates as an initial setting, and the designator and auxiliary figure are displayed.

[0031] In S303, the display unit 202 acquires a user input for designating a candidate. In the following S304, the display unit 202 displays a designator and an auxiliary figure according to the acquired designation of the candidate. In S305, the display unit 202 acquires a user input for confirming the designated candidate as a human body to be used for parameter calculation. Next, the calculation unit 203 calculates and registers parameters for estimating the size of the human body corresponding to each position in the image based on the position and size of the human body confirmed to be used.

[0032] According to such a configuration, in an information processing device that calculates parameters for estimating the size of a detection target corresponding to each position in an image based on the positions and sizes of at least three detection targets, a polygon having each candidate for the detection target as a vertex can be displayed. Next, a user input for confirming the candidates as detection targets to be used when calculating the parameters can be acquired, and the above-mentioned parameters can be calculated. Therefore, by visually presenting the positional relationship of the candidates for the detection targets to be used at that time before calculating the parameters for estimating the size of the detection target for each position, it is possible to easily acquire a confirmation instruction from the user when an appropriate candidate is selected.

[0033] [Embodiment 2] The information processing device 100 used in the traffic volume measurement system according to this embodiment will be described below. Here, the traffic volume measurement system is a system that detects detection targets in an image by image analysis and counts the number of detection targets that pass through a detection line provided in the image. The detection targets are described as human bodies or vehicles, but are not limited to detectable targets such as animals. This analysis process is performed for each detection target whose number is to be counted. In order to reduce the analysis load, the analysis process is performed only in the area near the detection line, not the entire image, but may be performed on the entire image, or may be performed on a partial area of ​​interest in addition to the area near the detection line.

[0034] Here, the detection line is a line segment or a line-segment-like region provided in an image to measure the number of passing objects to be detected, as used in the above-mentioned traffic volume measurement system. The detection line and the partial area for analysis set near the detection line can be any of those used in general traffic volume measurement systems, and a detailed description of the known technology will be omitted. Hereinafter, partial images in the partial area are extracted from multiple images included in the video, and an analysis process is performed on the extracted partial images to measure the flow of the objects to be detected. Hereinafter, partial images corresponding to such partial areas will also be referred to as partial areas.

[0035] The information processing device 100 according to this embodiment has a similar hardware configuration and functional configuration as shown in FIG. 1 and FIG. 2 of the first embodiment, and can perform similar processing, so duplicated explanations will be omitted.

[0036] The analysis unit 205 according to this embodiment measures the flow of the detection target based on the moving image. To this end, the display unit 202 sets a detection line on the image, which indicates the position where the flow of the detection target is measured. The analysis unit 205 can measure the flow rate of the detection target that crosses the detection line and moves from one area divided by the detection line to the other area, that is, the number of detection targets that pass through.

[0037] Various methods can be used to measure the flow. For example, a method of detecting and tracking a person to be detected, or a method of directly calculating the flow rate by estimating the position, moving direction, moving speed, etc. of the person to be detected can be used. Examples of algorithms for implementing such measurement methods include a matching method, a method using optical flow, a method using machine learning, and a method using a neural network. Furthermore, a combination of these methods can also be used.

[0038] For measuring the flow, a partial image may be used alone or multiple partial images may be used simultaneously. When multiple partial images are used, partial images taken at the same time may be used or partial images taken at different times may be used.

[0039] Specific processing methods by the analysis unit 205 include, for example, the following methods. First, the analysis unit 205 estimates the position of the detection target around the detection line at time t1 by inputting each partial image at time t1 into a neural network. Similarly, the analysis unit 205 estimates the position of the measurement target around the detection line at time t2 by inputting each partial image at time t2 into this neural network. This neural network can be trained to estimate the position of the measurement target (e.g., a person's head) in an image from the image. In addition, as another method for improving the estimation accuracy, a neural network trained to estimate the density distribution of the measurement target in an image from the image and a neural network trained to estimate the position of the measurement target from the density distribution may be used in combination. With this method, the analysis unit 205 can estimate the position of the measurement target in each different region independently for each different region by using partial images extracted from the region.

[0040] Next, the analysis unit 205 estimates the trajectory of the measurement object between time t1 and time t2 by matching the estimated position of the measurement object at time t1 with the estimated position of the measurement object at time t2. As a matching method, a method of minimizing a cost according to the distance between the measurement objects to be matched can be used, for example, the Hungarian matching method. When the trajectory estimated in this way intersects with the detection line, it can be determined that one measurement object has passed the detection line. Such matching processing and trajectory estimation may be performed simultaneously based on the positions of the measurement objects detected from each partial image.

[0041] However, the method of measuring the flow is not limited to the above method. The analysis unit 205 may estimate the trajectory of the measurement object in each region or measure the flow of the measurement object using partial images extracted from each different region independently. For example, the analysis unit 205 may not only estimate the position of the measurement object for each partial image, but also estimate the trajectory of the measurement object for each partial image. In addition, partial images at the same position at time t1 and time t2 may be input to a neural network to estimate the position, moving direction, moving speed, etc. of the measurement object, thereby estimating the flow of the measurement object.

[0042] The display unit 202 according to this embodiment can display the acquired image as a preview image for setting the detection line, in addition to the same processing as in the first embodiment. The display unit 202 may display a detection line specifier used when specifying the detection line by superimposing it on the image. In this case, the display unit 202 may acquire a user input for operating the specifier for setting the detection line by a click operation using a mouse cursor, or by a touch operation on a touch panel. Here, the start point and end point of the detection line are arranged by a user operation, and the line segment from the start point to the end point is set as the detection line input by the user. This processing is not particularly limited as long as it acquires a user input for setting the detection line. For example, the display unit 202 may acquire the designation of the coordinates of the start point and end point of the detection line as the user input. The setting unit 204 may acquire the designation of two points without distinguishing between the start point and the end point, and then acquire the direction of the detection line, or may acquire the input of the start point, direction, and length. The display unit 202 acquires the setting of a predetermined number of detection lines (for example, 1, 2, or more), which is equal to or greater than 1, and the number can be determined arbitrarily. Note that, although the detection lines are basically described as straight lines, they may be any broken line or curved line, for example.

[0043] The display unit 202 according to the present embodiment sets a partial area (patch) for analysis near the detection line. The display unit 202 sets the patches so that the entire detection line is included by a plurality of patches. Here, the display unit 202 sets the size of the patch using the size of the detection target estimated for the position of the patch by using parameters calculated by the calculation unit 203 in the same manner as in the first embodiment. For example, the display unit 202 can set the size of the patch, each of which has its center on the detection line, based on the size of the detection target estimated from the position of the center. Here, the patch is a square area with a side length 6.5 times the size of the detection target corresponding to the position of the center of the patch (for example, the shoulder width of a human body), but the size and shape are not particularly limited and may be, for example, a rectangle, a trapezoid, or a circle. The information processing device 100 can perform analysis processing on the area within this patch and measure the number of detection targets.

[0044] Furthermore, when the patch is a trapezoid, the display unit 202 can set the lengths of the upper and lower bases of the patch based on the size of the detection target corresponding to the position of the center (e.g., center of gravity) of the patch. For example, the display unit 202 may set the length of the upper base of the patch to four times the size of the detection target at the center of gravity, and the length of the lower base of the patch to 6.5 times the size of the detection target at the center of gravity. By setting the patch to a trapezoid, it is possible to perform analysis processing within the patch taking into account changes in the size of the detection target due to perspective. Note that the trapezoidal patch is set so that the depth direction due to perspective in the image is perpendicular to the upper and lower bases of the trapezoid, but may be adjusted according to desired conditions.

[0045] In order to suppress measurement omissions near the ends of the detection line, the display unit 202 may set a partial area (detection patch) in which the number of detection targets is not measured but detection and tracking processing is performed, in addition to the above-mentioned patch, near the detection line. Here, the detection patch is a partial area that is set for each patch and completely contains the corresponding patch. The shape, size, and position of the detection patch are not limited as long as it completely contains the corresponding patch. Herein, it is assumed that the detection patch is an area obtained by excluding the patch from a partial area that has the same shape as the patch, has a size of a predetermined ratio to the patch (patch size < detection patch size), and is set with the center coordinates aligned with the corresponding patch. In the following description, the patch is a square area with one side 6.5 times the shoulder width of the human body, while the detection patch is an area obtained by excluding the patch from a square area with one side 8 times the shoulder width of the human body. By generating such a detection patch, it is possible to suppress measurement omissions of detection targets by starting detection and tracking processing in the detection patch, which is the area of ​​the outer edge of the patch, before measuring the number of detection targets in the analysis processing in the patch.

[0046] 8 shows an example of a preview image displayed by the display unit 202 according to this embodiment. The image group 800 is a collection of two or more images 801. The image 801 shows a human body 802 and a vehicle 803, which are detection targets, and these are detected by a detection process in the traffic volume measurement system. The image group 800 may also include images that do not show the detection targets.

[0047] 9 is a diagram for explaining an example of processing for setting patches for a detection line, performed by the information processing device 100 according to this embodiment. In an image 901, a detection line 903 is displayed, and a plurality of patches 902 are set in the vicinity of the detection line 903 so as to encompass the entire detection line 903 within itself. Here, each patch is a square area whose center is on the detection line 903, and the size of each patch is set as described above based on the position of the center.

[0048] In the image 904, two patches 905 are set in the vicinity of the detection lines 906 so as to encompass each of the two existing detection lines 906. Here, each patch is a trapezoidal region whose center of gravity is on the detection line, and the lengths of the upper and lower bases are set based on the coordinates of the center of gravity. The lengths of the upper and lower bases of this trapezoid may also be set to be equal.

[0049] According to this configuration, it is possible to calculate parameters for estimating the size of the detection target corresponding to each position in the image based on the positions and sizes of the detection targets that the user has confirmed as at least three detection targets by the same process as in the first embodiment. Next, a detection line for measuring the flow of the detection targets is set, and multiple partial regions having sizes based on the size of the detection targets estimated using the parameters calculated from the positions of the partial regions are set near the detection line. Furthermore, it is possible to measure the flow of the detection targets passing through the detection line using partial images extracted from the moving image in the partial regions. Therefore, it is possible to visually and intuitively set the detection targets used to calculate the parameters, and measure the flow of the detection targets based on the settings.

[0050] (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0051] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0052] 201: Image acquisition section, 202: Display section, 203: Calculation section, 204: Setting section, 205: Analysis section

Claims

1. a first display means for displaying, on an image, a polygon having vertices each corresponding to a position of at least three candidates for the detection target in the image; a determining means for determining the at least three candidates of the detection target as the at least three detection targets based on a user input; a calculation means for calculating a parameter for estimating a size of the detection target corresponding to each position in the image based on the determined positions and sizes of the at least three detection targets; An information processing device comprising:

2. a second display means for displaying the detection target on the image; a first acquisition means for acquiring a user input for designating candidates for the at least three detection targets from among the detection targets displayed on the second display means; The information processing device according to claim 1 , further comprising:

3. The method further includes a second acquisition means for acquiring evaluation information indicating an evaluation of suitability of the at least three detection target candidates as the at least three detection targets, The information processing apparatus according to claim 2 , wherein the second display means further displays the evaluation information on the image.

4. The information processing apparatus according to claim 3 , wherein the second acquisition means acquires the evaluation information based on an area of ​​the polygon.

5. The information processing device according to claim 4 , wherein in the evaluation information, the larger the area of ​​the polygon, the higher the evaluation of the appropriateness.

6. A first setting unit is configured to set a first partial area on the image for tracking the detection target, The information processing apparatus according to claim 3 , wherein the second acquisition means acquires the evaluation information based on areas of the first partial region and the polygon.

7. The information processing apparatus according to claim 6 , wherein the second acquisition means acquires the evaluation information based on a ratio between an area of ​​the first partial region and an area of ​​the polygon.

8. The second acquisition means includes: if the area of ​​the polygon is equal to or smaller than a predetermined percentage of the area of ​​the first subregion, setting the appropriateness assessment to a first degree; if the area of ​​the polygon is greater than the predetermined ratio of the area of ​​the first partial region and is equal to or smaller than the area of ​​the first partial region, setting the evaluation of the appropriateness to a second degree higher than the first degree; 8. The information processing apparatus according to claim 7, wherein, when the area of ​​the polygon is larger than the area of ​​the first partial region, the evaluation of the appropriateness is set to a third degree higher than the first degree.

9. The information processing apparatus according to claim 6 , wherein the second acquisition means acquires the evaluation information based on an area of ​​an area where the first partial region and the polygon overlap.

10. The second acquisition means includes: If the first subregion and the polygon do not have an overlapping region, the appropriateness is set to a first degree; When the first partial region and the polygon have an overlapping region, If the area of ​​the overlapping region is smaller than the area of ​​the first partial region, the evaluation of the appropriateness is set to a second degree higher than the first degree; 10. The information processing apparatus according to claim 9, wherein, when an area of ​​the overlapping region is equal to an area of ​​the first partial region, the evaluation of the appropriateness is set to a third degree higher than the second degree.

11. a second setting means for setting a second partial area on an inner edge of the image; The information processing apparatus according to claim 3 , wherein the second acquisition means acquires the evaluation information based on an area of ​​an overlapping region between the second partial region and the polygon.

12. The second acquisition means includes: When the second partial region and the polygon have an overlapping region, When the area of ​​the overlapping region is larger than a threshold determined based on the area of ​​the second partial region and the area of ​​the polygon, the appropriateness is set to a first degree; When the area of ​​the overlapping region is equal to or smaller than a threshold determined based on the area of ​​the second partial region and the area of ​​the polygon, the evaluation of the appropriateness is set to a second degree higher than the first degree; 12 . The information processing apparatus according to claim 11 , wherein, when the second partial region and the polygon do not have an overlapping region, the evaluation of the appropriateness is set to a third degree higher than the second degree.

13. The information processing device according to claim 3, characterized in that the second acquisition means sets the evaluation of the appropriateness lower when all of the at least three detection target candidates are located within a partial area having a predetermined width centered on a line segment passing through the image than when they are not.

14. 14. The information processing device according to claim 3, further comprising: a display unit configured to display a warning to a user when the appropriateness setting indicated by the evaluation information is set lower than a predetermined condition.

15. The information processing apparatus according to claim 3 , wherein the second display means changes a fill color or a frame color of the polygon in accordance with the evaluation of the appropriateness.

16. a third setting means for setting a detection line in a moving image, the detection line indicating a position for measuring the flow of the detection object; An extraction means for extracting a plurality of partial images set in the vicinity of the detection line from each of a plurality of images in the moving image, an extraction means for setting sizes of the plurality of partial images based on a size of the detection target corresponding to a position of the partial image, the size being estimated using the parameters; a measuring means for measuring a flow of the detection object passing through the detection line by using the partial image; The information processing device according to claim 1 , further comprising:

17. displaying, on an image, a polygon having vertices each corresponding to a position of at least three candidates for the detection target in the image; determining the at least three candidate detection targets as the at least three detection targets based on a user input; calculating a parameter for estimating the size of the detection target corresponding to each position in the image based on the determined positions and sizes of the at least three detection targets; An information processing method comprising:

18. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 16.

Images