Information Processing Apparatus, Information Processing Method, and Program

Abstract

To provide an information processing apparatus, an information processing method, and a program for reducing a burden imposed on a user when converting the position of a subject in an image into a position in the other coordinate system.SOLUTION: An information processing apparatus 100 comprises: an acquisition unit 101 that acquires an image to be processed; a tracking unit that, for at least one or more objects detected from the acquired image, generates first information including the positions and sizes of the objects in a first coordinate system for indicating the positions in the image; and a conversion unit that, based on the positions and sizes of the objects satisfying a predetermined condition, converts the positions in the first coordinate system into those in a second coordinate system and outputs the converted positions.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In order to detect the actions and situations of people, technologies have been developed to detect human bodies in videos and measure the positions of people and the distances between people. In Patent Document 1, a method is disclosed for converting the position of a person in an image into three-dimensional coordinates by the user inputting the height, depression angle, angle of view, and focal length of the camera lens. Further, in Patent Document 2, a method is disclosed for converting the position of a person in an image into three-dimensional coordinates by the user inputting a correspondence table between the coordinates of the image and the two-dimensional position of the floor surface.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in Patent Documents 1 and 2, since input by the user is required at the time of system installation, the burden on the user is high.

[0005] An object of the present invention is to reduce the burden on the user when converting the position of a subject in an image to a position in another coordinate system.

Means for Solving the Problems

[0006] To achieve the object of the present invention, for example, an information processing apparatus according to an embodiment includes the following configuration. That is, for at least one or more objects detected from an image, acquisition means for acquiring the position and size of the object in a first coordinate system for indicating the position in the image, Calculating means for calculating a parameter for estimating a position in a second coordinate system different from the first coordinate system from a position in the first coordinate system based on the position and size of the object obtained by the obtaining means, and the parameter Based on this, output means for outputting the position of at least one or more objects detected from the image in the first coordinate system to the position The first converted to the second coordinate system. It is characterized by comprising.

Advantages of the Invention

[0007] Reduce the burden on the user when converting the position of the subject in the image to the position in another coordinate system.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Best Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0010] [Embodiment 1] The information processing apparatus according to Embodiment 1 calculates a parameter for converting coordinates on an image into coordinates in a second coordinate system based on information indicating the position and size in a first coordinate system on the image of a subject detected from the image. That is, when performing conversion from coordinates on an image to three-dimensional coordinates, the conversion parameter can be calculated by a simple method that does not require parameters input by the user. Such an information processing apparatus can be implemented as a camera system that transmits and receives data when detecting a person's individual action (that is, when there is a risk of shoplifting) by analyzing the video of the position of a person in a store and the surrounding situation in a surveillance system in a retail store such as a convenience store. In the present embodiment, for the purpose of preventing shoplifting, a notification is transmitted to an external analysis device or a notification unit (not shown) held by the user so as to call out to a person who may shoplift.

[0011] FIG. 2 is a block diagram showing an example of the hardware configuration of the information processing apparatus according to the present embodiment. The information processing apparatus 100 includes an input unit 201, a display unit 202, an I / F unit 203, a CPU 204, a RAM 205, a ROM 206, an HDD 207, and a data bus 208. The input unit 201 is, for example, a keyboard and a mouse, or a touch panel, etc., and acquires user input. The display unit 202 is a display device such as a liquid crystal display, etc., and displays the result or process of the processing by the information processing apparatus 100 and presents it to the user. The I / F unit 203 performs transmission and reception of various information between the information processing apparatus 100 and other devices via the Internet.

[0012] The CPU 204 loads the control computer program stored in the ROM 206 into the RAM 205 and executes various control processes. The RAM 205 is used as a region for temporarily storing the program executed by the CPU 204, or a temporary storage region such as a work memory. The HDD 207 stores image data, setting parameters, or various programs, etc. These respective units are connected by the data bus 208, and data transmission and reception are performed. In the present embodiment, it is assumed that the processing according to the present embodiment is realized by the CPU 204 executing the image processing program stored in the ROM 206 or the HDD 207, but it may also be realized by dedicated hardware having each functional unit of FIG. 1. Note that image processing and image analysis may be performed using a GPU (Graphics Processing Unit) instead of the CPU. Further, the HDD 207 can acquire data such as image data from an external device via the I / F unit 203, and can transmit and receive the acquired data between the CPU 204, the RAM 205, and the ROM 206.

[0013] FIG. 1 is a block diagram showing an example of the functional configuration of the information processing apparatus according to the present embodiment. The information processing apparatus 100 includes an acquisition unit 101, a tracking unit 102, a management unit 103, a filter unit 104, a calculation unit 105, a conversion unit 106, an estimation unit 107, a determination unit 108, and a transmission unit 109.

[0014] The acquisition unit 101 acquires an image to be processed. The acquisition unit 101 may acquire an image captured by an imaging unit (not shown) such as a security camera, may acquire an image stored in a storage device such as a hard disk, or may acquire an image via a network such as the Internet. The acquisition unit 101 according to the present embodiment acquires a plurality of temporally consecutive images (videos), and the following-described processing is performed using each of them. The acquisition unit 101 transmits the acquired image to the tracking unit 102.

[0015] The tracking unit 102 detects a subject in the image acquired from the acquisition unit 101, and generates human body information (first information) by performing a tracking process of associating the detected subjects between images. Here, the subject is a human body, and the human body information is information indicating a rectangle enclosing the head of the human body (for example, a bounding box). However, for example, the rectangle may enclose the entire human body or a part that is not the head, and its shape does not have to be a rectangle. That is, the first information includes the position or size of the detected person (object) (the rectangle indicating them). Further, it may include information that can be distinguished from others (for example, the following person ID). Note that the subject may be other than a person, and may be an object such as a wheelchair, an animal, a robot, or an automobile. The tracking unit 102 according to the present embodiment generates human body information by using machine learning to generate the center coordinates, width, height, and detection reliability of the rectangle. The tracking process is a process of associating the detected human body between consecutive frames (images), and since a known technique is used, a detailed description is omitted. As the tracking process, the tracking unit 102 performs association between frames using the center coordinates, width, and height of the rectangle and the predicted position based on the past tracking results, and assigns a tracking ID and tracking reliability for identifying each subject. The tracking ID is set so that the same identifier is assigned to the same human body between each image. The tracking unit 102 transmits the generated human body information and tracking ID to the management unit 103 and the conversion unit 106. Hereinafter, the human body information will be described as simply referring to both a rectangle in a single image and a series of rectangles indicating the same subject tracked between images.

[0016] Figure 3 is an example of the result of the tracking process that displays the rectangles in the image indicated by the human body information generated by the tracking unit 102. In the image 300, each human body including the human body 301, the shield 302, the tracking ID 303, and the human body information (rectangle) 304 are displayed. The tracking unit 102 can detect the human body 301 by performing the human body detection and tracking process, and generate the tracking ID 303 and the human body information 304. In the example of Figure 3, seven pieces of human body information are generated, and each is assigned a tracking ID from 1 to 7.

[0017] The management unit 103 manages the human body information and the tracking ID generated by the tracking unit 102. In the present embodiment, the management unit 103 performs management by holding the human body information and the tracking ID for a predetermined period. The management unit 103 may hold the human body information and the tracking ID until an instruction from the user via the input unit 201 is obtained, or may hold them for a predetermined period (for example, one day, which may be arbitrarily set by the user) and automatically discard the information after that period has passed. Further, the management unit 103 transmits the held human body information and the tracking ID to the filter unit 104. Figure 4 is an example of the display of the human body information managed and accumulated by the management unit 103. In Figure 4, all the human body information managed by the management unit 103 is drawn on the image 400 (the tracking ID is not shown).

[0018] The filter unit 104 extracts human body information that satisfies the filter conditions from a plurality of pieces of human body information. Further, for the extracted person information, the size and position included in the person information are corrected. Here, extraction of person information used to estimate parameters for correcting the coordinates and size of the human body information is performed based on preset filter conditions. The filter conditions are conditions that the human body information is required to satisfy over a plurality of frames. In the present embodiment, subsequent parameter calculation processing is performed using the human body information that satisfies the filter conditions. A specific example of the filter conditions is that the variation in the position or size of the person information per a predetermined period is within a predetermined reference value. Also, filter conditions may be set to exclude immediately after the start of tracking and the end of tracking from the person information about a certain tracking ID. The conversion parameters in the present embodiment are designed based on the rule that assuming that the actual size of a person is constant, there is a correspondence relationship between the distance in the real space and the size of the person in the image (the smaller the distance, the larger the size of the person, and the farther the distance, the smaller the size of the person). Therefore, by setting the filter conditions, by extracting persons of various sizes, it is possible to improve the accuracy of the parameters while saving the user's labor.

[0019] The filter unit 104 can perform filter processing to correct coordinates and size by applying a Kalman filter based on the time-series information of the position and size. In that case, subsequent processing is performed using the corrected values. Desired conditions may be set as filter conditions by the user in order to adjust the effects of false tracking or size or position deviation. For example, the filter conditions may be that in images captured at multiple times, the detection or tracking reliability is equal to or higher than a certain level (threshold value), the deviation of the position and size between frames is equal to or less than a certain level, or the amount of movement in the vertical or horizontal direction is equal to or more than a predetermined value. Also, the filter conditions may be that a predetermined time or more has elapsed since the start of tracking, that is, a predetermined number or more of frames are included since the start of tracking of the subject. Further, the filter conditions may be that it is a predetermined time or more before the end of tracking, that is, a predetermined number or more of frames are included since the tracking of the subject has ended. Furthermore, the filter conditions may be to satisfy any combination of these conditions. Note that it may be assumed that the human body information selected by user input via the input unit 201 satisfies the filter conditions. Note that the filter conditions may be other conditions as long as they can select person information suitable for parameter estimation. By selecting person information suitable for parameter estimation, the accuracy of conversion can be improved, and a display that is easy for the user to visually recognize (with little sense of discomfort) can be realized.

[0020] Also, the filter unit 104 may exclude the detected person from the processing target in the subsequent parameter calculation processing based on the attributes of the detected human body. For example, the filter unit can perform filter processing such as excluding a person under a predetermined age from the processing target based on the age, gender, or clothing of the person. Since these attributes are detected by general detection processing, the description thereof is omitted.

[0021] The filter unit 104 transmits the human body information after the filter process to the calculation unit 105 and the conversion unit 106. Note that the parameters required for the filter process according to this embodiment are arbitrarily input by the user or another system via the input unit 201, and the filter unit 104 performs the filter process by acquiring them. Here, the parameters that can be input and changed may be, for example, values used as threshold values in each process used for the filter conditions, or the correction strength of the Kalman filter, and are not particularly limited. Each threshold value used here may be set as a desired value by the user, or an arbitrary value may be assigned in advance. An example of the filter process is shown in FIG. 9, and a specific description will be given later.

[0022] The calculation unit 105 calculates parameters (conversion parameters) for converting the coordinates in the image coordinate system into coordinates in a different coordinate system. To that end, the calculation unit 105 obtains parameters indicating the relationship between the position of the human body information on the image and the object size (the size of the subject). The following equation (1) is a relational expression between the position information on the detection plane (a virtual plane on which the center coordinates of the subject move) and the object size, which is used to estimate the object size W from the position information (x, y). W = a(x - xm) + b(y - ym) + wm Equation (1)

[0023] Here, a and b, which are the coefficients of the linear equation, are transformation parameters, and (x, y) is the position (coordinate) on the image coordinates. Also, xm and ym are the average values of the x - coordinate and y - coordinate of the position of the human body included in the generated human body information, respectively. Further, wm is the average value of the size of the human body included in the generated human body information. These transformation parameters a and b are parameters for calculating the distance between the camera that captured the image to be processed and the subject at the position (x, y), taking into account the depth in the image. Hereinafter, the position of the human body information is described as the center coordinates of the rectangle representing the human body information, and the size of the human body is described as the size of the rectangle indicated by the human body information. The transformation parameters a and b can be calculated by the least - squares method using the human body information obtained from the image group accumulated by the surveillance camera. The calculation unit 105 transmits the values of xm, ym, and wm, and the calculated transformation parameters a and b to the transformation unit 106.

[0024] Figure 5 is an example diagram showing, with shading of colors on the image 500, the relationship between the coordinates on the detection plane and the object size 501 generated using the least - squares method. In Figure 5, it is shown that the closer the color is to black, the smaller the object size at that position, and the closer the color is to white, the larger the object size at that position. That is, in the image 500, a depth is set such that it becomes farther as it is closer to the upper - right end.

[0025] The transformation unit 106 uses the values received from the calculation unit 105 to transform the position of the human body included in the human body information to a position in a different coordinate system and outputs it. In the present embodiment, the transformation unit 106 transforms the coordinates in the image coordinate system to three - dimensional coordinates. For example, the transformation unit 106 calculates an estimated value W’ of the object size at the position (x, y) of the human body information according to the following formula (2). W’ = a(x - xm)+b(y - ym)+wm Formula (2)

[0026] Subsequently, the conversion unit 106 calculates the three-dimensional coordinates (X, Y, Z) of the human body information based on the following formula (3) using the coordinates (x, y) and the calculated W'. Here, the three-dimensional coordinates (X, Y, Z) are in the world coordinate system. The floor surface in the real space to be imaged is regarded as the X-axis (for example, the direction indicating east-west) and the Y-axis (for example, the direction indicating north-south), and the vertical direction extending from the floor surface to the ceiling direction is regarded as the Z-axis (indicating the height of the space). Z = focal × B / W' X = Z × (x - cx) / focal Formula (3) Y = Z × (y - cy) / focal

[0027] Here, B is the average value of the preset human body size, and any value such as an average value of 0.43 assuming the shoulder width is set. Also, focal is the focal length of the camera, which may be the one described in the extended area of the image, or may be obtained and used from the camera parameters, or may be preset. cx and cy are the coordinates of the center of the image in the image coordinate system and can be calculated based on the image size. The conversion unit 106 transmits the calculated position to the estimation unit 107 and the determination unit 108.

[0028] FIG. 6 is a diagram showing an example of the position information generated when the conversion unit 106 converts the human body information shown in FIG. 3. At 600, the converted position information 601 shown by a circle and the shielding object 602 are displayed. The shielding object is, for example, a display shelf in the store, and the method of obtaining this position will be described later. Also, in this embodiment, the conversion to the position in the three-dimensional coordinate system centered on the camera is performed, but if the distance between people can be calculated, the conversion to another coordinate system may be performed. Note that, in order to facilitate the expression of the distance between people, in FIGS. 6 to 8, the description is made using a bird's-eye view of the imaged space (store) from directly above. When performing the conversion to the coordinates in such a bird's-eye view, the information processing device 100 may be provided with, for example, a gravity sensor and obtain parameters such as the position and depression angle of the camera with respect to the space where imaging is performed, but this configuration and processing are not particularly essential.

[0029] The estimation unit 107 estimates the position of the shielding object in the coordinate system converted by the conversion unit 106. In the present embodiment, the estimation unit 107 creates information indicating the position of the shielding object based on the distribution of the three-dimensional coordinates of each human body information converted by the conversion unit 106. For example, as shown in FIG. 6, the estimation unit 107 divides a map of three-dimensional coordinates into partial regions of a predetermined size (for example, 1 mm square) in a grid pattern, and measures the number of central positions of the human body information existing in each grid. FIG. 7 is an example of a diagram in which the positions after conversion of the tracking results of each human body information are plotted as broken lines on a map of three-dimensional coordinates as shown in FIG. 6. A map (hereinafter referred to as a shielding map) in which the number of central positions of the human body information in each grid is normalized in the range from 0 to 1 is shown in FIG. 8. In FIG. 8, the estimation unit 107 shows the grids with the number of central positions being 0 in black, and it can be estimated that the region where these grids exist is a region where people do not move, that is, a region where a shielding object exists. The estimation unit 107 transmits the information indicating the position of the shielding object (the shielding map in the present embodiment) created here to the determination unit 108.

[0030] The determination unit 108 determines whether each person is alone or not based on the position after conversion acquired from the conversion unit 106 and the information indicating the position of the shielding object acquired from the estimation unit 107, and transmits the determination result to the transmission unit 109. The determination unit 108 can determine whether each person is alone or not based on, for example, the distance d between each person at the position after conversion and the presence or absence of a shielding object. Here, the determination unit 108 determines that the person i is alone when all the three-dimensional coordinates corresponding to the other person j satisfy the following conditions with respect to the three-dimensional coordinates of the human body information corresponding to a certain person i. In this example, when the distance d between the persons is equal to or greater than a predetermined threshold value, or when there is a shielding object between the persons, it is determined that the person i is alone. The determination unit 108 according to the present embodiment calculates the distance d between the persons according to the following formula (4). d = sqrt((X i - X j ) 2 + (Y i - Y j ) 2 + (Z i - Z j )2 ) Equation (4)

[0031] Here, sqrt is a function for obtaining the square root, and the position of person i is (X i , Y i , Z i ), and the position of another person j is (X j , Y j , Z j ). The value of the threshold used for the determination of d is not particularly limited and can be arbitrarily set by the user in view of being used for the determination of whether it is alone or not.

[0032] Next, the determination of whether there is an obstacle between persons will be described. The determination unit 108 uses the normalized value in the occlusion map acquired from the estimation unit 107 and the line segment (straight line) connecting (X i , Y i ) to (X j , Y j ) on the occlusion map to perform the above determination. First, the determination unit 108 converts the normalized value v in each grid through which the line segment connecting (X i , Y i ) to (X j , Y j ) on the occlusion map passes into an occlusion value Occ (=1 - v). Next, the determination unit 108 calculates any one of the sum, average value, and maximum value of the occlusion values in each linear grid, and if the value is greater than or equal to a preset threshold, it determines that there is an obstacle between the persons.

[0033] The transmission unit 109 transmits the information indicating the person determined to be alone to another analysis device or the notification unit held by the user via the I / F unit 203. By obtaining the information indicating the person who is alone in this way, it can contribute to the implementation of a system that recommends speaking to a single person for theft prevention.

[0034] The information processing apparatus 100 can use the input unit 201 and the display unit 202 to adjust the parameters used in the filter unit 104, check the processing results by the filter unit 104 or the estimation unit 107, or check the operation of the processing by the conversion unit 106. FIG. 9 is a diagram showing an example of a UI for executing and checking such processing. The setting UI 901 on the screen 900 is used for setting the parameters (threshold values) used in the filter unit 104. In the example of FIG. 9, by operating the bar displayed in the setting UI 901, the threshold value of the detection reliability, the correction intensity of the deviation of the detection position, and the correction intensity of the deviation of the detection size can be changed, and the changed setting can be reflected by pressing the apply button 902. Also, a preview is displayed on the screen 900, and the human body information 903 generated by the tracking unit 102 and the occlusion map 904 generated by the estimation unit 107 can be confirmed according to the parameter settings. The user may be able to exclude the corresponding human body information from the processing target, for example, by clicking on the human body information 903 on the preview. Also, the user may be able to edit and correct the values described in the occlusion map 904. Further, the conversion unit 106 can convert the position on the image specified by the user on the screen 900 into a position in three-dimensional coordinates and present it to the user. That is, a coordinate conversion test screen may be displayed on the screen 900, and the operation of the conversion unit 106 may be confirmed for the position specified by the user. Here, when the user clicks on an arbitrary position on the coordinate conversion test screen 905 where the image is displayed, a virtual person is plotted on the corresponding coordinates. Then, the conversion unit 106 converts the position of the virtual person on the image into a position in three-dimensional coordinates and displays it on the conversion result screen 906 and presents it to the user.

[0035] In this embodiment, the conversion parameters a and b are calculated based on Equation (1), but the equations used are only examples and are not particularly limited to these. For example, instead of Equation (1), the relationship between the position information on the detection plane and the object size may be defined by the following Equation (5). W = ax + by + c Equation (5)

[0036] Here, W is the human body size, (x, y) is the position on the image coordinates, and a, b, and c are the estimated conversion parameters. When using Equation (5) instead of Equation (1), the conversion unit 106 also calculates the object size W' using a similar equation instead of Equation (2).

[0037] Also, in this embodiment, the estimation unit 107 measures and adds the number of human body information for each grid and performs normalization. However, if more human body information is added to the area where the person has moved, it is not particularly limited to this method. For example, the estimation unit 107 may not directly measure the number of human body information on the grid, but may also add the measured number to the neighboring grids. For example, the estimation unit 107 may adopt an addition method such as adding 1 to the grid containing human body information and adding 0.25 to the eight neighboring grids. According to such a method, it is possible to create a shielding map with a margin against the occurrence of displacement of the position of human body information and the like.

[0038] FIG. 10 is a flowchart showing an example of the processing performed during the system setting of the information processing apparatus according to this embodiment (that is, the processing in the preparation stage for calculating the conversion parameters). In step S1001, the acquisition unit 101 acquires an image to be processed. In step S1002, the tracking unit 102 performs tracking processing on the acquired image and generates human body information and a tracking ID. In step S1003, the management unit 103 stores the generated human body information and the tracking ID. In step S1004, the management unit 103 determines whether the number of the generated human body information is equal to or greater than a predetermined threshold. In order to estimate the conversion parameters, it is necessary to extract a certain number or more of high-quality human body information. If the number of human body information is equal to or greater than the threshold, the process proceeds to step S1005; otherwise, the process returns to step S1001. Note that instead of determining based on the number of human body information, it may be determined that a certain period of time or more has elapsed since the start of imaging, and the process may proceed to S1005. Any other determination method may be used as long as it is a method that has the possibility of acquiring a certain number or more of suitable human body information.

[0039] In step S1005, the filter unit 104 performs a filtering process based on the generated human body information. That is, based on a preset condition, the human body information is extracted. In step S1006, the calculation unit 105 calculates conversion parameters for converting the coordinates in the image coordinate system to coordinates in a different coordinate system based on the human body information on which the filtering process has been performed. In step S1007, the conversion unit 106 converts the position of the human body information on which the filtering process has been performed into coordinates in three-dimensional coordinates, and the estimation unit 107 creates a shielding map using the converted position and estimates the position of the shielding object. In step S1008, the estimation unit 107 records the conversion parameters and the shielding map and ends the setting process of the system.

[0040] Figure 11 is a flowchart showing an example of the process when the information processing apparatus according to the present embodiment performs image analysis (execution stage). For example, it is a process executed when monitoring a store in real time. The process of Figure 11 is performed using the system set by the process shown in Figure 10. In step S1101, the acquisition unit 101 acquires an image to be processed. In step S1102, the tracking unit 102 performs a tracking process on the acquired image and generates human body information and a tracking ID. In step S1103, the conversion unit 106 acquires the conversion parameters and the shielding map recorded in step S1008. In step S1104, the conversion unit 106 converts the position of the generated human body information into coordinates in three-dimensional coordinates based on the conversion parameters. In step S1105, the determination unit 108 determines whether each person whose position has been converted is alone or not. In step S1106, the transmission unit 109 transmits information indicating the person determined to be alone to another analysis device or a notification unit possessed by the user and ends the process. Note that the transmission unit 109 may return the process to S1101 after the process of step S1106.

[0041] According to such processing, based on the position and size at the coordinates on the image of the human body, parameters for converting coordinates from the coordinates on the image to coordinates in a different coordinate system can be calculated. Also, among the human bodies on the image, those that are alone can be discriminated, and information indicating a person who is acting alone can be provided in a system that prompts a shoplifting prevention call. Therefore, the load on the user when installing such a system can be reduced, and the accuracy of the solitary action determination can be improved.

[0042] In addition, in the present embodiment, the subsequent conversion parameter generation process and the occlusion map creation process are described as being performed using the human body information selected by the filter process of the filter unit 104. However, the human body information used in the conversion parameter generation process and the occlusion map creation process may be different. For example, conditions for the human body information to be preferentially used may be set for each of these processes (that is, separate filter conditions are set), and each process may be performed using the human body information that satisfies the conditions. For example, the filter unit 104 may set, as filter conditions in the conversion parameter generation process, that the person is moving in the vertical direction and that the deviation in size is equal to or less than a predetermined value. And the filter unit 104 may set, as filter conditions in the occlusion map creation process, that the person has a movement speed between frames that is equal to or less than a predetermined value and that the deviation in position is equal to or less than a predetermined value.

[0043] [Embodiment 2] The information processing apparatus according to Embodiment 1 extracted a human body determined to be alone in the image as a person who might shoplift. On the other hand, in Embodiment 2, when extracting a person who might shoplift, the movement route of the person in the image is considered. For example, by determining a region where the person is not moving as a region where obstacles such as shelves exist, it is considered whether there are obstacles between people. For this purpose, the information processing apparatus according to the present embodiment analyzes the density of the people in the image instead of the single-person determination in Embodiment 1. The information processing apparatus 1200 according to the present embodiment has the same functional units as those shown in FIG. 1 of Embodiment 1 except that it has an analysis unit 1201 instead of the determination unit 108, and redundant descriptions are omitted.

[0044] FIG. 12 is a block diagram showing an example of the functional configuration of the information processing apparatus 1200 according to the present embodiment. The analysis unit 1201 analyzes the density of each person on the image acquired by the acquisition unit 101. The analysis unit 1201 according to the present embodiment performs the above-described analysis process using the position of the human body information on the three-dimensional coordinates converted by the conversion unit 106. For this purpose, the analysis unit 1201 calculates the distance d between each person in the same manner as the process performed by the determination unit 108, and then determines the presence or absence of an obstacle between the people. The analysis unit 1201 calculates the density for each person based on the calculated d and the determination result of the presence or absence of an obstacle, and transmits the calculated density to the transmission unit 109.

[0045] The analysis unit 1201 according to the present embodiment calculates the density based on the number of combinations of people satisfying a predetermined condition. Here, the density is defined as the total number of pairs of people who satisfy the condition that the distance d between the people is equal to or less than a preset threshold value and there is no obstacle between the people. The threshold value used for the determination of the distance d may be the same value as that used in the single determination by the determination unit 108 of Embodiment 1, or an arbitrary value may be set separately. The determination of the presence or absence of an obstacle is performed in the same manner as the process performed by the determination unit 108 of Embodiment 1.

[0046] In addition, the method for calculating the density is not limited to the above-described method, and may be calculated by any method that calculates an index related to the proximity and density of the three-dimensional positions of each person as the density. For example, when the value of the variance of the coordinates of the human body information on the three-dimensional coordinates is equal to or greater than a predetermined value, the analysis unit 1201 may assume that the person within the aggregation range is alone.

[0047] The transmission unit 109 transmits the density of each person to another analysis device or a notification unit (not shown) possessed by the user via the I / F unit 203. The transmission unit 109 according to the present embodiment can transmit a notification to the user so as to make a call as a person who is unlikely to be noticed by other people and who may commit shoplifting because the density value is small.

[0048] According to such processing, it is possible to discriminate those with low density among the human bodies in the image and provide information indicating a person who is acting alone in a system that prompts a call to prevent shoplifting.

[0049] (Other Embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiment to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. Further, it can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0050] The invention is not limited to the above-described embodiment, and various changes and modifications are possible without departing from the spirit and scope of the invention. Therefore, the claims are attached to disclose the scope of the invention.

Description of Reference Numerals

[0051] 100: Information processing apparatus, 101: Acquisition unit, 102: Tracking unit, 103: Management unit, 104: Filter unit, 105: Calculation unit, 106: Conversion unit, 107: Estimation unit, 108: Determination unit, 109: Transmission unit

Claims

1. Acquisition means for acquiring the position and size of at least one object detected from an image in a first coordinate system for indicating the position of the object in the image; Calculation means for calculating a parameter for estimating a position in a second coordinate system different from the first coordinate system from the position in the first coordinate system based on the position and size of the object acquired by the acquisition means; Output means for outputting a position obtained by converting the position of at least one object detected from the image in the first coordinate system to the second coordinate system based on the parameter; An information processing apparatus comprising the same.

2. Analysis means for analyzing the position of the object based on the position of the object in the second coordinate system converted based on the parameter; Notification means for notifying a user based on the result of the analysis of the position of the object; The information processing apparatus according to claim 1, further comprising the same.

3. The analysis means determines, as the analysis of the position of the object, whether there is another object in the vicinity of the object or whether the number of other objects existing in the vicinity of the object is less than or equal to a predetermined value. The information processing apparatus according to claim 2, characterized in that.

4. The information processing apparatus according to claim 2 or 3, characterized in that the notification to the user is to prompt the user to make a call to the object in the vicinity of which there is no other object or the object in the vicinity of which the number of other objects is less than or equal to a predetermined value.

5. Further comprising generation means for generating information indicating the position of an occluder in the image based on the position of the object in the second coordinate system; The analysis means is characterized in that, for the object, an object whose distance between objects is equal to or less than a predetermined value and for which no such shielding object exists between the objects is regarded as another object existing in the vicinity of the object, in the information processing apparatus according to any one of claims 2 to 4.

6. The information indicating the position of the shielding object is information indicating the distribution of the positions of the objects in the second coordinate system for each partial region of the image in the second coordinate system, in the information processing apparatus according to claim 5.

7. The analysis means determines whether or not the shielding object exists between the objects based on the information indicating the distribution of the positions in the partial region through which the line segment connecting the object and another object passes in the second coordinate system, in the information processing apparatus according to claim 6.

8. The apparatus further comprises a selection means for selecting an object satisfying a predetermined condition from among at least one or more objects detected from the image. The output means outputs, based on the position and size of the object selected by the selection means, the position obtained by converting the position of at least one or more objects detected from the image in the first coordinate system to the second coordinate system, in the information processing apparatus according to any one of claims 1 to 7.

9. The selection means selects, from among a plurality of objects in the image, an object for which the reliability of the information indicating the position in the first coordinate system acquired by the acquisition means is equal to or greater than a predetermined value, in the information processing apparatus according to claim 8.

10. The apparatus further comprises a holding means for holding object information detected from images captured at a plurality of times, for the position and size of the objects detected from the image. The selection means selects, based on the object information, an object that satisfies the predetermined condition during a predetermined period from among the objects detected from the image, in the information processing apparatus according to claim 8 or 9.

11. The selection means selects an object whose displacement in position in the first coordinate system during the predetermined period is equal to or less than a predetermined value, and an object whose displacement in size in the first coordinate system during the predetermined period is equal to or less than a predetermined value. The information processing apparatus according to claim 10, characterized in that.

12. The selection means selects an object whose lateral movement amount or longitudinal movement amount in position in the first coordinate system during the predetermined period is equal to or greater than a predetermined value. The information processing apparatus according to claim 10 or 11, characterized in that.

13. The selection means selects an object for which the position in the first coordinate system has been acquired continuously for a predetermined time or longer. The information processing apparatus according to any one of claims 10 to 12, characterized in that.

14. The selection means selects objects whose positions or sizes are different so as to be more than a predetermined number. The information processing apparatus according to any one of claims 8 to 13, characterized in that.

15. The parameter is a parameter for calculating the distance from the position of the subject in the first coordinate system on the image to the subject from the imaging device that captured the image. The information processing apparatus according to any one of claims 1 to 7, characterized in that.

16. The parameter is a parameter that assigns different weights to the respective values of the vertical position and the horizontal position in the first coordinate system. The information processing apparatus according to claim 15, characterized in that.

17. The object is a human body, and the information indicating the position and size in the first coordinate system is the coordinates and size of the head of the human body in the image. The information processing apparatus according to any one of claims 1 to 14, characterized in that.

18. A step of obtaining the position and size of at least one or more objects detected from an image in a first coordinate system for indicating the position of the object in the image; A step of calculating a parameter for estimating a position in a second coordinate system different from the first coordinate system from a position in the first coordinate system based on the position and the size; A step of outputting a position obtained by converting the position in the first coordinate system of at least one object detected from the image into the second coordinate system based on the parameter; An information processing method characterized by comprising the above.

19. A program for causing a computer to function as each means of the information processing apparatus according to any one of Claims 1 to 17.

Images