Method for generating interval grid for estimating proximity

Abstract

A method for generating an interval grid for estimating proximity is provided. A method for generating an interval grid for estimating proximity of objects, according to an embodiment of the present invention, sets a reference point in an image, and generates mark points that constitute an interval grid of the image on the basis of a ratio of an actual size of a measurer located on a reference line connecting the set reference point and a screen center to a size on an image screen. Accordingly, by generating an interval grid in the image scene by using only a single camera, it is possible to estimate the distance or proximity between people appearing in an image scene through a simple numerical comparison without complex calculation.

Description

A method for generating interval grids for proximity estimation

[0001] The present invention relates to a distance estimation technique based on image processing, and more particularly, to a method for generating an interval grid for estimating the positions of people appearing in an image captured by a single camera and their proximity to each other.

[0002] Automated human behavioral recognition can be applied to a wide range of fields, including security, public safety, caregiving, sports judging, and gaming. By enabling real-time monitoring of criminal activities like theft and violence through behavioral recognition of human interactions, various social problems can be addressed.

[0003] With the advancement of artificial intelligence, behavior recognition technology through image analysis has also made significant progress. However, automated behavior recognition requires high-performance systems. Recognizing human interactions, in particular, demands even greater computational resources. If human interactions can be recognized in a lightweight computing environment requiring fewer resources, popularization of the system is expected to facilitate widespread provision of social safety services.

[0004] Typically, for action recognition, video streams are analyzed sequentially, frame by frame. For interactive action recognition, analyzing every frame of the video, rather than only when characters enter a distance or area where interaction is possible, can reduce the computational burden. However, determining whether interaction is possible requires specialized distance measurement devices and additional processing, potentially increasing the system's cost. Therefore, a cost-effective method for determining the distance of characters in a video is needed.

[0005] The present invention has been devised to solve the above problems, and an object of the present invention is to provide a method for generating a gap grid in a video scene captured using a single camera and estimating the distance or proximity between people appearing in the scene.

[0006] In order to achieve the above object, a method for generating an interval grid for estimating the proximity of objects according to one embodiment of the present invention includes the steps of: setting a reference point in an image; and generating mark points that constitute an interval grid of an image based on a ratio of an actual size of a measuring instrument located on a reference line connecting the set reference point and the center of the screen to a size on the image screen.

[0007] The reference point setting step may include: a step of detecting lines within an image; a step of removing lines that are vertical or horizontal to a horizontal line among the detected lines; and a step of finding the intersection point of the remaining lines to set a vanishing point.

[0008] The calculation step may include a step of detecting a bounding box of an observer appearing in an image; a step of calculating a ratio of an actual size at the observer's position and an image screen size based on the height of the observer and the height of the detected bounding box;

[0009] The detection step may be to detect a bounding box that contains all body features detected from the observer.

[0010] The step of generating a marker point may include: a step of obtaining a marker line by connecting a reference point and a center point at the bottom of the screen; a step of detecting a measuring object and obtaining a bounding box; a step of generating a horizontal line passing through a point where the measuring object is located on the reference line; a step of calculating a ratio between an image pixel and an actual measurement unit at the position of the measuring object based on the height of the bounding box and the height of the measuring object; and a step of generating marker points by dividing the horizontal line into predetermined lengths centered on the reference line.

[0011] The bounding box acquisition step may be to detect a bounding box that includes all body feature points detected from the observer, and the horizontal line generation step may be to generate a horizontal line passing through feature points on the sole or ankle side of the measurer.

[0012] The step of generating a marker point may further include: a step of detecting a measuring object moving along a reference line to obtain a bounding box; a step of generating a horizontal line passing through a point where the measuring object is located on the reference line; a step of calculating a ratio between an image pixel and an actual measurement unit at the position of the measuring object based on the height of the bounding box and the height of the measuring object; and a step of generating marker points by dividing the horizontal line into predetermined lengths centered on the reference line.

[0013] The step of generating marker points may include a step of dividing marker points into vertical groups centered on a reference point; and a step of generating a line closest to all marker points belonging to the vertical group using linear regression when there are three or more points belonging to the same vertical group.

[0014] The step of generating a marker point may further include a step of generating an interval grid by linear interpolation for the intersections of the adjusted vertical lines and the horizontal lines.

[0015] According to another aspect of the present invention, an image system is provided, characterized by including: a communication unit for acquiring an image; a processor for setting a reference point in the acquired image and generating mark points constituting an interval grid of the image based on a ratio of an actual size of a measuring device located on a reference line connecting the set reference point and the center of the screen to a size on the image screen;

[0016] According to another aspect of the present invention, a method for estimating proximity of objects is provided, comprising: a step of generating marker points that constitute an interval grid of an image based on a ratio of an actual size of a measurer appearing in an image to a size on an image screen; a step of generating an interval grid of an image based on the generated marker points; and a step of estimating a proximity between objects based on positions of grids where objects are located in the generated interval grid.

[0017] According to another aspect of the present invention, an image processing system is provided, characterized by comprising: a step of acquiring an image; a processor that generates mark points to constitute an interval grid of the image based on a ratio of an actual size of a measuring object appearing in the acquired image and a size on an image screen, generates an interval grid of the image based on the generated mark points, and estimates a proximity between objects based on positions of grids where objects are located in the generated interval grid.

[0018] As described above, according to embodiments of the present invention, by creating a gap grid in a video scene using only a single camera, it is possible to estimate the distance or proximity between people appearing in a video scene through simple numerical comparison without complex calculations.

[0019] In addition, according to embodiments of the present invention, applications such as interaction recognition can be performed cost-effectively based on the distance or proximity between people estimated in a video scene.

[0020] Figure 1. Interval grid generation process

[0021] Figure 2. Example scenario for creating a gap grid.

[0022] Figure 3. Creating a reference point based on a vanishing point

[0023] Figure 4. Camera parameter measurement method

[0024] Figure 5. Example of distance measurement

[0025] Figure 6-8. Example of creating a marker point

[0026] Figure 9-10. Creating a grid using marker points.

[0027] Figure 11. Example of estimating user location and proximity between users using a spacer grid.

[0028] Fig. 12. Image processing system

[0029] Hereinafter, the present invention will be described in more detail with reference to the drawings.

[0030] In an embodiment of the present invention, a method for generating an interval grid for proximity estimation and a method for estimating proximity using an estimated interval grid are proposed.

[0031] FIG. 1 is a diagram illustrating a process of a method for generating a grid according to one embodiment of the present invention. As illustrated, the method for generating a grid according to one embodiment of the present invention comprises a reference point generating step (S110), a camera parameter measuring step (S120), and a marker point generating step (S130).

[0032] The reference point generation step (S110) is a process of establishing a reference point that serves as a measurement reference in a video scene. In the embodiment of the present invention, a vanishing point is used as the reference point.

[0033] The camera parameter measurement step (S120) calculates the ratio (cm: pixel ratio) of the actual measurement unit (e.g., cm) and the screen size (pixel) by measuring the distance between the camera and a person (measuring person).

[0034] The mark point generation step (S130) generates mark points, which are basic components of an interval grid, using object recognition and body feature points, and connects the generated mark points to generate an interval grid.

[0035] Figure 2 illustrates an example scenario for creating a gap grid. After installing the camera, the measurer positions himself in front of the camera to create the gap grid. The measurer holds a device (e.g., smartphone, tablet, etc.) capable of receiving the camera's image so that he or she can view the camera's scene and his or her own position. The camera's scene and the measurer's position can also be confirmed using a monitor or other device.

[0036] Figure 3 illustrates the reference point generation step (S110). A vanishing point is a point where parallel lines in physical space appear to meet at a single point in the image due to perspective. The vanishing point serves as a reference point for measuring horizontal distance. To determine the vanishing point, all lines in the image are first identified using an edge detection technique. Based on the angle between the detected lines and the horizon, lines that are completely vertical or completely horizontal are removed. Finally, the intersection of the remaining lines after filtering is calculated to find the vanishing point.

[0037] Fig. 4 shows the camera parameter measurement step (S120). In Fig. 4, h represents the height of a person, f represents the camera focal length, d represents the distance between the camera and the person, and a represents the height of the person in the image. b represents the height of the person in the image generated when the person moves toward the camera by a distance of m. At this time, the relationships a / f = h / d and b / f = h / (dm) are established, and using these two formulas, the distance d can be obtained as shown in the formula below.

[0038]

[0039] Figure 5 illustrates an example of distance measurement. The subject enters his or her height as the value h. A bounding box is derived using a human object detection algorithm, and body keypoint detection is also performed to ensure that the entire body, not just a portion of the body, is captured. The bounding box provides values ​​for a and b, which can be used to calculate the distance d between the subject and the camera.

[0040] Figures 6 to 8 illustrate the display point generation step (S130). To generate the display point, as illustrated in Figure 6, a reference line is obtained by connecting the previously obtained reference point (vanishing point) and the center point of the screen, and is displayed on the screen.

[0041] The measurement is performed while the next measurer moves along the reference line. First, as shown in Fig. 6, the measurer moves along the reference line so that the whole body is displayed at about the center of the screen. After the movement is completed, object detection and body feature point detection are performed. A horizontal line is obtained based on the feature point on the sole or ankle. Using the height information of the bounding box obtained from the object detection and the actual person's height information, the ratio between the image pixel and the actual measurement unit at the current position is obtained. The horizontal line is divided into specific units (e.g., 50 cm) centered on the reference line and a mark point is generated.

[0042] In the second measurement, as illustrated in Figure 7, the measurer moves as far as possible along the reference line within a range where object detection and body feature detection are possible. After completing the movement, a horizontal line is obtained based on the feature points on the sole or ankle. Then, using the height information of the bounding box obtained from the object detection and the actual human height information, the ratio between the image pixels and the actual measurement units at the current location is calculated. The horizontal line is divided into units set during the first mark point generation, centered on the reference line, and mark points are generated.

[0043] To improve accuracy, markers can be repeated multiple times after the second marker is generated. For example, as shown in Figure 8, the measurer moves along the reference line again, and after completing the movement, a horizontal line is obtained based on the feature points on the sole or ankle. Using the height information of the bounding box obtained from object detection and the actual height information of the person, the ratio between the image pixels and the actual measurement unit at the current location is calculated. The horizontal line is divided into units set during the first marker generation, centered on the reference line, and markers are then generated.

[0044] Figures 9 and 10 illustrate the process of generating a grid. To generate a grid from the previously generated marker points, the marker points are divided into vertical groups centered on the reference point in Figure 9, and if there are three or more points in a group, linear regression is used to find the line that most closely matches all marker points in that group.

[0045] Linear interpolation is applied to the intersections of the adjusted vertical lines and the previously generated horizontal lines to create an interval grid as in Fig. 10.

[0046] Figure 11 shows an example of estimating user locations and proximity between users using the generated interval grid. By comparing the locations of feature points on the soles or ankles of people appearing on the screen with the constituent points of the interval grid, the user locations and distances between them can be estimated. For example, the distance between 'User-1' and 'User-2' is estimated as the distance between 'the center position of the grid where User-1's ankle feature point is located' and 'the center position of the grid where User-2's ankle feature point is located.'

[0047] FIG. 12 is a diagram illustrating the configuration of an image processing system according to another embodiment of the present invention. As illustrated, the image processing system according to the embodiment of the present invention can be implemented as a computing system comprising a communication unit (210), an output unit (220), a processor (230), an input unit (240), and a storage unit (250).

[0048] The communication unit (210) is a communication interface for connection with an external network or external device, and receives images from the camera. The output unit (220) is an output means for displaying the results of calculations performed by the processor (230), and the input unit (240) is a user interface for receiving user commands and transmitting them to the processor (230).

[0049] The processor (230) generates an interval grid of an image according to the procedure illustrated in the aforementioned FIG. 1, estimates the distance or proximity between users appearing in the image based on the generated interval grid, and performs action recognition, interaction recognition, and other application services based on the estimated distance or proximity.

[0050] The storage unit (250) provides the storage space necessary for the processor (230) to function and operate.

[0051] So far, a preferred embodiment of a method for generating an interval grid for proximity estimation has been described in detail.

[0052] In the above embodiment, a gap grid is created in a video scene captured using a single camera, and the distance or proximity between people appearing in the scene is estimated, so that subsequent functions can be performed based on this.

[0053] Meanwhile, it goes without saying that the technical idea of ​​the present invention can also be applied to a computer-readable recording medium containing a computer program that performs the functions of the device and method according to the present embodiment. In addition, the technical idea according to various embodiments of the present invention can be implemented in the form of computer-readable code recorded on a computer-readable recording medium. The computer-readable recording medium can be any data storage device that can be read by a computer and store data. For example, the computer-readable recording medium can be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, etc. In addition, the computer-readable code or program stored on the computer-readable recording medium can be transmitted through a network connected between computers.

[0054] In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications can be made by a person having ordinary skill in the art to which the present invention pertains without departing from the gist of the present invention as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present invention.

Claims

1. Step of setting the reference point in the video; A method for generating an interval grid for estimating the proximity of objects, comprising: a step of generating marker points that constitute an interval grid of an image based on a ratio of an actual size of a measuring instrument positioned on a reference line connecting a set reference point and the center of the screen to an image screen size; 2. In claim 1, The baseline setting step is, A step of detecting lines within an image; A step of removing horizontal and vertical lines from among the detected lines; A method for generating an interval grid for estimating the proximity of objects, characterized by including a step of finding the intersection points of the remaining lines and setting a vanishing point.

3. In claim 1, The calculation steps are: A step of detecting the bounding box of an observer appearing in the video; A method for generating an interval grid for estimating the proximity of objects, characterized by including a step of calculating a ratio of an actual size to an image screen size at an observer position based on the height of the observer and the height of the detected bounding box.

4. In claim 3, The detection step is, A method for generating an interval grid for proximity estimation of objects, characterized by detecting a bounding box that includes all body feature points detected from an observer.

5. In claim 1, The steps to create a marker are: A step for obtaining a reference line by connecting the reference point and the center point at the bottom of the screen; A step of detecting a measurer and obtaining a bounding box; A step of creating a horizontal line passing through the point where the measuring instrument is located on the reference line; A step of calculating the ratio between an image pixel and an actual measurement unit at the position of the measurer, based on the height of the bounding box and the height of the measurer; A method for generating an interval grid for estimating the proximity of objects, characterized by including a step of generating indicator points by dividing a horizontal line into fixed lengths centered on a reference line.

6. In claim 5, The bounding box acquisition step is: Detect a bounding box that contains all body features detected by the observer, The horizon generation step is: A method for generating a grid for proximity estimation of objects, characterized by generating a horizontal line passing through a feature point on the sole or ankle side of a measurer.

7. In claim 5, The steps for creating a marker are: A step of detecting a measurer that has moved along a reference line and obtaining a bounding box; A step of creating a horizontal line passing through the point where the measuring instrument is located on the reference line; A step of calculating the ratio between an image pixel and an actual measurement unit at the position of the measurer, based on the height of the bounding box and the height of the measurer; A method for generating an interval grid for estimating the proximity of objects, characterized in that it further includes a step of generating indicator points by dividing a horizontal line into fixed lengths centered on a reference line.

8. In claim 5, The steps for creating a marker are: A step of dividing the marker points into vertical groups centered on the reference point; A method for generating an interval grid for estimating the proximity of objects, characterized in that it comprises the step of generating a line closest to all the marked points belonging to the vertical group using linear regression when there are three or more points belonging to the same vertical group.

9. In claim 8, The steps for creating a marker are: A method for generating an interval grid for proximity estimation of objects, characterized in that it further includes a step of generating an interval grid by linear interpolation for intersections of adjusted vertical lines and horizontal lines.

10. Communication unit for acquiring images; An image system characterized by including a processor for setting a reference point in an acquired image and generating mark points that constitute an interval grid of an image based on a ratio of an actual size of a measuring instrument located on a reference line connecting the set reference point and the center of the screen to a size on the image screen.

11. A step of generating marking points that constitute the interval grid of the image based on the ratio of the actual size of the measuring person appearing in the image and the size on the image screen; A step of generating an image grid based on the generated marker points; A method for estimating proximity of objects, characterized by including a step of estimating proximity between objects based on positions of grids in which objects are located in a generated interval grid.

12. Step of acquiring images; An image processing system characterized by including a processor for generating mark points that constitute an interval grid of an image based on a ratio of an actual size of a measurer appearing in an acquired image and a size on an image screen, generating an interval grid of the image based on the generated mark points, and estimating proximity between objects based on positions of grids where objects are located in the generated interval grid.

Images