Coordinate mapping method and positioning method and system for infrared light gun

Abstract

The invention relates to the technical field of image processing and coordinate mapping, in particular to a coordinate mapping method for an infrared light gun and a positioning method and system. According to the coordinate mapping method for the infrared light gun, the five light beads are arranged on the screen, the clockwise sequence of the shot light beads on the shot image is determined through vector cross product, and then the corresponding relation between the shot light beads and the real light beads on the screen is determined according to the collinear relation of the shot light beads. Therefore, the problem of screen deflection on the shot image can be effectively identified, the influence of the deflection angle on the identification precision is avoided, and the accurate analysis of the affine transformation matrix is ensured.

Description

Technical Field

[0001] The invention relates to the technical field of image processing and coordinate mapping, in particular to a coordinate mapping method and a positioning method and system for an infrared light gun. Background Art

[0002] The infrared light gun shoots the screen through the camera and analyzes the corresponding position of the camera image center on the screen as the mouse position. The mouse position is analyzed in combination with the light beads set on the screen.

[0003] At present, most infrared light guns use the GUN4IR system, which sets 4 Figure 1 The light beads are distributed in a rectangular shape. The GUN4IR system has the following disadvantages:

[0004] 1. The rotation angle of the light gun, i.e. the camera, is only about 30 degrees. Beyond this angle, the positioning error is extremely large.

[0005] 2. The time gun must be pointed upwards when it is started, otherwise it cannot be positioned;

[0006] 3. Relying on DIY hardware such as microcontrollers and mouse switches, the system cost is high.

[0007] In summary, the GUN4IR system cannot identify the rotation angle of the light bead positioning rectangle and can only preset the relative position between the camera and the screen. Once the relative position is deflected, the light spot cannot be accurately positioned, which seriously affects the user experience. Summary of the invention

[0008] In order to overcome the defects of the GUN4IR system in the above-mentioned prior art that it cannot recognize the rotation angle and has low image positioning accuracy, the present invention proposes a coordinate mapping method for an infrared light gun, which can recognize the relative rotation angle between the camera image and the screen, and locate the light bead in combination with the rotation angle, thereby greatly improving the accuracy of screen positioning; and this method does not rely on hardware, which is conducive to reducing costs.

[0009] The present invention proposes a coordinate mapping method for an infrared light gun. First, five light beads are set on the screen, wherein four light beads are arranged at four corner points of a rectangle, and the last light bead is arranged on any side of the rectangle; the captured image is analyzed to obtain the camera world coordinates of each light bead, the arrangement order in the clockwise direction, and the collinear relationship; the correspondence between the camera world coordinates and the real world coordinates of the light bead is obtained, and the affine transformation matrix is ​​analyzed.

[0010] Preferably, the camera world coordinates of each light bead are obtained by binarizing the captured image to generate a black-and-white image; selecting the connected areas corresponding to the light beads from the black-and-white image, extracting the center points of each connected area as the captured light beads, normalizing the coordinates of the captured light beads on the captured image, and using the obtained normalized coordinate values ​​as the camera world coordinates of the captured light beads.

[0011] Preferably, the method for obtaining the arrangement order and collinear relationship of the shooting light beads in the clockwise direction is: randomly select one of the five shooting light beads as the reference point A, construct a vector from point A to the remaining four shooting light beads, and determine the clockwise order of the five shooting light beads according to the vector cross product; in the clockwise direction, connect two adjacent shooting light beads in sequence to construct an edge vector, and calculate the cross product of adjacent edge vectors; and determine whether the three shooting light beads corresponding to the adjacent edge vectors corresponding to the minimum cross product are collinear.

[0012] Preferably, reference point A selects the shooting light bead with the smallest y-axis coordinate of the camera world coordinate.

[0013] Preferably, the captured image is first subjected to Gaussian blur processing and then to binarization processing to obtain a black and white image; the method for screening the connected areas corresponding to the light beads from the black and white image is: setting and adjusting screening conditions for one or more of the number of pixels in the connected area, the circular similarity, and the aspect ratio of the circumscribed rectangle, until 5 connected areas are screened out as the connected areas corresponding to the light beads.

[0014] Preferably, the screening conditions are set as follows: the number of pixels contained in the connected area is in the interval [200, 1500], the aspect ratio of the circumscribed rectangle of the connected area is in the interval [0.5, 1.5], and the circle similarity of the connected area is greater than or equal to 0.5.

[0015] An infrared light gun positioning method proposed by the present invention first uses the infrared light gun positioning method to obtain a two-dimensional affine transformation matrix, substitutes the camera world coordinates (0.5, 0.5) of the center point of the camera screen into the required two-dimensional affine transformation matrix to obtain the normalized coordinate value of the center point of the camera screen in the real world; then multiplies the normalized coordinate value of the center point of the camera screen by the screen resolution to obtain the screen mouse position in the real world coordinates.

[0016] Preferably, after obtaining the screen mouse position in the real world coordinates, the screen mouse position is smoothed by using a mean filter, and then the mouse is driven to move to the smoothed screen mouse position.

[0017] An infrared light gun positioning system proposed in the present invention includes a memory and a processor, wherein a computer program is stored in the memory, and the processor is connected to the memory, and the processor is used to execute the computer program to implement the infrared light gun positioning method.

[0018] Preferably, it also includes a camera and a screen; the light beads arranged on the screen are infrared lamp beads, and the camera is an infrared camera.

[0019] The advantages of the present invention are:

[0020] (1) The coordinate mapping method for the infrared light gun proposed in this application sets 5 light beads on the screen, determines the clockwise order of the light beads on the captured image by vector cross product, and then determines the correspondence between the light beads and the real light beads on the screen according to the collinearity of the light beads. This can effectively identify the problem of screen deflection on the captured image, avoid the influence of the deflection angle on the recognition accuracy, and ensure the accurate analysis of the affine transformation matrix.

[0021] (2) In this application, the position of the shooting light bead on the binary black and white image is determined by screening the connected area and extracting the center point, which ensures the efficient and accurate positioning of the shooting light bead. Then the coordinate position of the shooting light bead on the shooting image is normalized, and the obtained world coordinates of the shooting light bead camera are always between [0,1]; the calculation efficiency and accuracy are greatly improved by the normalized value calculation.

[0022] (3) In this application, the cross product is used to determine the clockwise direction and collinear relationship of the light beads in combination with the world coordinates of the camera that photographs the light beads. This calculation method completely relies on the program and does not require external equipment, which reduces the method's dependence on hardware, increases its scope of application, and saves hardware costs.

[0023] (4) In the present application, by selecting the coordinates of the reference point A, the efficiency of determining the correspondence between the photographed light bead and the real light bead can be further improved.

[0024] (5) The infrared light gun positioning method proposed in this application combines the known two-dimensional affine transformation relationship to map the normalized coordinates (0.5, 0.5) of the center point of the camera image to the normalized value of the real-world coordinates, and then calculates the screen mouse position in the real-world coordinates in combination with the screen resolution, which greatly improves the calculation efficiency and accuracy.

[0025] (6) In the present application, the mouse movement is driven according to the smoothing value of the screen mouse position, which improves the continuity of the mouse movement and is conducive to improving the user experience.

[0026] (7) The infrared light gun positioning system proposed in this application is implemented by combining infrared light beads and infrared cameras. The storage device as a carrier of the positioning method can be plug-and-play without the need for hardware support, which greatly improves the system's applicable scenarios and positioning accuracy; in particular, it improves the compatibility and user experience of the positioning method when applied to various shooting games such as PCs, consoles, and arcades. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Arrange the positions of the light beads on the screen of the GUN4IR system;

[0028] Figure 2 A flow chart of a coordinate mapping method for an infrared light gun proposed in this application;

[0029] Figure 3 This is the arrangement diagram of the light beads in this application;

[0030] Figure 4 The result of the rectangular deflection on the screen captured by the camera;

[0031] Figure 5 This is a flow chart of an infrared light gun positioning method proposed in this application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] Reference Figure 2 The coordinate mapping method for the infrared light gun proposed in this application includes the following steps S1-S6.

[0034] S1. First, five light beads are set on the screen, four of which are arranged at the four corners of the rectangle, and the last light bead is arranged on any side of the rectangle, specifically, it can be set at the midpoint of the top side, refer to Figure 3 shown.

[0035] S2. Obtain an image of the screen captured by the camera on the infrared light gun, and extract the normalized coordinate values ​​of the five light beads from the captured image.

[0036] This step S2 specifically includes the following sub-steps:

[0037] S21. First, Gaussian blur is performed on the captured image to reduce the influence of high-frequency noise.

[0038] S22, then grayscale the captured image and perform binary processing to generate a black and white image.

[0039] It is worth noting that, in actual implementation, step S22 may not be executed directly.

[0040] S23, find all connected areas on the black and white image;

[0041] S24, setting filtering conditions, and screening connected areas that meet the filtering conditions as legal areas;

[0042] In specific implementation, the filtering conditions can limit the number of pixels, circle similarity and aspect ratio of the connected area; for example: the number of pixels contained in the connected area is in the interval [200,1500], the aspect ratio of the circumscribed rectangle of the connected area is in the interval [0.5,1.5], the circle similarity of the connected area is greater than or equal to 0.5, etc.

[0043] S25. Determine whether the number of legal areas is 5; if not, return to step S24; if yes, extract the center point of each legal area as the shooting coordinate point (xn, yn) of the shooting light bead; 1≤n≤5.

[0044] In specific implementation, an approximate polygon algorithm can be used to extract the center point of the approximate polygon of the legal area as the shooting coordinate point of the shooting light bead. The shooting light bead is the light bead on the shooting image, and the shooting coordinate point is the pixel coordinate of the shooting light bead on the shooting image.

[0045] It is worth noting that at this time, the shooting coordinate points of each shooting light bead can be obtained, but the corresponding relationship between the shooting light bead and the light bead on the screen is unknown.

[0046] S26, normalizing the shooting coordinate points of each shooting light bead to obtain the normalized coordinate value of each shooting light bead as the camera world coordinate.

[0047] The normalized coordinate values ​​are calculated as:

[0048] x'n=xn / w

[0049] y'n=yn / h

[0050] Among them, w is the number of pixels on the screen width of the captured image, h is the number of pixels on the screen height of the captured image; xn is the pixel coordinate of the center point of the nth legal area in the screen width direction, yn is the pixel coordinate of the center point of the nth legal area in the screen height direction, that is, (xn,yn) is the coordinate of the nth shooting light bead in the captured image; x'n is the normalized x-axis coordinate of the legal area, y'n is the normalized y-axis coordinate of the legal area, that is, (x'n,y'n) is the camera world coordinate of the nth shooting light bead.

[0051] S3. Randomly select one of the five shooting light beads as the reference point A, construct a vector from point A to the other four shooting light beads, and determine the clockwise order of the five shooting light beads based on the vector cross product.

[0052] Specifically, assume that the remaining captured light beads are recorded as points BCDE; then construct vectors AB / AC / AD / AE; calculate the cross product of any two vectors. If the cross product of vectors AB and AC is positive, it means that vector AB is in the clockwise direction of vector AC, that is, B is in front of A in the clockwise direction, such as Figure 4 As shown; if the cross product of vectors AB and AC is negative, it means that vector AB is in the counterclockwise direction of vector AC, that is, C comes before B in the clockwise direction; if the cross product of vectors AB and AC is 0, it is judged that ABC is collinear.

[0053] S4. According to the positions of points ABCDE in the clockwise direction, connect two adjacent points in sequence to construct edge vectors, and calculate the cross product of adjacent edge vectors; obtain the three points corresponding to the adjacent edge vectors whose cross product is 0 to determine whether the three are collinear; if there is no adjacent edge vector whose cross product is 0, obtain the three points corresponding to the adjacent edge vector corresponding to the minimum cross product to determine whether the three are collinear.

[0054] Figure 4 The shooting light bead with the smallest y-axis coordinate is used as the reference point A, and the shooting light beads are arranged in a clockwise order according to ABCDE. The edge vectors include: AB, BC, CD, DE, EA; by calculating the cross product of adjacent edge vectors, it can be known that CD*DE=0, so it is determined that the three points CDE are collinear.

[0055] In this way, the three collinear captured light beads are known and the order of the five captured light beads in the clockwise direction is known.

[0056] S5, recording the collinear connecting line of the three captured light beads as an anchoring edge, determining the deflection direction of the light bead positioning rectangle on the captured image according to the vector position of the anchoring edge, and determining the corresponding relationship between each captured light bead and the placement position of the light bead in the real world in combination with the deflection direction and the order of the captured light beads;

[0057] S6. Determine the real world coordinates of the light beads according to their arrangement positions on the screen, and determine the two-dimensional affine transformation matrix according to the camera world coordinates and the real world coordinates of the light beads.

[0058] In the real world, the real world coordinates of the light bead at the upper left position of the rectangle are (0,0), the real world coordinates of the light bead at the upper right position of the rectangle are (1,0), the real world coordinates of the light bead at the lower left position of the rectangle are (0,1), and the real world coordinates of the light bead at the lower right position of the rectangle are (1,1);

[0059] The last light bead can determine the real-world coordinates based on its set position, e.g. Figure 4 In the illustrated embodiment, the fifth light bead is located in the upper middle position, and its real world coordinates can be determined to be (0.5, 0).

[0060] Specifically, in step S6, the camera world coordinates and real world coordinates of at least four light beads can be substituted into the two-dimensional affine transformation matrix to construct the following affine transformation equation, and then the least squares method is used to solve the affine transformation equation.

[0061] The affine transformation equation is:

[0062] a*P.x+b*P.y+tx=P'.x

[0063] c*P.x+d*P.y+ty=P'.y

[0064] Among them, a, b, c, d, tx, and ty are the parameters to be solved of the affine transformation equation, and are also the elements of the two-dimensional affine transformation matrix; P'.x is the camera world x-axis coordinate of the light bead, P'.y is the camera world y-axis coordinate of the light bead, Px is the real world x-axis coordinate of the light bead, and Py is the real world y-axis coordinate of the light bead.

[0065] Thus, in step S5, the anchor edge is determined to be CDE, which corresponds to the top edge of the rectangle determined by the light bead on the curtain. Figure 3 and Figure 4 , it can be determined that the camera image is deflected clockwise relative to the real curtain. Point C corresponds to the light bead at the upper left position of the screen, point D corresponds to the light bead at the upper middle position, point E corresponds to the light bead at the upper right position of the screen, point A corresponds to the light bead at the lower right position, and point B corresponds to the light bead at the lower left position. Substituting the normalized coordinates of points ABCDE and the real world coordinates of the corresponding light bead into the affine transformation equation, the affine transformation matrix can be solved.

[0066] Reference Figure 5 , an infrared light gun positioning method proposed in this application includes the following steps St1-St3.

[0067] St1. Substitute the camera world coordinates (0.5, 0.5) of the center point of the camera image into the required two-dimensional affine transformation matrix to obtain the normalized coordinate value of the center point of the camera image in the real world.

[0068] St2. Multiply the screen resolution by the normalized coordinate value of the center point of the camera image to obtain the screen mouse position in the real world coordinates.

[0069] St3, use mean filtering to smooth the screen mouse position to make its movement more coherent; then use the SetCursorPos function to move the mouse to the final calculated position, that is, the screen mouse position after smoothing.

[0070] The infrared light gun positioning system proposed in this application is based on the Figure 3An infrared light bead is set, and then the light gun uses an infrared camera. The system is provided with a memory for storing the above-mentioned infrared light gun coordinate mapping method and infrared light gun positioning method, and a processor for driving the program in the memory. In this way, the system can use the above-mentioned infrared light gun positioning method to determine the position of the screen mouse and drive it, greatly improving the accuracy of mouse positioning and driving. Moreover, in the present application, the computer program for executing the infrared light gun coordinate mapping method and the infrared light gun positioning method can be set on a plug-and-play storage device, thereby achieving more flexible use.

[0071] Of course, it is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, but also includes the same or similar structures that can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any reference numerals in the claims should not be regarded as limiting the claims involved.

[0072] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

[0073] The techniques, shapes, and structural parts not described in detail in the present invention are all well-known techniques.

Claims

1. A coordinate mapping method for an infrared light gun, characterized in that: First, five light beads are set on the screen, four of which are arranged at the four corners of a rectangle, and the last light bead is arranged on any side of the rectangle; the captured image is analyzed to obtain the camera world coordinates of each light bead, the arrangement order in the clockwise direction, and the collinear relationship; the correspondence between the camera world coordinates and the real world coordinates of the light beads is obtained, and the affine transformation matrix is ​​analyzed.

2. The coordinate mapping method for an infrared light gun according to claim 1, characterized in that: The camera world coordinates of each light bead are obtained by binarizing the captured image to generate a black-and-white image; selecting the connected areas of the corresponding light beads from the black-and-white image, extracting the center points of each connected area as the captured light beads, normalizing the coordinates of the captured light beads on the captured image, and using the obtained normalized coordinate values ​​as the camera world coordinates of the captured light beads.

3. The coordinate mapping method for an infrared light gun as claimed in claim 2, characterized in that: The method for obtaining the arrangement order and collinear relationship of the shooting light beads in the clockwise direction is as follows: randomly select one of the five shooting light beads as the reference point A, construct a vector from point A to the other four shooting light beads, and determine the clockwise order of the five shooting light beads according to the vector cross product; in the clockwise direction, connect two adjacent shooting light beads in sequence to construct an edge vector, and calculate the cross product of adjacent edge vectors; and determine whether the three shooting light beads corresponding to the adjacent edge vectors corresponding to the minimum cross product are collinear.

4. The coordinate mapping method for an infrared light gun as claimed in claim 3, characterized in that: Reference point A selects the shooting light bead with the smallest y-axis coordinate of the camera world coordinate.

5. The coordinate mapping method for an infrared light gun as claimed in claim 2, 3 or 4, characterized in that: The captured image is first subjected to Gaussian blur processing and then to binarization processing to obtain a black-and-white image; the method of screening the connected area corresponding to the light bead from the black-and-white image is: setting and adjusting screening conditions for one or more of the number of pixels in the connected area, the circular similarity, and the aspect ratio of the circumscribed rectangle until 5 connected areas are screened out as the connected areas corresponding to the light bead.

6. The coordinate mapping method for an infrared light gun as claimed in claim 5, characterized in that: The filtering conditions are set as follows: the number of pixels contained in the connected area is in the interval [200, 1500], the aspect ratio of the circumscribed rectangle of the connected area is in the interval [0.5, 1.5], and the circle similarity of the connected area is greater than or equal to 0.

5.

7. A positioning method using an infrared light gun as described in any one of claims 1 to 6, characterized in that: First, the infrared light gun positioning method as described in any one of claims 1 to 6 is used to obtain a two-dimensional affine transformation matrix, and the camera world coordinates (0.5, 0.5) of the center point of the camera screen are substituted into the required two-dimensional affine transformation matrix to obtain the normalized coordinate value of the center point of the camera screen in the real world; then the screen resolution is multiplied by the normalized coordinate value of the center point of the camera screen to obtain the screen mouse position in the real world coordinates.

8. The infrared light gun positioning method according to claim 1, characterized in that: After obtaining the screen mouse position in the real world coordinates, the screen mouse position is smoothed by using a mean filter, and then the mouse is driven to move to the smoothed screen mouse position.

9. An infrared light gun positioning system, characterized in that: It comprises a memory and a processor, the memory stores a computer program, the processor is connected to the memory, and the processor is used to execute the computer program to implement the infrared light gun positioning method as claimed in claim 7 or 8.

10. The infrared light gun positioning system according to claim 9, characterized in that: It also includes a camera and a screen; the light beads set on the screen adopt infrared lamp beads, and the camera adopts an infrared camera.

Images