A cursor mapping method and system

Abstract

This invention discloses a cursor mapping method and system, relating to the fields of human-computer interaction and spatial positioning technology. The method includes: acquiring an image sequence of a target interaction area, the sequence containing at least two mutually separated marker units; real-time identification of the marker units in the image sequence, extracting geometric parameters, and determining the axis of the coordinate system and calculating the pixel density and origin accordingly; obtaining the pixel size of the control area based on the pixel density and preset physical parameters of the control area, and generating the control area in real-time by combining the axis of the coordinate system and the origin; converting the image coordinates of the target interaction point into coordinates within the control area based on the axis of the coordinate system, the origin, and the pixel size of the control area, and mapping them to screen coordinates. This invention establishes a unique two-dimensional planar coordinate system through at least two mutually separated marker units, achieving precise cursor mapping, limiting offsets caused by camera module rotation, exhibiting strong anti-drift capabilities, and applicable to fields such as head-mounted interaction and AR / VR positioning.

Description

Technical Field

[0001] This invention relates to a cursor mapping method and system, belonging to the field of human-computer interaction and spatial positioning technology. Background Technology

[0002] In vision-based spatial interaction systems, camera devices (especially head-mounted camera modules) undergo minute displacements and deflections in response to user movements. Relying solely on inertial sensors or pure image feature tracking makes it difficult to establish a stable and repeatable spatial reference frame, leading to cursor drift and inaccurate interaction. Existing technologies include: positioning based on a single reflective point, which only provides positional information and cannot provide accurate coordinate axis and scaling ratio; L-shaped / T-shaped marker positioning, which can determine the plane and orientation, requires a large size and is unsuitable for space-constrained scenarios or those with high aesthetic requirements; and positioning schemes based on inertial sensor fusion suffer from cumulative errors, resulting in high system cost and power consumption. Summary of the Invention

[0003] The purpose of this invention is to provide a cursor mapping method and system that, by identifying at least two separate marker units in an image sequence, determines a unique two-dimensional plane coordinate system, thereby achieving a precise and robust mapping from image coordinates to screen coordinates.

[0004] To achieve the above objectives, the present invention is implemented using the following technical solution.

[0005] On one hand, the present invention provides a cursor mapping method, comprising:

[0006] Acquire an image sequence of the target interaction area, wherein the image sequence contains at least two mutually separate marker units;

[0007] Real-time identification of marker units in image sequences and extraction of geometric parameters of each marker unit;

[0008] Based on the geometric parameters of each marker unit, the axis of the coordinate system is determined and the pixel density and origin are calculated;

[0009] Based on the pixel density and preset physical parameters of the control area, the pixel size of the control area is obtained, and the control area is generated in real time by combining the axis and origin of the coordinate system.

[0010] Based on the coordinate system's axis, origin, and pixel size of the control area, the image coordinates of the target interaction point are converted into coordinates within the control area and then mapped to the target screen coordinates.

[0011] Optionally, real-time identification of marker units in the image sequence and extraction of the geometric parameters of each marker unit, including:

[0012] The image sequence is preprocessed to obtain a binary image;

[0013] In a binary image, all contours are detected. The minimum bounding rectangle method is used to calculate the pixel length of the long side and the pixel length of the short side of each contour. Candidate contours are then selected based on a preset range of short side pixel lengths.

[0014] The contour with the longest pixel length on the long side is selected as the first contour from the candidate contours, and the second contour is selected from the remaining candidate contours based on the preset matching conditions.

[0015] Extract the center coordinates, long side pixel length, short side pixel length, and long side direction vector of the minimum bounding rotation rectangle corresponding to the first and second contours, and use them as the geometric parameters of each marking unit.

[0016] Optionally, determining the axes of the coordinate system and calculating the pixel density includes:

[0017] Calculate the center-connecting vector based on the center coordinates of the two marked units;

[0018] Normalize the center line vector to obtain the first axial unit vector, and calculate the second axial unit vector that is perpendicular to the first axial unit vector;

[0019] Calculate the magnitude of the center line vector to obtain the center distance in pixels;

[0020] The pixel density is obtained by dividing the center distance pixel length by the preset physical center distance between the two marker units.

[0021] Optionally, the origin of the coordinate system is calculated, including:

[0022] Using the center of any marker unit as a reference point, move a preset pixel distance along each axis to obtain the origin of the coordinate system.

[0023] Optionally, determining the axes of the coordinate system and calculating the pixel density also includes:

[0024] A virtual reference system is synthesized, wherein the virtual reference system is a quadrilateral range covering the two marked units;

[0025] Calculate the average direction vector of the long side of the two marked units in the virtual reference frame, normalize it to obtain the first axial unit vector, and calculate the second axial unit vector that is perpendicular to the first axial unit vector.

[0026] Based on the pixel lengths of the long and short sides of the two marker units in the virtual reference frame, the pixel densities of the first and second axes are calculated.

[0027] Optionally, calculating the origin of the coordinate system also includes:

[0028] If two marker units are arranged horizontally, take any lower vertex of the virtual reference system as the reference point, and move it along the second axis by a set distance to obtain the origin of the coordinate system;

[0029] If two marker units are arranged vertically, take any right vertex of the virtual reference system as the reference point, and move it along the second axis by a set distance to obtain the origin of the coordinate system.

[0030] Optionally, the image coordinates of the target interaction point are converted into coordinates within the control area and mapped to target screen coordinates, including:

[0031] For any interactive point in the image, calculate its vector relative to the origin of the coordinate system;

[0032] Calculate the projected length of the vector along each axis;

[0033] Divide the projection length by the pixel size of the control area in each axis to calculate the normalized relative coordinates;

[0034] Multiply the normalized relative coordinates by the target screen resolution to obtain the screen coordinates;

[0035] The pixel dimensions of the control area along each axis are obtained in the following way:

[0036] Multiply the physical width in the physical parameters of the control area by the pixel density to obtain the width pixel size of the control area.

[0037] Multiply the physical height in the physical parameters of the control area by the pixel density to obtain the height pixel size of the control area.

[0038] The width and height pixel dimensions are respectively used as the pixel dimensions of the control area in each axis.

[0039] Optionally, the method further includes:

[0040] Based on the control area, and combined with the preset width of the cursor-catching area and the width of the fence area, a cursor-catching area for maintaining cursor visibility and a fence area for cursor filtering are generated during interaction.

[0041] Optionally, the two marking units include a first marking unit and a second marking unit with a length shorter than the first marking unit;

[0042] The direction from the center coordinate of the first marking unit to the center coordinate of the second marking unit is taken as the positive direction of the first axial unit vector.

[0043] In a second aspect, the present invention provides a cursor mapping system, comprising:

[0044] An image acquisition module is used to acquire an image sequence of a target interactive area, wherein the image sequence contains at least two mutually separated marker units;

[0045] The marker recognition module is used to identify marker units in an image sequence in real time and extract the geometric parameters of each marker unit;

[0046] The coordinate system construction module is used to determine the axis of the coordinate system and calculate the pixel density and origin based on the geometric parameters of each marker unit.

[0047] The control area generation module is used to obtain the pixel size of the control area based on the pixel density and preset control area physical parameters, and generate the control area in real time by combining the axis and origin of the coordinate system.

[0048] The coordinate mapping module is used to convert the image coordinates of the target interaction point into coordinates within the control area based on the axis, origin, and pixel size of the coordinate system, and then map them to the target screen coordinates.

[0049] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0050] This invention acquires image sequences containing at least two isolated marker units, identifies and extracts the geometric parameters of the marker units in real time, determines a unique two-dimensional coordinate plane, and updates the coordinate system in real time every frame, accurately reflecting the current physical position of the light spot and avoiding cumulative cursor drift caused by camera module movement. When the user is not paying attention to the cursor position, the movement of the cursor with the camera module has little impact on the user experience, thus reducing the requirement for a continuously stable coordinate system. Specifically, this invention uses a coordinate system establishment step based on the vector connecting the centers of the two marker units, which effectively suppresses vertex jitter and makes the cursor more stable. This invention achieves stable cursor mapping even with a small total length of marker units, reducing the size requirements for interactive markers. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the cursor mapping method provided in an embodiment of the present invention;

[0052] Figure 2 This is a schematic diagram of the synthesis of two coordinate systems using the center point method provided in an embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of the synthesis of three coordinate systems using the center point method provided in an embodiment of the present invention;

[0054] Figure 4 This is a schematic diagram of the synthesis of two marked vertex coordinate systems provided in an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of the synthesis of three marked vertex coordinate systems provided in an embodiment of the present invention;

[0056] Figure 6This is a schematic diagram of region division provided for an embodiment of the present invention.

[0057] Among them: 101, long strip; 102, short strip; 103, third reflective mark; 201, fenced area; 202, edge suction area; 203, control area. Detailed Implementation

[0058] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0059] Example 1

[0060] This embodiment introduces a cursor mapping method, such as... Figure 1 As shown, it specifically includes:

[0061] S1. Acquire an image sequence of the target interaction area, wherein the image sequence contains at least two mutually separate marker units;

[0062] Real-time video streams are captured by a camera module worn on the user's head, for example, the video stream has a resolution of 1920×1080 and a frame rate of 30fps.

[0063] Two rectangular reflective strips are selected as the marking unit, such as... Figure 2 The system includes a long strip 101, serving as the first reflective mark, measuring 35mm × 6mm, and a short strip 102, serving as the second reflective mark, measuring 21mm × 6mm. The two strips are arranged parallel to each other, spaced 4mm apart, with a physical center-to-center distance of 32mm. In this embodiment, the two mark units have different lengths to break the symmetry and uniquely determine the positive direction of the coordinate axis.

[0064] S2. Real-time identification of marker units in image sequences and extraction of geometric parameters of each marker unit;

[0065] For each frame of the image, the original color image is converted into a grayscale image to obtain a grayscale image;

[0066] The grayscale image is binarized by setting a threshold of 180 to obtain a binary image;

[0067] A morphological opening operation is performed on the binary image to eliminate isolated noise points, and then a morphological closing operation is performed to connect broken regions. The kernel size for both the opening and closing operations is 5×5.

[0068] After the above processing, a clear binary image is obtained.

[0069] In the binary image, all contours are searched to obtain a contour set. The minimum bounding rectangle of all contours is fitted using the minimum bounding rectangle method, and the pixel lengths of the long and short sides of each contour are calculated. Specifically: the smoothed pixel density calculated in the previous frame (using an initial estimate of 5.5px / mm@1920×1080 in the first frame) is multiplied by the physical width of stripe 101, 6mm, to obtain the expected width of stripe 101. Contours with widths between 0.1 and 1.9 times the expected value are retained as candidate contours.

[0070] The candidate contour set is sorted according to the length of the long side pixels. The contour with the longest long side pixel length is selected as the long strip 101 contour. The ratio of its long side pixel length to smooth pixel density is verified to see if it is close to the theoretical physical length: 35mm±5mm.

[0071] From the remaining candidate set, we search for short strip contours. Taking the specific geometric dimensions of strip 101 and strip 102 as examples, the matching conditions include:

[0072] The projection distance of the line connecting the center of strip 101 to the long side is close to the theoretical value of the physical center distance (32mm) multiplied by the smooth pixel density;

[0073] The vertical offset is less than half the physical length of the short side (3mm) multiplied by the smooth pixel density;

[0074] The deviation between the short pixel length and the product of the physical length of the short strip (21mm) and the smooth pixel density is less than 30%.

[0075] The above matching conditions apply to the filtering of the second labeling unit in any two labeling units. This example uses long bar 101 and short bar 102 as examples for illustration. The content described has any of the situations in the method proposed in this invention.

[0076] The smoothed pixel density of the previous frame refers to the smoothed pixel density result that is filtered and saved after the previous frame image has been processed, rather than directly using the original pixel density of the previous frame.

[0077] After determining the long bar 101 and the short bar 102, and meeting the update conditions, the pixel density calculation for the next frame is performed.

[0078] Extract the geometric parameters of the minimum bounding rectangle corresponding to the long and short outlines, including: center coordinates, long side pixel length, short side pixel length, and long side direction vector.

[0079] S3. Based on the geometric parameters of each marker unit, determine the axis of the coordinate system and calculate the pixel density and origin;

[0080] This embodiment provides the center point method. Figure 2In the diagram, L represents the first axial direction, and S represents the second axial direction, as shown below. Figure 2 As shown in (a), when the long strip 101 and the short strip 102 are arranged horizontally, the direction of the first axial direction is the X-axis direction, and the direction of the second axial direction is the Y-axis direction; as Figure 2 As shown in (b), when the long strip 101 and the short strip 102 are arranged vertically, the direction of the first axis is the Y-axis direction, and the direction of the second axis is the X-axis direction.

[0081] Specifically: Calculate the center-connecting vector based on the center coordinates of the two marked units.

[0082] ;

[0083] in, The vector connecting the centers, The x-coordinate of the center of the short bar. The vertical coordinate of the center of the short bar; The x-coordinate of the center of the bar. The vertical coordinate of the center of the bar.

[0084] Normalize the center line vector to obtain the X-axis unit vector. And calculate the Y-axis unit vector perpendicular to the X-axis unit vector. ;

[0085] Calculate the magnitude of the vector connecting the centers, which is equivalent to calculating the Euclidean distance between the two center coordinates, to obtain the pixel length of the center distance.

[0086] Divide the center distance pixel length by the preset physical center distance of 32mm to obtain the pixel density.

[0087] Using the center of any marked unit as a reference point, move a preset pixel distance along the X and Y axes to obtain the origin of the coordinate system. For example... Figure 2 As shown in (a), for example, taking the center of strip 101 as the reference point, move half the length of the long side along the negative direction of the first axis (i.e., the X-axis), and then move half the length of the short side plus a certain distance along the positive direction of the second axis (i.e., the Y-axis) (as long as the marking unit is not obscured; in actual engineering, this can be flexibly set according to the actual situation) to obtain the origin of the coordinate system. The purpose of this offset process is to prevent the marking unit from being obscured during the interaction process, which would cause the system to fail to be established, and to ensure the continuity and stability of the positioning.

[0088] It should be noted that the second axial unit vector in this step is obtained by rotating the first axial direction 90° clockwise. In other embodiments, it can also be obtained by rotating it 90° counterclockwise. As for the selection of the reference point, it can also be based on the short strip 102. When the two marking units are the same size, the center of either marking unit can be arbitrarily selected as the reference point.

[0089] like Figure 2 As shown in (b), in the vertical arrangement of long strips 101 and short strips 102, with the center of long strip 101 as the reference point, move half the length of the long side along the negative direction of the first axis (Y-axis), and then move half the length of the short side plus a distance along the positive direction of the second axis (X-axis). The final purpose is the same as that in the horizontal arrangement described above.

[0090] S4. Based on the pixel density and preset physical parameters of the control area, obtain the pixel size of the control area, and generate the control area in real time by combining the axis and origin of the coordinate system.

[0091] Specifically, the physical size of the control area is converted to pixel size:

[0092] Width pixels ;in, The physical width of the control area. Pixel density.

[0093] height pixels .in, The physical height of the control area.

[0094] Generate width pixels along the X-axis and height pixels along the Y-axis to generate the four vertices of the control area:

[0095] Top left ;

[0096] Top right ;

[0097] Bottom left ;

[0098] Bottom right ;

[0099] In a preferred embodiment, the edge-absorbing area 202 and the fence area 201 are generated according to the preset edge-absorbing area width and fence area width, such as... Figure 6 As shown, the edge-catching area 202 is used to keep the cursor visible and interactive even when the cursor slightly moves out of the control area, preventing cursor flickering caused by jitter; the fence area 201 is used to filter out light spots that are too far from the control area, in which the cursor is invisible and non-interactive. The above areas are used together for cursor validity filtering during the interaction process; Figure 6 It also includes control area 203.

[0100] S5. Based on the axis of the coordinate system, the origin, and the pixel size of the control area, convert the image coordinates of the target interaction point into coordinates within the control area, and map them to the target screen coordinates.

[0101] For any interaction point P in the image, calculate its vector relative to the origin Origin of the coordinate system: V = P - Origin;

[0102] Calculate the projected length of V along the X-axis: ;

[0103] Calculate the projected length of V along the Y-axis: ;

[0104] Calculate normalized relative coordinates: relative

[0105] relatively ;

[0106] Will and The coordinates are limited to the range of [0,1] to ensure that the final mapped screen coordinates are always within the target screen range, thus avoiding the cursor from going beyond the screen boundary due to calculation errors (such as edge processing and projection deviation).

[0107] Mapped to screen coordinates: screen ;in For target width resolution;

[0108] Screen ;in, For target height resolution.

[0109] Output screen coordinates .

[0110] It should be noted that this invention does not attempt to suppress the influence of camera movement on the cursor. This is because: when the user does not initiate a click interaction, the cursor movement with the camera has no substantial impact on the user experience; however, when the user initiates a click, this coordinate system can provide accurate mapping based on the real-time updates of the current frame and quickly and conveniently move the cursor to the target position for human-computer interaction.

[0111] The specific working process is as follows: When the user's head turns, causing the camera to move, the position of the marking unit in the image changes accordingly, and the coordinate system is updated accordingly;

[0112] When the user is not interacting with the computer through the light spot, the user does not pay attention to changes in the cursor position;

[0113] When a user is about to click, they will naturally look back at the screen, at which point the cursor position will become visible.

[0114] Users move the cursor to the target by fine-tuning their finger position, and then execute the interactive event to complete the click;

[0115] At the instant the light spot reappears, this coordinate system provides a mapping based on the marker unit of the current frame, ensuring that the cursor is located where the finger is currently pointing.

[0116] The screen coordinates corresponding to the physical location.

[0117] The entire process described above conforms to the natural habits of human-computer interaction—first locate, then click, without requiring the camera to remain stationary.

[0118] Expansion of the number of marker units:

[0119] The above content uses two marker units (long bar 101 and short bar 102) as an example to illustrate the coordinate system establishment method. In practical applications, three or more non-collinear marker units can be used, but the coordinate system establishment method remains the same: the axis and pixel density are still determined by the center line connecting only two of the marker units (e.g., the pair with the largest area and the farthest distance).

[0120] like Figure 3 As shown, the main function of adding the third reflective mark 103 is:

[0121] Verify the validity of the coordinate system: The center point of the third reflective mark 103 should fall within the expected position of the coordinate system determined by the first two marks (long bar 101 and short bar 102); otherwise, the recognition result of the current frame is deemed unreliable.

[0122] Smooth coordinate output: Use the center points of multiple markers to perform a weighted average, which improves the stability of the coordinate system and suppresses jitter;

[0123] Improve coordinate accuracy: By utilizing the deviation between the actual and theoretical positions of the third mark in the Y-axis direction, the Y coordinate of the light spot is corrected, thereby improving the positioning accuracy of the cursor in the vertical direction;

[0124] Anti-occlusion capability: When one of the markers is occluded, it can switch to the other two markers to continue working.

[0125] Those skilled in the art will understand that the above methods are uniformly applicable to two, three or more marking units, and all fall within the protection scope of this invention.

[0126] Example 2

[0127] Based on the same inventive concept as Embodiment 1, the only difference between this embodiment and Embodiment 1 is that, in S3, this embodiment provides an alternative approach, using the vertex method for system construction. The specific steps are as follows:

[0128] S3. Based on the geometric parameters of each marker unit, determine the axis of the coordinate system and calculate the pixel density and origin;

[0129] S3.1, Synthesizing a Virtual Reference Frame:

[0130] like Figure 4 As shown, the four vertices of each rectangle are sorted (in this example, the vertices are named according to their horizontal positions) to obtain the labeled vertices: top left LU, bottom left LD, top right RU, and bottom right RD. 'a' represents the long 101 prefix, and 'b' represents the short 102 prefix.

[0131] Based on the relative positions of the two rectangles (horizontally or vertically arranged), the outermost vertices are taken to synthesize the four vertices of the virtual reference. The final virtual reference system is a quadrilateral range covering the two marked units.

[0132] S3.2 Identify the two long sides of the virtual reference system, calculate the average direction vector of the two long sides of the virtual reference system, normalize it and use it as the first axial unit vector, and calculate the second axial unit vector that is perpendicular to the first axial unit vector.

[0133] By calculating the pixel lengths of the long and short sides of the two marker cells in the virtual reference frame, the pixel density along the first axis and the pixel density along the second axis are obtained.

[0134] First axial pixel density = Pixel length of the long side of the virtual reference frame / Physical length of the long side.

[0135] Second axial pixel density = Pixel length of the short side of the virtual reference frame / Physical length of the short side.

[0136] This method is suitable for scenarios requiring high precision and can reduce drift caused by perspective distortion.

[0137] In some implementations, a uniform pixel density is used as a simplified approach, significantly reducing computational complexity while maintaining sufficient accuracy. Specifically:

[0138] Calculate the average pixel length of the two longer sides ;

[0139] Based on the total physical length of the marking unit = Long bars + gaps + short bars, calculate pixel density: .

[0140] S3.3 Calculate the origin of the coordinate system, including the following steps:

[0141] like Figure 4 As shown in (a), if the two marker units are arranged horizontally, the first axis L corresponds to the X-axis direction and the second axis S corresponds to the Y-axis direction. The lower left vertex aLD of the virtual reference system is taken as the reference point, and the coordinate system origin is obtained by moving along the Y-axis by a set distance. In other embodiments, vertex bRD can also be selected as the reference point.

[0142] like Figure 4As shown in (b), if the two marker units are arranged vertically, the first axis L corresponds to the Y-axis direction and the second axis S corresponds to the X-axis direction. The upper right vertex aRU of the virtual reference system is taken as the reference point, and the coordinate system origin is obtained by moving along the X-axis by a set distance. In other embodiments, vertex bRD can also be selected as the reference point.

[0143] The spacing settings mentioned above are actually in pixel format: Where GapPx is the pixel-based spacing setting. The spacing is set in physical form. Pixel density.

[0144] The purpose of moving from the reference point is also to prevent the finger from obscuring the marker unit during the interaction, which could lead to the inability to establish a system and ensure the continuity and stability of the positioning.

[0145] When the number of marker units increases, a virtual reference frame is generated using the same method as the two marker units mentioned above, and subsequent steps are the same, such as... Figure 5 The diagram shows a composite coordinate system of three marker units using the vertex method. The function of the third reflective marker 103 is the same as that described in Example 1.

[0146] Example 3

[0147] Based on the same concept as other embodiments, this embodiment provides a cursor mapping system, including:

[0148] An image acquisition module is used to acquire an image sequence of a target interactive area, wherein the image sequence contains at least two mutually separated marker units;

[0149] The marker recognition module is used to identify marker units in an image sequence in real time and extract the geometric parameters of each marker unit;

[0150] The parameter extraction module is used to determine the axis of the coordinate system and calculate the pixel density and origin based on the geometric parameters of each marker unit.

[0151] The control area generation module is used to generate the pixel size of the control area based on the coordinate system's axis, pixel density, and origin.

[0152] The coordinate mapping module is used to convert the image coordinates of the target interaction point into coordinates within the control area based on the axis, origin, and pixel size of the coordinate system, and then map them to the target screen coordinates.

[0153] Specifically, as a preferred implementation, the image acquisition module adopts a wearable camera module, which integrates an infrared camera and an infrared fill light, and is worn on the user's head.

[0154] The marker recognition module, parameter extraction module, control area generation module, and coordinate mapping module can be integrated into the same processor (such as the processor built into a wearable camera module), or they can be distributed in different devices. For example:

[0155] Front-end acquisition + back-end processing: The wearable camera module is only responsible for image acquisition. It sends the image sequence to a computer or mobile phone via wired (USB) or wireless (WiFi / Bluetooth) connection, and the processor in the computer or mobile phone performs recognition, coordinate system calculation and coordinate mapping.

[0156] Integrated processing: All modules are integrated into the wearable camera module, and the calculation is completed by the built-in processor (such as ESP32) of the module, and the final screen coordinates are sent to the controlled device wirelessly.

[0157] Hybrid processing: The wearable camera module completes image acquisition and preliminary processing (such as image preprocessing and marker unit recognition), and wirelessly transmits the extracted geometric parameters to a computer or mobile phone, which then performs coordinate system calculations and coordinate mapping. This method reduces front-end power consumption while retaining the flexibility of back-end processing, and can repeatedly leverage the high-performance computing capabilities of computers or mobile phones.

[0158] The specific functions of each module described above are explained in the relevant content of the method in Embodiment 1, and will not be repeated here.

[0159] In summary, this invention acquires image sequences containing at least two isolated marker units, identifies and extracts the geometric parameters of the marker units in real time, determines a unique two-dimensional coordinate plane, and updates the coordinate system in real time every frame, accurately reflecting the current physical position of the light spot and avoiding cumulative cursor drift caused by camera module movement. When the user is not paying attention to the cursor position, the movement of the cursor with the camera module has little impact on the user experience, thus reducing the requirement for a continuously stable coordinate system. Specifically, this invention uses a coordinate system establishment step based on the vector connecting the centers of the two marker units, which effectively suppresses vertex jitter and makes the cursor more stable. This invention achieves stable cursor mapping even with a small total length of marker units, reducing the size requirements for interactive markers.

[0160] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0161] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0162] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0163] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0164] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A cursor mapping method, characterized in that, include: Acquire an image sequence of the target interaction area, wherein the image sequence contains at least two mutually separate marker units; Real-time identification of marker units in image sequences and extraction of geometric parameters of each marker unit; Based on the geometric parameters of each marker unit, the axis of the coordinate system is determined and the pixel density and origin are calculated; Based on the pixel density and preset physical parameters of the control area, the pixel size of the control area is obtained, and the control area is generated in real time by combining the axis and origin of the coordinate system. Based on the coordinate system's axis, origin, and pixel size of the control area, the image coordinates of the target interaction point are converted into coordinates within the control area and then mapped to the target screen coordinates.

2. The cursor mapping method according to claim 1, characterized in that, Real-time identification of labeled units in image sequences, extraction of geometric parameters of each labeled unit, including: The image sequence is preprocessed to obtain a binary image; In a binary image, all contours are detected. The minimum bounding rectangle method is used to calculate the pixel length of the long side and the pixel length of the short side of each contour. Candidate contours are then selected based on a preset range of short side pixel lengths. The contour with the longest pixel length on the long side is selected as the first contour from the candidate contours, and the second contour is selected from the remaining candidate contours based on the preset matching conditions. Extract the center coordinates, long side pixel length, short side pixel length, and long side direction vector of the minimum bounding rotation rectangle corresponding to the first and second contours, and use them as the geometric parameters of each marking unit.

3. The cursor mapping method according to claim 2, characterized in that, Determine the axes of the coordinate system and calculate the pixel density, including: Calculate the center-connecting vector based on the center coordinates of the two marked units; Normalize the center line vector to obtain the first axial unit vector, and calculate the second axial unit vector that is perpendicular to the first axial unit vector; Calculate the magnitude of the center line vector to obtain the center distance in pixels; The pixel density is obtained by dividing the center distance pixel length by the preset physical center distance between the two marker units.

4. The cursor mapping method according to claim 3, characterized in that, Calculate the origin of the coordinate system, including: Using the center of any marker unit as a reference point, move a preset pixel distance along each axis to obtain the origin of the coordinate system.

5. The cursor mapping method according to claim 2, characterized in that, Determining the axes of the coordinate system and calculating pixel density also includes: A virtual reference system is synthesized, wherein the virtual reference system is a quadrilateral range covering the two marked units; Calculate the average direction vector of the long side of the two marked units in the virtual reference frame, normalize it to obtain the first axial unit vector, and calculate the second axial unit vector that is perpendicular to the first axial unit vector. Based on the pixel lengths of the long and short sides of the two marker units in the virtual reference frame, the pixel densities of the first and second axes are calculated.

6. The cursor mapping method according to claim 5, characterized in that, Calculating the origin of the coordinate system also includes: If two marker units are arranged horizontally, take any lower vertex of the virtual reference system as the reference point, and move it along the second axis by a set distance to obtain the origin of the coordinate system; If two marker units are arranged vertically, take any right vertex of the virtual reference system as the reference point, and move it along the second axis by a set distance to obtain the origin of the coordinate system.

7. The cursor mapping method according to claim 1, characterized in that, Convert the image coordinates of the target interaction point to coordinates within the control area, and then map them to target screen coordinates, including: For any interactive point in the image, calculate its vector relative to the origin of the coordinate system; Calculate the projected length of the vector along each axis; Divide the projection length by the pixel size of the control area in each axis to calculate the normalized relative coordinates; Multiply the normalized relative coordinates by the target screen resolution to obtain the screen coordinates; The pixel dimensions of the control area along each axis are obtained in the following way: Multiply the physical width in the physical parameters of the control area by the pixel density to obtain the width pixel size of the control area. Multiply the physical height in the physical parameters of the control area by the pixel density to obtain the height pixel size of the control area. The width and height pixel dimensions are respectively used as the pixel dimensions of the control area in each axis.

8. The cursor mapping method according to claim 1, characterized in that, The method further includes: Based on the control area, and combined with the preset width of the cursor-catching area and the width of the fence area, a cursor-catching area for maintaining cursor visibility and a fence area for cursor filtering are generated during interaction.

9. The cursor mapping method according to claim 3, characterized in that, The two marking units include a first marking unit and a second marking unit that is shorter than the first marking unit; The direction from the center coordinate of the first marking unit to the center coordinate of the second marking unit is taken as the positive direction of the first axial unit vector.

10. A cursor mapping system, characterized in that, include: An image acquisition module is used to acquire an image sequence of a target interactive area, wherein the image sequence contains at least two mutually separated marker units; The marker recognition module is used to identify marker units in an image sequence in real time and extract the geometric parameters of each marker unit; The coordinate system construction module is used to determine the axis of the coordinate system and calculate the pixel density and origin based on the geometric parameters of each marker unit. The control area generation module is used to obtain the pixel size of the control area based on the pixel density and preset control area physical parameters, and generate the control area in real time by combining the axis and origin of the coordinate system. The coordinate mapping module is used to convert the image coordinates of the target interaction point into coordinates within the control area based on the axis, origin, and pixel size of the coordinate system, and then map them to the target screen coordinates.

Images