AR (Augmented Reality) interaction and spatial positioning system and method for three-dimensionally covering invisible two-dimensional code

Abstract

The invention discloses an AR interaction and spatial positioning system and method for three-dimensionally covering an invisible two-dimensional code, and belongs to the field of AR interaction, and the system comprises an invisible two-dimensional code module, an AR recognition module, a spatial positioning module and an AR interaction module. The invisible two-dimensional code module codes preset AR interaction information to generate invisible two-dimensional codes, and prints the invisible two-dimensional codes on a plurality of surfaces or different angle positions of a physical object; the AR identification module simultaneously captures a plurality of invisible two-dimensional codes through a specific waveband camera array and decodes the invisible two-dimensional codes to obtain AR interaction information; the spatial positioning module calculates the spatial attitude and position information of the object by analyzing the relative positions and directions of the plurality of two-dimensional codes; and the AR interaction module superposes corresponding AR content on the user equipment according to the obtained AR interaction information and the calculated spatial attitude and position information, detects an interaction operation between the user and the AR content and updates AR content display in real time.

Description

Technical Field

[0001] The present invention relates to the field of AR interaction, and in particular to an AR interaction and spatial positioning system and method with a three-dimensionally covered invisible two-dimensional code. Background Art

[0002] Augmented reality (AR) technology provides users with a rich interactive experience by superimposing virtual information with real-world scenes. This technology has broad application prospects in education, entertainment, medical care, industry, etc. However, traditional AR technology has some limitations in the use of identification codes, which often affect the aesthetics of physical objects and limit the application scenarios of AR technology.

[0003] Existing AR recognition technologies mainly include the following:

[0004] 1. Recognition methods based on specific image markers, such as ARToolKit. This method uses predefined black and white square patterns as markers. Although the recognition accuracy is high, the markers are often abrupt and affect the overall appearance of the object.

[0005] 2. Recognition method based on natural feature points, such as Vuforia, etc. This method uses the natural features of the image for recognition and does not require specific markers, but has high requirements on ambient light and surface texture of objects, and does not work well on some smooth or single-color objects.

[0006] 3. Recognition method based on 3D object model. This method realizes recognition by matching the actual object with the pre-established 3D model. It is suitable for objects with complex shapes, but it has large computational complexity, poor real-time performance, and requires the establishment of an accurate 3D model library in advance.

[0007] 4. Positioning and identification method based on GPS and sensors. This method is mainly used for large-scale outdoor AR applications, but it has low accuracy and is not effective in indoor environments.

[0008] 5. Identification method based on visible light communication. This method uses the flashing of LED lights to transmit information and can achieve tagless identification, but it requires special hardware equipment support and has limited application scenarios.

[0009] In summary, although these methods can realize AR recognition and interaction, they all have some limitations, such as visual interference, environmental dependence, high computational complexity, limited information capacity, susceptibility to interference and privacy protection. Summary of the invention

[0010] The purpose of the present invention is to provide an AR interaction and spatial positioning system and method with a three-dimensionally covered invisible two-dimensional code to solve the problems raised in the above background technology.

[0011] In order to solve the above technical problems, the present invention provides the following technical solution: an AR interaction and spatial positioning method of a three-dimensionally covered invisible QR code, the method comprising the following steps:

[0012] Step S100: Encode the preset AR interaction information to generate an invisible QR code;

[0013] Step S200: printing the invisible two-dimensional code on multiple surfaces or at different angles of the target physical object;

[0014] Step S300: using a camera to scan the target physical object to obtain a plurality of invisible two-dimensional code images, and processing the plurality of invisible two-dimensional code images to obtain AR interaction information;

[0015] Step S400: Analyze the relative positions and directions of multiple invisible two-dimensional code images, and calculate the spatial posture and position information of the target physical object;

[0016] Step S500: Based on the acquired AR interaction information and the calculated spatial posture and position information, corresponding AR content is superimposed on the user device, while detecting the interaction operation between the user and the AR content and updating the AR content display in real time.

[0017] Furthermore, the process of encoding the AR interaction information into an invisible QR code in step S100 includes:

[0018] Step 101: Convert AR interaction information into a binary data stream;

[0019] Step 102: Generate a two-dimensional code matrix from the binary data stream according to a preset coding rule;

[0020] Step 103: Replace the black and white modules in the two-dimensional code matrix with infrared absorption or reflection materials to generate an invisible two-dimensional code pattern.

[0021] Furthermore, in step S200, the generated invisible two-dimensional code is printed on multiple surfaces or at different angles of the target physical object using ink.

[0022] The above steps S100 and S200 convert the AR interaction information into a visually almost invisible invisible QR code through coding technology and print it on multiple surfaces or different angles of the physical object, so as to store and transmit the AR information without interfering with the appearance of the physical object, which not only ensures the integrity of the original visual aesthetics of the physical object without interference, but also realizes the concealment of the technical application.

[0023] Furthermore, in step S300, a camera is used to scan the target physical object to obtain a plurality of invisible two-dimensional code images, and the camera is a near-infrared camera;

[0024] The processing of the multiple invisible two-dimensional code images includes the following specific processing steps: image enhancement and decoding. The image enhancement process includes:

[0025] Step 301: grayscale the captured invisible two-dimensional code image;

[0026] Step 302: binarize the image using an adaptive threshold segmentation algorithm;

[0027] Step 303: Apply morphological operations to remove noise and fill holes;

[0028] Step 304: Perform perspective correction to obtain a standard front view image;

[0029] The decoding process includes: locating the QR code area in the standard front view image, reading the binary matrix data in the area, applying the QR code decoding algorithm to parse the data, and verifying and parsing the AR interaction information.

[0030] In the above steps, the camera of a specific band can recognize and capture invisible QR code images that are difficult to detect with the naked eye. Capturing multiple invisible QR code images at the same time helps to accurately calculate the spatial posture and position of the object in the future. Image enhancement technology can improve the clarity of the QR code image, making it easier to decode. The decoding process extracts the AR interaction information stored in the QR code, providing data support for the subsequent overlay of AR content on the user's device.

[0031] Furthermore, in step S400, the spatial posture and position information of the object are calculated by the following method:

[0032] Step S401: analyzing the distortion degree of multiple QR codes and estimating the tilt angle of each QR code relative to the camera, specifically including the steps of image acquisition, image preprocessing, QR code positioning, feature point extraction, distortion analysis, perspective transformation, angle calculation, multi-source fusion, and time series filtering;

[0033] The image acquisition uses a near-infrared camera to capture a high-resolution image containing the invisible QR code;

[0034] The image preprocessing includes performing denoising, contrast enhancement, and image binarization conversion on the collected high-resolution images;

[0035] The two-dimensional code positioning adopts an edge detection algorithm based on Hough transform;

[0036] The feature point extraction uses a Harris corner detector to extract the coordinates of the four corner points of the QR code and calculate the coordinates of the center point of the QR code;

[0037] The distortion analysis calculates the deviation between the quadrilateral formed by the four corner points and the standard square, and measures the diagonal length ratio and the adjacent side length ratio;

[0038] The perspective transformation constructs a 3x3 perspective transformation matrix H, satisfying H*[x,y,1]^T=[X,Y,1]^T, where (x,y) is the distorted image coordinate, (X,Y) is the coordinate corresponding to the ideal square, and T is the transposition symbol;

[0039] The angle calculation performs SVD decomposition on the matrix H, extracts the rotation matrix R, and converts R into Euler angles (α, β, γ):

[0040] α = arctan(R32 / R33);

[0041] β = arcsin (-R31);

[0042] γ = arctan(R21 / R11);

[0043] The multi-source fusion, when multiple QR codes are detected, uses a weighted average method to fuse the inclination results;

[0044] The time series filtering uses an extended Kalman filter (EKF), the state vector includes a position and an angle, and the vector is the angle value calculated for the current frame;

[0045] Step S402: Establishing a spatial coordinate system of the object based on the relative position relationship of the multiple two-dimensional codes:

[0046] First, select the QR code with the highest definition as the origin, and then determine the x, y, and z axis directions of the coordinate system based on the positions of other QR codes relative to the origin;

[0047] Step S403: using the triangulation principle to calculate the precise spatial position and posture of the object relative to the camera, specifically including the following steps:

[0048] Solve the PnP (Perspective-n-Point) problem based on the known camera intrinsic parameter matrix and the actual size of the QR code;

[0049] The RANSAC algorithm is used to improve the solution accuracy, and the number of iterations is not less than m times, where m>=100;

[0050] Output the 6 degrees of freedom posture of the object relative to the camera, including three spatial position parameters x, y, z and three spatial direction parameters roll, pitch, yaw.

[0051] The above steps can calculate the posture and position information of the object in three-dimensional space by analyzing the relative positions and directions of multiple invisible QR codes distributed at different positions of the physical object. This information is crucial for achieving more accurate AR content positioning and rendering, and can enhance the immersion and realism of the AR experience.

[0052] Furthermore, step S500 determines the specific AR content that needs to be superimposed and displayed based on the acquired AR interaction information, and superimposes the corresponding AR content on the user device in combination with the calculated spatial posture and position information, while detecting the user's interaction operation with the AR content and updating the AR content display in real time.

[0053] In the above steps, based on the decoded AR interaction information and the calculated spatial posture and position information, the user device can overlay virtual AR content on the real view of the physical object. This overlay effect enhances the user's visual experience and seamlessly combines the physical object with the virtual information. At the same time, these interactive operations are detected in real time, and the display of AR content is updated according to the operation instructions, achieving a more dynamic and personalized user experience.

[0054] An AR interaction and spatial positioning system with a three-dimensionally covered invisible QR code, the system comprising an invisible QR code module, an AR recognition module, a spatial positioning module, and an AR interaction module;

[0055] The invisible QR code module is used to encode the preset AR interaction information to generate invisible QR codes and print them on multiple surfaces or different angles of the physical object;

[0056] The AR recognition module is used to scan the target physical object through the camera, capture and decode multiple invisible QR codes at the same time, and obtain AR interaction information;

[0057] The spatial positioning module is used to calculate the spatial posture and position information of the object by analyzing multiple invisible two-dimensional codes covered in three dimensions;

[0058] The AR interaction module overlays corresponding AR content on the user device according to the acquired AR interaction information and the calculated spatial posture and position information, and detects the interaction operation between the user and the AR content and updates the AR content display in real time.

[0059] Furthermore, the invisible QR code generation module includes an information encoding unit, an invisible processing unit, and a printing unit;

[0060] The information encoding unit is used to convert the AR interaction information into a binary data stream and generate a two-dimensional code matrix;

[0061] The invisible processing unit is used to replace the black and white modules in the two-dimensional code matrix with infrared absorption or reflection materials;

[0062] The printing unit is used to print the processed invisible two-dimensional code pattern on multiple surfaces or different angles of the physical object.

[0063] Furthermore, the AR recognition module is used to simultaneously scan multiple surfaces of the target physical object through a camera, capture multiple invisible two-dimensional code images and process them to obtain AR interaction information, and the camera is a near-infrared camera;

[0064] The processing of the multiple invisible two-dimensional code images includes the following specific processing steps: image enhancement and decoding. The image enhancement process includes:

[0065] Step 301: grayscale the captured invisible two-dimensional code image;

[0066] Step 302: binarize the image using an adaptive threshold segmentation algorithm;

[0067] Step 303: Apply morphological operations to remove noise and fill holes;

[0068] Step 304: Perform perspective correction to obtain a standard front view image;

[0069] The decoding process includes: locating the QR code area in the standard front view image, reading the binary matrix data in the area, applying the QR code decoding algorithm to parse the data, and verifying and parsing the AR interaction information;

[0070] Furthermore, the spatial positioning module includes a QR code position analysis unit, a spatial coordinate system establishment unit and a triangulation unit, which calculates the spatial posture and position information of the object by analyzing multiple invisible QR codes covered in three dimensions;

[0071] The two-dimensional code position analysis unit estimates the tilt angle of each two-dimensional code relative to the camera by analyzing the degree of distortion of multiple two-dimensional code images, and specifically includes steps including image acquisition, image preprocessing, two-dimensional code positioning, feature point extraction, distortion analysis, perspective transformation, angle calculation, multi-source fusion, and time series filtering;

[0072] The image acquisition uses a near-infrared camera to capture a high-resolution image containing the invisible QR code;

[0073] The image preprocessing includes performing denoising, contrast enhancement, and image binarization conversion on the collected high-resolution images;

[0074] The two-dimensional code positioning adopts an edge detection algorithm based on Hough transform;

[0075] The feature point extraction uses a Harris corner detector to extract the coordinates of the four corner points of the QR code and calculate the coordinates of the center point of the QR code;

[0076] The distortion analysis calculates the deviation between the quadrilateral formed by the four corner points and the standard square, and measures the diagonal length ratio and the adjacent side length ratio;

[0077] The perspective transformation constructs a 3x3 perspective transformation matrix H, satisfying H*[x,y,1]^T=[X,Y,1]^T, where (x,y) is the distorted image coordinate, (X,Y) is the coordinate corresponding to the ideal square, and T is the transposition symbol;

[0078] The angle calculation performs SVD decomposition on the matrix H, extracts the rotation matrix R, and converts R into Euler angles (α, β, γ):

[0079] α = arctan(R32 / R33);

[0080] β = arcsin (-R31);

[0081] γ = arctan(R21 / R11);

[0082] The multi-source fusion, when multiple QR codes are detected, uses a weighted average method to fuse the inclination results;

[0083] The time series filtering uses an extended Kalman filter (EKF), the state vector includes a position and an angle, and the vector is the angle value calculated for the current frame;

[0084] The spatial coordinate system establishing unit establishes a spatial coordinate system of the object according to the relative position relationship of the multiple two-dimensional codes; firstly, the two-dimensional code with the highest definition is selected as the origin, and then the x, y, and z axis directions of the coordinate system are determined according to the positions of the other two-dimensional codes relative to the origin;

[0085] The triangulation unit uses the known geometric relationship between the two-dimensional codes and the camera parameters to calculate the precise spatial position and posture of the object relative to the camera through the triangulation principle, which specifically includes the following steps:

[0086] Solve the PnP (Perspective-n-Point) problem based on the known camera intrinsic parameter matrix and the actual size of the QR code;

[0087] The RANSAC algorithm is used to improve the solution accuracy, and the number of iterations is not less than m times, where m>=100;

[0088] Output the 6 degrees of freedom posture of the object relative to the camera, including three spatial position parameters x, y, z and three spatial direction parameters roll, pitch, yaw.

[0089] Furthermore, the AR interaction module determines the specific AR content that needs to be superimposed and displayed based on the acquired AR interaction information, and superimposes the corresponding AR content on the user device in combination with the calculated spatial posture and position information, while detecting the user's interaction operations with the AR content and updating the AR content display in real time.

[0090] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0091] The present invention creatively introduces invisible QR codes as AR recognition media, which not only ensures that the integrity of the original visual aesthetics of the physical object is not disturbed, but also realizes the concealment of the technical application and improves the harmony of the user experience;

[0092] By using a camera with a specific wavelength for recognition, the present invention significantly improves the accuracy and stability of the recognition process, effectively overcomes the influence of factors such as ambient light and background interference on the recognition effect, and ensures the accurate triggering and display of AR content;

[0093] With the help of three-dimensional overlay QR code technology, the present invention realizes the precise positioning of objects in three-dimensional space. This technological breakthrough greatly enhances the realism and immersion of AR interaction, allowing users to experience a more accurate and natural fusion of virtual and reality.

[0094] The present invention focuses on the simplicity of structure and the efficiency of implementation in system design, which not only reduces the complexity of the system but also facilitates subsequent upgrades and maintenance. The implementation of the present invention not only enriches the application scenarios of AR technology, but also promotes its in-depth application in multiple fields, laying a solid foundation for the rapid promotion and wide application of the technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0096] Figure 1 It is a method flow chart of the AR interaction and spatial positioning method of three-dimensionally covering invisible QR code;

[0097] Figure 2 It is a system flow chart of the AR interaction and spatial positioning system with three-dimensional coverage of invisible QR codes. DETAILED DESCRIPTION

[0098] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0099] See also Figure 1-Figure 2 The present invention provides a technical solution: an AR interaction and spatial positioning method of a three-dimensionally covered invisible two-dimensional code, the method comprising the following steps:

[0100] Step S100: Encode the preset AR interaction information to generate an invisible QR code;

[0101] Step S200: printing the invisible two-dimensional code on multiple surfaces or at different angles of the target physical object;

[0102] Step S300: using a camera to scan the target physical object to obtain a plurality of invisible two-dimensional code images, and processing the plurality of invisible two-dimensional code images to obtain AR interaction information;

[0103] Step S400: Analyze the relative positions and directions of multiple invisible two-dimensional code images, and calculate the spatial posture and position information of the target physical object;

[0104] Step S500: Based on the acquired AR interaction information and the calculated spatial posture and position information, corresponding AR content is superimposed on the user device, while detecting the interaction operation between the user and the AR content and updating the AR content display in real time.

[0105] Furthermore, the process of encoding the AR interaction information into an invisible QR code in step S100 includes:

[0106] Step 101: Convert AR interaction information into a binary data stream;

[0107] Step 102: Generate a two-dimensional code matrix from the binary data stream according to a preset coding rule;

[0108] Step 103: Replace the black modules in the two-dimensional code matrix with infrared absorbing materials and the white modules with infrared reflecting materials to generate an invisible two-dimensional code pattern.

[0109] In step S100, when the preset AR interaction information is encoded to generate an invisible QR code, the AR interaction information includes a unique identifier, a content type tag (such as a 3D model, video, text, etc.), a content link or data, and a 7-bit spatial positioning parameter; in order to improve the reliability of decoding, the Reed-Solomon error correction algorithm is used in the encoding process, and the error correction level is defined to be no less than 15%; according to the amount of information, the size of the generated QR code matrix is ​​adaptively selected between 21x21 and 177x177; if the AR interaction information exceeds the capacity of a single QR code, it is necessary to perform fragment encoding and add sequence information in each fragment; if the lighting conditions of the expected use environment are complex, consider increasing the error correction level to more than 25% to ensure the correct decoding of the information.

[0110] Furthermore, step S200 uses infrared material ink to apply the generated invisible two-dimensional code to multiple surfaces of the physical object through a suitable printing method, ensuring that it is invisible to the human eye but can be captured by a specific camera.

[0111] In step S200, the printed material needs to be a material with significant absorption or reflection characteristics in the wavelength range of 750-1000nm; in order to ensure the recognition effect of the two-dimensional code, the printing accuracy is strictly controlled, and the side length error of each module does not exceed 5%; in order to achieve accurate spatial positioning, at least 3 non-coplanar positions are selected on the surface of the object for printing; if a metal or highly reflective surface is encountered, it is first treated with a special anti-reflective coating and then printed; and for objects with curved surfaces, flexible printing technology is used to ensure the integrity and recognizability of the two-dimensional code.

[0112] The above steps S100 and S200 convert the AR interaction information into a visually almost invisible invisible QR code through coding technology and print it on multiple surfaces or different angles of the physical object, so as to store and transmit the AR information without interfering with the appearance of the physical object, which not only ensures the integrity of the original visual aesthetics of the physical object without interference, but also realizes the concealment of the technical application.

[0113] Furthermore, in step S300, a camera is used to scan the target physical object to obtain multiple invisible two-dimensional code images, and the camera is a near-infrared camera with an operating wavelength range of 750-1000nm;

[0114] The processing of the multiple invisible two-dimensional code images includes the following specific processing steps: image enhancement and decoding. The image enhancement process includes:

[0115] Step 301: grayscale the captured invisible two-dimensional code image;

[0116] Step 302: binarize the image using an adaptive threshold segmentation algorithm;

[0117] Step 303: Apply morphological operations to remove noise and fill holes;

[0118] Step 304: Perform perspective correction to obtain a standard front view image;

[0119] The decoding process includes: locating the QR code area in the standard front view image, reading the binary matrix data in the area, applying the QR code decoding algorithm to parse the data, and verifying and parsing the AR interaction information.

[0120] In step S300, a camera array with a specific wavelength is used to simultaneously scan multiple surfaces of a physical object. This step has high requirements for hardware equipment: the camera array should include at least 3 independent near-infrared cameras, and the field of view of each camera should not be less than 60° to ensure that multiple surfaces of the object can be captured; image quality is also a key factor, requiring a resolution of not less than 1280x720 pixels and a frame rate of not less than 30fps to ensure image clarity and real-time performance; in order to achieve precise spatial positioning, the synchronization error between cameras is strictly controlled within 10ms; if the ambient light is insufficient, an infrared fill light with matching wavelength is required; for larger objects, a lens with adjustable focal length is used to adapt to different scanning distances.

[0121] During the image enhancement and decoding process of multiple captured invisible QR code images, the perspective correction algorithm is set to handle a maximum viewing angle tilt of 45°, taking into account the perspective problems that may exist in actual use; the robustness of the decoding process is very important, and the system is set to handle a maximum of 30% local occlusion or damage; when poor image quality is detected, such as low contrast, multi-frame accumulation enhancement processing is started; if the decoding fails, the system should immediately provide feedback to the user and provide guidance for rescanning to improve the user experience.

[0122] In the above steps, the camera of a specific band can recognize and capture invisible QR code images that are difficult to detect with the naked eye. Capturing multiple invisible QR code images at the same time helps to accurately calculate the spatial posture and position of the object in the future. Image enhancement technology can improve the clarity of the QR code image, making it easier to decode. The decoding process extracts the AR interaction information stored in the QR code, providing data support for the subsequent overlay of AR content on the user's device.

[0123] Furthermore, in step S400, the spatial posture and position information of the object are calculated by the following method:

[0124] Step S401: analyzing the distortion degree of multiple QR codes and estimating the tilt angle of each QR code relative to the camera, specifically including the steps of image acquisition, image preprocessing, QR code positioning, feature point extraction, distortion analysis, perspective transformation, angle calculation, multi-source fusion, and time series filtering;

[0125] The image acquisition uses a near-infrared camera to capture a high-resolution image containing the invisible QR code, with a resolution of no less than 1280x720 pixels;

[0126] The image preprocessing includes using a 3x3 Gaussian filter to denoise the collected high-resolution image, dividing the image into a plurality of sub-blocks of equal size, each sub-block having a size of 8x8 pixels, applying a CLAHE algorithm to enhance contrast, and using an Otsu algorithm to binarize the image;

[0127] The two-dimensional code positioning adopts the edge detection algorithm based on Hough transform, and the positioning accuracy is controlled at the sub-pixel level;

[0128] The feature point extraction uses a Harris corner detector to extract the coordinates of the four corner points of the QR code with an accuracy of ±0.5 pixels, and calculates the coordinates of the center point of the QR code;

[0129] The distortion analysis calculates the deviation between the quadrilateral formed by the four corner points and the standard square, and measures the diagonal length ratio and the adjacent side length ratio;

[0130] The perspective transformation constructs a 3x3 perspective transformation matrix H, satisfying H*[x,y,1]^T=[X,Y,1]^T, where (x,y) is the distorted image coordinate, (X,Y) is the coordinate corresponding to the ideal square, and T is the transposition symbol;

[0131] The angle calculation performs SVD decomposition on the matrix H, extracts the rotation matrix R, and converts R into Euler angles (α, β, γ):

[0132] α = arctan(R32 / R33);

[0133] β = arcsin (-R31);

[0134] γ = arctan(R21 / R11);

[0135] The multi-source fusion, when multiple QR codes are detected, uses a weighted average method to fuse the inclination results, with the weights being determined based on the clarity and area of ​​the QR codes;

[0136] The time series filtering uses an extended Kalman filter (EKF), the state vector includes a position and an angle, and the vector is the angle value calculated for the current frame;

[0137] Step S402: Establishing a spatial coordinate system of the object based on the relative position relationship of the multiple two-dimensional codes:

[0138] First, select the QR code with the highest definition as the origin, and then determine the x, y, and z axis directions of the coordinate system based on the positions of other QR codes relative to the origin;

[0139] Step S403: using the triangulation principle to calculate the precise spatial position and posture of the object relative to the camera, specifically including the following steps:

[0140] Solve the PnP (Perspective-n-Point) problem based on the known camera intrinsic parameter matrix and the actual size of the QR code;

[0141] The RANSAC algorithm is used to improve the solution accuracy, and the number of iterations is not less than m times, where m>=100;

[0142] Output the 6 degrees of freedom posture of the object relative to the camera, including three spatial position parameters x, y, z and three spatial direction parameters roll, pitch, yaw.

[0143] In step S400, the spatial posture and position information of the object are calculated by analyzing the relative positions and directions of multiple QR codes. This step has high requirements on accuracy, and the spatial positioning accuracy must reach ±5mm (position) and ±1° (posture). In order to support real-time tracking, the positioning calculation frequency cannot be lower than 60Hz. The system has a certain fault tolerance and can handle the situation of temporary occlusion of up to 2 QR codes to maintain continuous positioning. When a fast-moving object is detected, a motion prediction algorithm is enabled to reduce tracking delays. For complex scenes, the system can track at least 5 independent objects at the same time to meet the needs of multi-target interaction.

[0144] The above steps can calculate the posture and position information of the object in three-dimensional space by analyzing the relative positions and directions of multiple invisible QR codes distributed at different positions of the physical object. This information is crucial for achieving more accurate AR content positioning and rendering, and can enhance the immersion and realism of the AR experience.

[0145] Furthermore, step S500 determines the specific AR content that needs to be superimposed and displayed based on the acquired AR interaction information, and superimposes the corresponding AR content on the user device in combination with the calculated spatial posture and position information, while detecting the user's interaction operations with the AR content and updating the AR content display in real time. The types of AR content include 3D models, animations, videos, audio, text information, etc.

[0146] In step S500, the specific AR content that needs to be superimposed and displayed is determined based on the obtained AR interaction information, and the corresponding AR content is superimposed on the user device in combination with the calculated spatial posture and position information; in order to ensure a good user experience, the loading time of the AR content should not exceed 3 seconds, and the rendering frame rate of the 3D model should be maintained above 30fps; the registration error between the AR content and the real scene is controlled within 1cm; in the case of unstable network conditions, the system provides an offline caching mechanism to ensure the normal operation of basic functions; for large AR content, progressive loading is implemented, and the low-detail version is displayed first, and then the high-detail content is loaded step by step to balance the loading speed and display quality.

[0147] Detect user interactions with AR content. For touch screen devices, the system supports single-finger or multi-finger gestures, including clicks, slides, zooms, rotations, and other operations. If AR glasses are used, gesture recognition is supported with a recognition accuracy of no less than 95%. To ensure a good interactive experience, the response time of interactive operations is controlled within 100ms. In high-interference environments such as strong light and fast movement, the system enables a predictive algorithm to improve the stability of the interaction. For some complex operations, the system also provides voice-assisted instructions as an alternative interaction method to enhance the system's usability and adaptability.

[0148] In the above steps, based on the decoded AR interaction information and the calculated spatial posture and position information, the user device can overlay virtual AR content on the real view of the physical object. This overlay effect enhances the user's visual experience and seamlessly combines the physical object with the virtual information. At the same time, these interactive operations are detected in real time, and the display of AR content is updated according to the operation instructions, achieving a more dynamic and personalized user experience.

[0149] An AR interaction and spatial positioning system with a three-dimensionally covered invisible QR code, the system comprising an invisible QR code module, an AR recognition module, a spatial positioning module, and an AR interaction module;

[0150] The invisible QR code module is used to encode the preset AR interaction information to generate invisible QR codes and print them on multiple surfaces or different angles of the physical object;

[0151] The AR recognition module is used to scan the target physical object through the camera, capture and decode multiple invisible QR codes at the same time, and obtain AR interaction information;

[0152] The spatial positioning module is used to calculate the spatial posture and position information of the object by analyzing multiple invisible two-dimensional codes covered in three dimensions;

[0153] The AR interaction module overlays corresponding AR content on the user device according to the acquired AR interaction information and the calculated spatial posture and position information, and detects the interaction operation between the user and the AR content and updates the AR content display in real time.

[0154] Furthermore, the invisible QR code generation module includes an information encoding unit, an invisible processing unit, and a printing unit;

[0155] The information encoding unit is used to convert the AR interaction information into a binary data stream and generate a two-dimensional code matrix;

[0156] The invisible processing unit is used to replace the black and white modules in the two-dimensional code matrix with infrared absorption or reflection materials;

[0157] The printing unit is used to print the processed invisible two-dimensional code pattern on multiple surfaces or different angles of the physical object.

[0158] Furthermore, the AR recognition module is used to simultaneously scan multiple surfaces of the target physical object through a camera, capture multiple invisible QR code images and process them to obtain AR interaction information;

[0159] The camera is a near-infrared camera with an operating wavelength range of 750-1000nm;

[0160] The processing of the multiple invisible two-dimensional code images includes the following specific processing steps: image enhancement and decoding. The image enhancement process includes:

[0161] Step 301: grayscale the captured invisible two-dimensional code image;

[0162] Step 302: binarize the image using an adaptive threshold segmentation algorithm;

[0163] Step 303: Apply morphological operations to remove noise and fill holes;

[0164] Step 304: Perform perspective correction to obtain a standard front view image;

[0165] The decoding process includes: locating the QR code area in the standard front view image, reading the binary matrix data in the area, applying the QR code decoding algorithm to parse the data, and verifying and parsing the AR interaction information;

[0166] Furthermore, the spatial positioning module includes a QR code position analysis unit, a spatial coordinate system establishment unit and a triangulation unit, which calculates the spatial posture and position information of the object by analyzing multiple invisible QR codes covered in three dimensions;

[0167] The two-dimensional code position analysis unit estimates the tilt angle of each two-dimensional code relative to the camera by analyzing the degree of distortion of multiple two-dimensional code images, and specifically includes steps including image acquisition, image preprocessing, two-dimensional code positioning, feature point extraction, distortion analysis, perspective transformation, angle calculation, multi-source fusion, and time series filtering;

[0168] The image acquisition uses a near-infrared camera to capture a high-resolution image containing the invisible QR code, with a resolution of no less than 1280x720 pixels;

[0169] The image preprocessing includes using a 3x3 Gaussian filter to denoise the collected high-resolution image, dividing the image into a plurality of sub-blocks of equal size, each sub-block having a size of 8x8 pixels, applying a CLAHE algorithm to enhance contrast, and using an Otsu algorithm to binarize the image;

[0170] The two-dimensional code positioning adopts the edge detection algorithm based on Hough transform, and the positioning accuracy is controlled at the sub-pixel level;

[0171] The feature point extraction uses a Harris corner detector to extract the coordinates of the four corner points of the QR code with an accuracy of ±0.5 pixels, and calculates the coordinates of the center point of the QR code;

[0172] The distortion analysis calculates the deviation between the quadrilateral formed by the four corner points and the standard square, and measures the diagonal length ratio and the adjacent side length ratio;

[0173] The perspective transformation constructs a 3x3 perspective transformation matrix H, satisfying H*[x,y,1]^T=[X,Y,1]^T, where (x,y) is the distorted image coordinate, (X,Y) is the coordinate corresponding to the ideal square, and T is the transposition symbol;

[0174] The angle calculation performs SVD decomposition on the matrix H, extracts the rotation matrix R, and converts R into Euler angles (α, β, γ):

[0175] α = arctan(R32 / R33);

[0176] β = arcsin (-R31);

[0177] γ = arctan(R21 / R11);

[0178] The multi-source fusion, when multiple QR codes are detected, uses a weighted average method to fuse the inclination results, with the weights being determined based on the clarity and area of ​​the QR codes;

[0179] The time series filtering uses an extended Kalman filter (EKF), the state vector includes a position and an angle, and the vector is the angle value calculated for the current frame;

[0180] The spatial coordinate system establishing unit establishes a spatial coordinate system of the object according to the relative position relationship of the multiple two-dimensional codes; firstly, the two-dimensional code with the highest definition is selected as the origin, and then the x, y, and z axis directions of the coordinate system are determined according to the positions of the other two-dimensional codes relative to the origin;

[0181] The triangulation unit uses the known geometric relationship between the two-dimensional codes and the camera parameters to calculate the precise spatial position and posture of the object relative to the camera through the triangulation principle, which specifically includes the following steps:

[0182] Solve the PnP (Perspective-n-Point) problem based on the known camera intrinsic parameter matrix and the actual size of the QR code;

[0183] The RANSAC algorithm is used to improve the solution accuracy, and the number of iterations is not less than m times, where m>=100;

[0184] Output the 6 degrees of freedom posture of the object relative to the camera, including three spatial position parameters x, y, z and three spatial direction parameters roll, pitch, yaw.

[0185] Furthermore, the AR interaction module determines the specific AR content that needs to be superimposed and displayed based on the acquired AR interaction information, and superimposes the corresponding AR content on the user device in combination with the calculated spatial posture and position information. At the same time, it detects the user's interaction operations with the AR content and updates the AR content display in real time. The types of AR content include 3D models, animations, videos, audio, text information, etc.

[0186] In the present invention, different encoding algorithms can be used to generate invisible QR codes, cameras of other bands can be used for image capture, or different image processing techniques can be used to enhance QR code images; similarly, the spatial positioning algorithm can be optimized or replaced according to the specific application scenario, and the presentation method of AR content can also be adjusted according to the characteristics of the user's device.

[0187] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0188] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for AR interaction and spatial positioning of a three-dimensionally covered invisible QR code, characterized in that: The method comprises the following steps: Step S100: Encode the preset AR interaction information to generate an invisible QR code; Step S200: printing the invisible two-dimensional code on multiple surfaces or at different angles of the target physical object; Step S300: using a camera to scan the target physical object to obtain a plurality of invisible two-dimensional code images, and processing the plurality of invisible two-dimensional code images to obtain AR interaction information; Step S400: Analyze the relative positions and directions of multiple invisible two-dimensional code images, and calculate the spatial posture and position information of the target physical object; Step S500: Based on the acquired AR interaction information and the calculated spatial posture and position information, the corresponding AR content is superimposed on the user device, and the interaction operation between the user and the AR content is detected and the AR content display is updated in real time.

2. The AR interaction and spatial positioning method of a three-dimensionally covered invisible two-dimensional code according to claim 1 is characterized by: The process of encoding the AR interaction information into an invisible QR code in step S100 includes: Step 101: Convert AR interaction information into a binary data stream; Step 102: Generate a two-dimensional code matrix from the binary data stream according to a preset coding rule; Step 103: Replace the black and white modules in the two-dimensional code matrix with infrared absorption or reflection materials to generate an invisible two-dimensional code pattern.

3. The AR interaction and spatial positioning method of a three-dimensionally covered invisible two-dimensional code according to claim 1 is characterized by: In step S200, the generated invisible two-dimensional code is printed on multiple surfaces or at different angles of the target physical object using ink.

4. The AR interaction and spatial positioning method of a three-dimensionally covered invisible two-dimensional code according to claim 1, characterized in that: In step S300, a camera is used to scan the target physical object to obtain multiple invisible two-dimensional code images, and the camera is a near-infrared camera; The processing of the multiple invisible two-dimensional code images includes the following specific processing steps: image enhancement and decoding. The image enhancement process includes: Step 301: grayscale the captured invisible two-dimensional code image; Step 302: binarize the image using an adaptive threshold segmentation algorithm; Step 303: Apply morphological operations to remove noise and fill holes; Step 304: Perform perspective correction to obtain a standard front view image; The decoding process includes: locating the QR code area in the standard front view image, reading the binary matrix data in the area, applying the QR code decoding algorithm to parse the data, and verifying and parsing the AR interaction information.

5. The AR interaction and spatial positioning method of a three-dimensionally covered invisible two-dimensional code according to claim 1, characterized in that: In step S400, the spatial posture and position information of the object are calculated by the following method: Step S401: analyzing the distortion degree of multiple QR codes and estimating the tilt angle of each QR code relative to the camera, specifically including the steps of image acquisition, image preprocessing, QR code positioning, feature point extraction, distortion analysis, perspective transformation, angle calculation, multi-source fusion, and time series filtering; The image acquisition uses a near-infrared camera to capture a high-resolution image containing the invisible QR code; The image preprocessing includes performing denoising, contrast enhancement, and image binarization conversion on the collected high-resolution images; The two-dimensional code positioning adopts an edge detection algorithm based on Hough transform; The feature point extraction uses a Harris corner detector to extract the coordinates of the four corner points of the QR code and calculate the coordinates of the center point of the QR code; The distortion analysis calculates the deviation between the quadrilateral formed by the four corner points and the standard square, and measures the diagonal length ratio and the adjacent side length ratio; The perspective transformation constructs a 3x3 perspective transformation matrix H, satisfying H*[x,y,1]^T=[X,Y,1]^T, where (x,y) is the distorted image coordinate, (X,Y) is the coordinate corresponding to the ideal square, and T is the transposition symbol; The angle calculation performs SVD decomposition on the matrix H, extracts the rotation matrix R, and converts R into Euler angles (α, β, γ): α = arctan(R32 / R33); β = arcsin (-R31); γ = arctan(R21 / R11); The multi-source fusion, when multiple QR codes are detected, uses a weighted average method to fuse the inclination results; The time series filtering uses an extended Kalman filter (EKF), the state vector includes a position and an angle, and the vector is the angle value calculated for the current frame; Step S402: Establishing a spatial coordinate system of the object based on the relative position relationship of the multiple two-dimensional codes: First, select the QR code with the highest definition as the origin, and then determine the x, y, and z axis directions of the coordinate system based on the positions of other QR codes relative to the origin; Step S403: using the triangulation principle to calculate the precise spatial position and posture of the object relative to the camera, specifically including the following steps: Solve the PnP (Perspective-n-Point) problem based on the known camera intrinsic parameter matrix and the actual size of the QR code; The RANSAC algorithm is used to improve the solution accuracy, and the number of iterations is not less than m times, where m>=100; Output the 6 degrees of freedom posture of the object relative to the camera, including three spatial position parameters x, y, z and three spatial direction parameters roll, pitch, yaw.

6. The AR interaction and spatial positioning system with three-dimensional coverage of invisible QR code is characterized by: The system includes an invisible QR code module, an AR recognition module, a spatial positioning module, and an AR interaction module; The invisible QR code module is used to encode the preset AR interaction information to generate invisible QR codes and print them on multiple surfaces or different angles of the physical object; The AR recognition module is used to scan the target physical object through the camera, capture and decode multiple invisible QR codes at the same time, and obtain AR interaction information; The spatial positioning module is used to calculate the spatial posture and position information of the object by analyzing multiple invisible two-dimensional codes covered in three dimensions; The AR interaction module overlays corresponding AR content on the user device according to the acquired AR interaction information and the calculated spatial posture and position information, and detects the interaction operation between the user and the AR content and updates the AR content display in real time.

7. The AR interaction and spatial positioning system with three-dimensional cover invisible two-dimensional code according to claim 6 is characterized by: The invisible two-dimensional code generation module includes an information encoding unit, an invisible processing unit, and a printing unit; The information encoding unit is used to convert the AR interaction information into a binary data stream and generate a two-dimensional code matrix; The invisible processing unit is used to replace the black and white modules in the two-dimensional code matrix with infrared absorption or reflection materials; The printing unit is used to print the processed invisible two-dimensional code pattern on multiple surfaces or different angles of the physical object.

8. The AR interaction and spatial positioning system with three-dimensionally covered invisible two-dimensional code according to claim 6, characterized in that: The AR recognition module is used to simultaneously scan multiple surfaces of the target physical object through a camera, capture multiple invisible two-dimensional code images and process them to obtain AR interaction information, and the camera is a near-infrared camera; The processing of the multiple invisible two-dimensional code images includes the following specific processing steps: image enhancement and decoding. The image enhancement process includes: Step 301: grayscale the captured invisible two-dimensional code image; Step 302: binarize the image using an adaptive threshold segmentation algorithm; Step 303: Apply morphological operations to remove noise and fill holes; Step 304: Perform perspective correction to obtain a standard front view image; The decoding process includes: locating the QR code area in the standard front view image, reading the binary matrix data in the area, applying the QR code decoding algorithm to parse the data, and verifying and parsing the AR interaction information.

9. The AR interaction and spatial positioning system with three-dimensional invisible QR code coverage according to claim 6, characterized in that: The spatial positioning module includes a two-dimensional code position analysis unit, a spatial coordinate system establishment unit and a triangulation unit, and calculates the spatial posture and position information of the object by analyzing multiple invisible two-dimensional codes covered in three dimensions; The two-dimensional code position analysis unit estimates the tilt angle of each two-dimensional code relative to the camera by analyzing the degree of distortion of multiple two-dimensional code images, and specifically includes steps including image acquisition, image preprocessing, two-dimensional code positioning, feature point extraction, distortion analysis, perspective transformation, angle calculation, multi-source fusion, and time series filtering; The image acquisition uses a near-infrared camera to capture a high-resolution image containing the invisible QR code; The image preprocessing includes performing denoising, contrast enhancement, and image binarization conversion on the collected high-resolution images; The two-dimensional code positioning adopts an edge detection algorithm based on Hough transform; The feature point extraction uses a Harris corner detector to extract the coordinates of the four corner points of the QR code and calculate the coordinates of the center point of the QR code; The distortion analysis calculates the deviation between the quadrilateral formed by the four corner points and the standard square, and measures the diagonal length ratio and the adjacent side length ratio; The perspective transformation constructs a 3x3 perspective transformation matrix H, satisfying H*[x,y,1]^T=[X,Y,1]^T, where (x,y) is the distorted image coordinate, (X,Y) is the coordinate corresponding to the ideal square, and T is the transposition symbol; The angle calculation performs SVD decomposition on the matrix H, extracts the rotation matrix R, and converts R into Euler angles (α, β, γ): α = arctan(R32 / R33); β = arcsin (-R31); γ = arctan(R21 / R11); The multi-source fusion, when multiple QR codes are detected, uses a weighted average method to fuse the inclination results; The time series filtering uses an extended Kalman filter (EKF), the state vector includes a position and an angle, and the vector is the angle value calculated for the current frame; The spatial coordinate system establishing unit establishes a spatial coordinate system of the object according to the relative position relationship of the multiple two-dimensional codes; firstly, the two-dimensional code with the highest definition is selected as the origin, and then the x, y, and z axis directions of the coordinate system are determined according to the positions of the other two-dimensional codes relative to the origin; The triangulation unit uses the known geometric relationship between the two-dimensional codes and the camera parameters to calculate the precise spatial position and posture of the object relative to the camera through the triangulation principle, which specifically includes the following steps: Solve the PnP (Perspective-n-Point) problem based on the known camera intrinsic parameter matrix and the actual size of the QR code; The RANSAC algorithm is used to improve the solution accuracy, and the number of iterations is not less than m times, where m>=100; Output the 6 degrees of freedom posture of the object relative to the camera, including three spatial position parameters x, y, z and three spatial direction parameters roll, pitch, yaw.

10. The AR interaction and spatial positioning system with three-dimensional cover invisible two-dimensional code according to claim 6, characterized in that: The AR interaction module determines the specific AR content that needs to be superimposed and displayed based on the acquired AR interaction information, and superimposes the corresponding AR content on the user device in combination with the calculated spatial posture and position information, while detecting the user's interaction operation with the AR content and updating the AR content display in real time.

Images