Augmented reality image positioning display method

Abstract

The invention relates to the technical field of image positioning, in particular to an augmented reality image positioning display method. The method comprises the following steps: S1, selecting a visual mark, and capturing a scene image and a video stream through a camera module; s2, detecting a predefined mark in the image by using a computer vision algorithm; s3, estimating the posture of the camera relative to the mark according to the known size and the position of the camera in the image; s4, converting the 3D model to a world coordinate system by using the estimated camera attitude parameters; s5, drawing the images into a current view by using a graphic rendering engine Unity so as to ensure that the images are kept at correct relative positions with marks in an actual environment; and S6, combining the rendered virtual content with the original video stream to form a real image and output the real image. A reliable reference point is provided through the visual mark, so that the system can quickly and accurately identify a specific position in a complex environment.

Description

Technical Field

[0001] The present invention relates to the field of image positioning technology, and in particular to an augmented reality image positioning display method. Background Art

[0002] The augmented reality (AR) image positioning display method refers to the process of accurately superimposing computer-generated virtual information onto the real-world view seen by the user through a series of complex software and hardware technologies. This method first relies on advanced computer vision technology, using cameras to capture real-time environmental images, and analyzing scene structures through algorithms such as feature point detection, matching and tracking to determine the exact location of virtual content in physical space. At the same time, with the help of simultaneous positioning and mapping (SLAM) technology, the system can dynamically understand and locate its position relative to the surrounding environment without pre-building a map, thereby ensuring that virtual objects can be stably and naturally integrated into the real scene. In order to provide a more realistic and interactive experience, this method will also integrate 3D modeling and rendering technology to adjust the appearance of virtual objects according to the captured ambient lighting conditions, making them look like they actually exist in the environment. In addition, by fusing data from multiple sensors such as accelerometers, gyroscopes and GPS, positioning accuracy and response speed can be further improved, and virtual content can be properly aligned even when moving. Human-computer interaction technology allows users to interact with virtual elements through gestures, voice or touch, enhancing immersion and user experience.

[0003] In the existing augmented reality image positioning and display methods, changes in ambient lighting conditions, lack of complex textures, or rapid movement in dynamic scenes may cause failures in feature point detection and tracking, thereby affecting the stability and accuracy of virtual content; at the same time, the positioning accuracy that relies on sensors such as GPS is often unsatisfactory in indoor or urban environments with tall buildings, which limits the wide applicability of AR applications. In addition, long-term use of AR devices may cause visual fatigue or discomfort to users, because current display technology is difficult to fully simulate natural visual experience, and hardware devices such as head-mounted displays may affect user experience due to weight and wearing comfort issues. Finally, network latency and bandwidth limitations can also affect the real-time performance of AR applications based on cloud processing, especially when large amounts of data transmission or complex calculations are required. As a result, the existing AR image positioning and display methods cannot provide ideal results in some application scenarios. Therefore, an augmented reality image positioning and display method is provided. Summary of the invention

[0004] The purpose of the present invention is to provide an augmented reality image positioning and display method to solve the problem raised in the above background technology that in an existing augmented reality image positioning and display method, changes in ambient lighting conditions, lack of complex textures, or rapid movement in dynamic scenes may cause failures in feature point detection and tracking, thereby affecting the stability and accuracy of virtual content; at the same time, the positioning accuracy that relies on sensors such as GPS is often unsatisfactory in indoor or urban environments with tall buildings, which limits the wide applicability of AR applications. In addition, long-term use of AR devices may cause visual fatigue or discomfort to users, because current display technology is difficult to fully simulate natural visual experience, and hardware devices such as head-mounted displays may affect user experience due to weight and wearing comfort issues. Finally, network latency and bandwidth limitations can also affect the real-time performance of AR applications based on cloud processing, especially when large amounts of data transmission or complex calculations are required. This makes the existing AR image positioning and display methods unable to provide ideal effects in some application scenarios.

[0005] To achieve the above object, the present invention provides an augmented reality image positioning display method, comprising the following steps: S1, select visual markers and capture scene images and video streams through the camera module; S2, detect pre-defined markers in the image using computer vision algorithms; S3, estimate the pose of the camera relative to the marker based on its known size and position in the image; S4, using the estimated camera posture parameters, converting the 3D model to the world coordinate system; S5. Use the graphics rendering engine Unity to draw them into the current view to ensure that they maintain the correct relative position with the markers in the actual environment; S6. Merge the rendered virtual content with the original video stream to form a real image output.

[0006] As a further improvement of the technical solution, in S1, the specific steps of selecting a visual marker and capturing a scene image and a video stream through a camera module are as follows: S1.1. Select a visual marker; Among them, visual markers are easily detectable and have unique characteristics that are used to distinguish them from other elements in the environment; S1.2 Ensure that visual markers are fixed in the environment to maintain their effectiveness as reference points; S1.3. Use the camera module to capture real-time video stream to ensure that the camera can cover the area where the visual marker is located.

[0007] As a further improvement of the technical solution, in S2, a computer vision algorithm is used to detect a predefined marker in the image. S2.1, apply image preprocessing techniques to reduce noise and improve the accuracy of subsequent processing; S2.2, using normalized cross-correlation template matching algorithm to identify visual markers; S2.3. Calculate the bounding box and contour of the marker to determine its location in the image.

[0008] As a further improvement of the technical solution, in S2.1, the image preprocessing technology includes grayscale and Gaussian filtering; The grayscale expression is: ; in, Indicates the grayscale image located at The pixel value of the position; Indicates that the red channel in the color image is The pixel value of the position; Indicates that the green channel in the color image is The pixel value of the position; Indicates that the blue channel in the color image is The pixel value of the position; The Gaussian filter expression is: ; in, Indicates that the Gaussian filter The value of the position; represents the standard deviation of the Gaussian function; represents the base of natural logarithms; ; in, Indicates that the image after Gaussian filtering is located at The pixel value at the location.

[0009] As a further improvement of the technical solution, in S2.2, the expression involved in identifying the visual marker using the normalized cross-correlation template matching algorithm is: ; in, represents the output value of normalized cross-correlation; Indicates the position offset of the template image in the image to be matched; represents the pixel value in the image to be matched, Represents the coordinate position in the image; Indicates that the image A sub-region centered at , the size of which is the same as the template; represents the pixel value in the template image, Indicates the coordinate position in the template; Represents all pixel values ​​in the template image; Indicates that the image The average pixel value of the sub-region centered at ; Represents the average pixel value of the template image; For all pixel positions Perform summation; Represents the row index in the image; Represents the column index in the image.

[0010] As a further improvement of the technical solution, in S3, the specific steps of estimating the posture of the camera relative to the marker according to its known size and position in the image are: S3.1, collect the 3D corner point coordinates of the marker in the world coordinate system and their 2D projection point coordinates in the image; Among them, the 3D corner point coordinates in the world coordinate system for: ; in, Indicates the mark The coordinates of the corner points in the world coordinate system; represents homogeneous coordinates; Represents an index variable; 2D projected point coordinates in the image for: ; in, Indicates the mark The pixel coordinates of the corner points in the image; S3.2. Use the pre-calibrated camera parameters to obtain the intrinsic parameter matrix including focal length and principal point offset ; Among them, the internal parameter matrix for: Get the intrinsic matrix from the camera calibration process , which contains the focal length and And the principal point offset and ; ; in, represents the internal parameter matrix; Representing images focal length of direction; Representing images focal length of direction; Indicates that the main point is Directional deviation; Indicates that the main point is Directional deviation; S3.3, establish a mathematical relationship from 3D world coordinates to 2D image coordinates; Among them, according to the perspective projection model, the 3D world coordinates are established to 2D image coordinates The mathematical relationship between ; in, represents the scaling factor; represents the rotation matrix; represents the translation vector; S3.4. Solving the rotation matrix using iterative optimization method and translation vectors ; S3.5, calculating the difference between the 3D point after being projected onto the image plane through the currently estimated camera pose and the actual observed 2D point; The process of calculating the difference between the 3D point after being projected onto the image plane by the currently estimated camera pose and the actual observed 2D point is: For each 3D point , using the currently estimated rotation matrix R and translation vector t, calculate its reprojection point in the image ; ; in, represents the reprojected point in the image; Represents a perspective projection operation; Calculate reprojection points The actual observation point The Euclidean distance between them is used as the reprojection error; ; in, represents the reprojection error; S3.6. Output the rotation matrix of the camera relative to the marker and translation vectors .

[0011] As a further improvement of the present technical solution, in S3.4, an iterative optimization method is used to solve the rotation matrix and translation vectors The specific process is: Define the reprojection error function , which represents the difference between the 3D point after being projected onto the image plane by the currently estimated camera pose and the actual observed 2D point; Use an iterative optimization algorithm to minimize the reprojection error ; The resulting expression is: ; in, Indicates the number of points.

[0012] As a further improvement of the technical solution, in S4, the specific steps of converting the 3D model to the world coordinate system using the estimated camera posture parameters are: S4.1, converting vertex coordinates of the 3D model from a model coordinate system to a world coordinate system; S4.2. Apply the exchange method to adapt the 3D model to the actual environment.

[0013] As a further improvement of the technical solution, in S5, the specific steps of using the graphics rendering engine Unity to draw them into the current view are: S5.1. Create a scene in Unity, import the 3D model and set the properties; S5.2, updating the transformation component of the model according to the calculated position of the 3D model in the world coordinate system; S5.3, adjusting camera parameters to match the camera perspective in the real world; S5.4. Implement AR rendering logic to ensure that virtual objects are seamlessly integrated with the real environment.

[0014] As a further improvement of the technical solution, in S6, the specific steps of merging the rendered virtual content with the original video stream to form a real image output are: S6.1, obtaining the original image of each frame from the camera; S6.2. Render virtual content from the same perspective in Unity; S6.3, using Alpha blending to overlay virtual content onto the original image; S6.4. Output the AR image to the display screen.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. In this augmented reality image positioning display method, a reliable reference point is provided by a visual marker, so that the system can quickly and accurately identify a specific location in a complex environment. Template matching can stably detect markers under different lighting conditions and viewing angles, improving the robustness and adaptability of the system. The position and direction of the camera relative to the marker can be accurately calculated through the PnP problem solving estimation method.

[0016] 2. In this method of positioning and displaying augmented reality images, by converting the 3D model from the model coordinate system to the world coordinate system, the virtual object can be accurately placed at a specific location in the real world. The graphics rendering engine Unity provides powerful tools and flexible APIs to efficiently render complex 3D models and achieve real-time updates. Using techniques such as Alpha blending to smoothly overlay virtual content onto the original video stream ensures the visual consistency of the final output image. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The figure is a flow chart of the overall method of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Example: See also Figure 1 As shown, this embodiment provides an augmented reality image positioning display method, comprising the following steps: S1, select visual markers and capture scene images and video streams through the camera module; In this example, the specific steps involved in selecting visual markers and capturing scene images and video streams through the camera module are: S1.1. Select a visual marker; Among them, visual markers are easily detectable and have unique characteristics that are used to distinguish them from other elements in the environment; S1.2 Ensure that visual markers are fixed in the environment to maintain their effectiveness as reference points; S1.3. Use the camera module to capture real-time video stream to ensure that the camera can cover the area where the visual marker is located.

[0020] S2, detect pre-defined markers in the image using computer vision algorithms; In this example, a computer vision algorithm is used to detect pre-defined markers in an image. S2.1, apply image preprocessing techniques to reduce noise and improve the accuracy of subsequent processing; In this example, the image preprocessing techniques include grayscale and Gaussian filtering; The grayscale expression is: ; in, Indicates the grayscale image located at The pixel value of the position; Indicates that the red channel in the color image is The pixel value of the position; Indicates that the green channel in the color image is The pixel value of the position; Indicates that the blue channel in the color image is The pixel value of the position; The Gaussian filter expression is: ; in, Indicates that the Gaussian filter The value of the position; represents the standard deviation of the Gaussian function; represents the base of natural logarithms; ; in, Indicates that the image after Gaussian filtering is located at The pixel value at the location.

[0021] Specifically, grayscale reduces the amount of data and simplifies the complexity of subsequent processing by converting color images into grayscale images, while retaining the key structural information of the image. Gaussian filtering is used to smooth images, reduce noise and prevent excessive loss of edge information, which helps to improve the accuracy and stability of feature detection and matching. Images processed by grayscale and Gaussian filtering can make the recognition of visual markers more robust, especially in the case of lighting changes or slight occlusions, thereby ensuring that virtual content can be accurately and stably superimposed on specific locations in the real world, improving the quality of the user's AR experience.

[0022] S2.2, using normalized cross-correlation template matching algorithm to identify visual markers; In this example, the expression involved in identifying the visual marker using the normalized cross-correlation template matching algorithm is: ; in, represents the output value of normalized cross-correlation; Indicates the position offset of the template image in the image to be matched; represents the pixel value in the image to be matched, Represents the coordinate position in the image; Indicates that the image A sub-region centered at , the size of which is the same as the template; represents the pixel value in the template image, Indicates the coordinate position in the template; Represents all pixel values ​​in the template image; Indicates that the image The average pixel value of the sub-region centered at ; Represents the average pixel value of the template image; For all pixel positions Perform summation; Represents the row index in the image; Represents the column index in the image.

[0023] Specifically, the best matching point is found by calculating the similarity between the template image and the sub-regions of each possible position in the image to be matched. This similarity is quantified, where a high value indicates a high similarity, that is, the template and the image sub-region are highly consistent. This enables the system to accurately identify predefined markers in complex and changing environments, thereby providing reliable reference points for subsequent virtual content overlays, ensuring that virtual objects can be stably and correctly attached to specific locations in the real world, thereby enhancing the realism and interactivity of the user's AR experience.

[0024] S2.3. Calculate the bounding box and contour of the marker to determine its location in the image.

[0025] S3, estimate the pose of the camera relative to the marker based on its known size and position in the image; In this example, the specific steps to estimate the pose of the camera relative to the marker based on its known size and position in the image are: S3.1, collect the 3D corner point coordinates of the marker in the world coordinate system and their 2D projection point coordinates in the image; Among them, the 3D corner point coordinates in the world coordinate system for: ; in, Indicates the mark The coordinates of the corner points in the world coordinate system; represents homogeneous coordinates; Represents an index variable; 2D projected point coordinates in the image for: ; in, Indicates the mark The pixel coordinates of the corner points in the image; S3.2. Use the pre-calibrated camera parameters to obtain the intrinsic parameter matrix including focal length and principal point offset ; Among them, the internal parameter matrix for: Get the intrinsic matrix from the camera calibration process , which contains the focal length and And the principal point offset and ; ; in, represents the internal parameter matrix; Representing images focal length of direction; Representing images focal length of direction; Indicates that the main point is Directional deviation; Indicates that the main point is Directional deviation; S3.3, establish a mathematical relationship from 3D world coordinates to 2D image coordinates; Among them, according to the perspective projection model, the 3D world coordinates are established to 2D image coordinates The mathematical relationship between ; in, represents the scaling factor; represents the rotation matrix; represents the translation vector; S3.4. Solving the rotation matrix using iterative optimization method and translation vectors ; In this example, an iterative optimization method is used to solve the rotation matrix and translation vectors The specific process is: Define the reprojection error function , which represents the difference between the 3D point after being projected onto the image plane by the currently estimated camera pose and the actual observed 2D point; Use an iterative optimization algorithm to minimize the reprojection error ; The resulting expression is: ; in, Indicates the number of points.

[0026] Specifically, an iterative optimization method is used to solve the rotation matrix and translation vectors The specific process is to minimize the reprojection error function , which quantifies the difference between the 3D point after being projected onto the image plane by the currently estimated camera pose and the actual observed 2D point. By minimizing this error, the position and orientation of the camera relative to the marker can be accurately determined. This process ensures that virtual content can be superimposed on the specified location in the real world with the highest accuracy, and remains stable and accurate even when the user moves the device or environmental conditions change. The resulting rotation matrix and translation vector provide the necessary transformation parameters for the rendering of virtual objects, greatly improving the realism and interactivity of the AR experience, and is one of the key technologies for achieving high-quality, real-time AR applications.

[0027] S3.5, calculating the difference between the 3D point after being projected onto the image plane through the currently estimated camera pose and the actual observed 2D point; The process of calculating the difference between the 3D point after being projected onto the image plane by the currently estimated camera pose and the actual observed 2D point is: For each 3D point , using the currently estimated rotation matrix R and translation vector t, calculate its reprojection point in the image ; ; in, represents the reprojected point in the image; Represents a perspective projection operation; Calculate reprojection points The actual observation point The Euclidean distance between them is used as the reprojection error; ; in, represents the reprojection error; S3.6. Output the rotation matrix of the camera relative to the marker and translation vectors .

[0028] Specifically, the rotation matrix for: ; in, Indicates the rotation information of the camera; Translation Vector for: ; in, Represents the translation information of the camera relative to the marker.

[0029] Specifically, by collecting the 3D corner coordinates of the marker in the world coordinate system and their 2D projection point coordinates in the image, and combining them with the pre-calibrated camera intrinsic parameter matrix, a mathematical relationship from 3D world coordinates to 2D image coordinates can be established. The rotation matrix RR and translation vector tt are solved using an iterative optimization method to accurately describe the position and orientation of the camera relative to the marker. The reprojection error is calculated and continuously optimized to ensure the accuracy of the estimation results. The final output rotation matrix and translation vector provide the necessary transformation parameters for subsequent virtual content rendering, so that virtual objects can be stably and accurately attached to the marker, and maintain the correct relative position even when the user moves the device, thereby enhancing the realism and interactivity of the AR experience.

[0030] S4, using the estimated camera posture parameters, converting the 3D model to the world coordinate system; In this example, the specific steps of converting the 3D model to the world coordinate system using the estimated camera pose parameters are as follows: S4.1, converting vertex coordinates of the 3D model from a model coordinate system to a world coordinate system; S4.2. Apply the exchange method to adapt the 3D model to the actual environment.

[0031] Specifically, the exchange methods include scaling, rotation, translation, etc.; Scaling: Adjust the size of the 3D model to make it appear to be in proportion to its surroundings.

[0032] Rotation: Use a rotation matrix to adjust the orientation of a 3D model to match the perspective of a marker or user.

[0033] Translation: Adjust the position of the 3D model by the translation vector to ensure it is placed correctly in the intended location.

[0034] By applying a rotation matrix and a translation vector, each vertex of the 3D model is transformed from its original model coordinate system to a world coordinate system that is aligned with the real world. This transformation ensures that the virtual object is rendered in the correct spatial position, consistent with markers or other references in the real environment.

[0035] S5. Use the graphics rendering engine Unity to draw them into the current view to ensure that they maintain the correct relative position with the markers in the actual environment; Specifically, Unity is a versatile graphics rendering engine widely used in game development and real-time 3D content creation. It provides a powerful toolset and easy-to-use interface, supporting developers to create a variety of interactive experiences from 2D games, 3D games to virtual reality (VR), augmented reality (AR) applications and simulation training. Through its efficient rendering pipeline, physics engine, animation system and cross-platform publishing capabilities, Unity enables developers to quickly build high-quality visual content and easily deploy it to multiple target platforms, including PCs, mobile devices, consoles and web pages. In addition, Unity also has a huge community and rich resource library, which greatly promotes learning and innovation.

[0036] In this example, the specific steps to draw them into the current view using the graphics rendering engine Unity are: S5.1. Create a scene in Unity, import the 3D model and set the properties; S5.2, updating the transformation component of the model according to the calculated position of the 3D model in the world coordinate system; S5.3, adjusting camera parameters to match the camera perspective in the real world; S5.4. Implement AR rendering logic to ensure that virtual objects are seamlessly integrated with the real environment.

[0037] Specifically, the specific steps of using the graphics rendering engine Unity to draw virtual content into the current view ensure that virtual objects can be seamlessly and realistically integrated into the real world. By creating a scene in Unity and importing a 3D model, and updating the model's transformation components according to the calculated position in the world coordinate system, the position, size, and direction of the virtual object in space can be precisely controlled. Adjusting the camera parameters to match the camera perspective of the real world ensures the consistency of the virtual content and the user's perspective. Implementing AR rendering logic further optimizes the fusion effect of virtual objects with the real environment, making them look like a natural part of the real world. This process not only enhances the user's immersion and interactive experience, but also provides solid technical support for building complex and dynamic AR applications. It is a key link in achieving high-quality, real-time AR experience.

[0038] S6. Merge the rendered virtual content with the original video stream to form a real image output.

[0039] In this example, the specific steps of merging the rendered virtual content with the original video stream to form a real image output are: S6.1, obtaining the original image of each frame from the camera; S6.2. Render virtual content from the same perspective in Unity; S6.3, using Alpha blending to overlay virtual content onto the original image; Specifically, Alpha Blending is a technique used in computer graphics to composite two or more images together, where one image is partially transparent. By adjusting the alpha channel value of each pixel (usually ranging from 0 to 1, where 0 is completely transparent and 1 is completely opaque), you can control the degree to which the color of the pixel blends with the background color. Alpha blending allows the creation of smooth transparency effects, shadows, reflections, and other visual effects, and is widely used in video editing, game development, and user interface design to achieve a more realistic and attractive visual experience.

[0040] S6.4. Output the AR image to the display screen.

[0041] Specifically, by acquiring the original image of each frame from the camera and rendering the virtual content in Unity with the same viewing angle, the virtual object is smoothly superimposed on the original image using Alpha blending technology, and finally the fused AR image is output to the display. This process not only achieves a seamless combination of virtual and reality, but also ensures visual consistency, making the virtual object look like a real part of the environment. This greatly enhances the user's immersion and interactive experience, and is a key technical link in achieving high-quality, real-time AR applications, allowing users to enjoy a more realistic and interactive AR experience in various application scenarios.

[0042] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only preferred examples of the present invention, and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. A method for positioning and displaying an augmented reality image, characterized in that: The following steps are involved: S1, select visual markers and capture scene images and video streams through the camera module; S2, detect pre-defined markers in the image using computer vision algorithms; S3, estimate the pose of the camera relative to the marker based on its known size and position in the image; S4, using the estimated camera posture parameters, converting the 3D model to the world coordinate system; S5. Use the graphics rendering engine Unity to draw them into the current view to ensure that they maintain the correct relative position with the markers in the actual environment; S6. Merge the rendered virtual content with the original video stream to form a real image output.

2. The augmented reality image positioning display method according to claim 1, characterized in that: In S1, the specific steps of selecting visual markers and capturing scene images and video streams through a camera module are as follows: S1.

1. Select a visual marker; Among them, visual markers are easily detectable and have unique characteristics that are used to distinguish them from other elements in the environment; S1.2 Ensure that visual markers are fixed in the environment to maintain their effectiveness as reference points; S1.

3. Use the camera module to capture real-time video stream to ensure that the camera can cover the area where the visual marker is located.

3. The augmented reality image positioning display method according to claim 1, characterized in that: In S2, the specific steps of using a computer vision algorithm to detect a predefined marker in an image are: S2.

1. Apply image preprocessing techniques to reduce noise and improve the accuracy of subsequent processing; S2.2, using normalized cross-correlation template matching algorithm to identify visual markers; S2.

3. Calculate the bounding box and contour of the marker and determine its position in the image.

4. The method for positioning and displaying augmented reality images according to claim 3, characterized in that: In S2.1, the image preprocessing technology includes grayscale and Gaussian filtering; The grayscale expression is: ； in, Indicates the grayscale image located at The pixel value of the position; Indicates that the red channel in the color image is The pixel value of the position; Indicates that the green channel in the color image is The pixel value of the position; Indicates that the blue channel in the color image is The pixel value of the position; The Gaussian filter expression is: ； in, Indicates that the Gaussian filter The value of the position; represents the standard deviation of the Gaussian function; represents the base of natural logarithms; ； in, Indicates that the image after Gaussian filtering is located at The pixel value at the location.

5. The method for positioning and displaying augmented reality images according to claim 3, characterized in that: In S2.2, the expression involved in identifying the visual marker using the normalized cross-correlation template matching algorithm is: ； in, represents the output value of normalized cross-correlation; Indicates the position offset of the template image in the image to be matched; represents the pixel value in the image to be matched, Represents the coordinate position in the image; Indicates that the image A sub-region centered at , the size of which is the same as the template; represents the pixel value in the template image, Indicates the coordinate position in the template; Represents all pixel values ​​in the template image; Indicates that the image The average pixel value of the sub-region centered at ; Represents the average pixel value of the template image; For all pixel positions Perform summation; Represents the row index in the image; Represents the column index in the image.

6. The augmented reality image positioning display method according to claim 1, characterized in that: In S3, the specific steps of estimating the posture of the camera relative to the marker based on its known size and position in the image are: S3.1, collect the 3D corner point coordinates of the marker in the world coordinate system and their 2D projection point coordinates in the image; Among them, the 3D corner point coordinates in the world coordinate system for: ； in, Indicates the mark The coordinates of the corner points in the world coordinate system; represents homogeneous coordinates; Represents an index variable; 2D projected point coordinates in the image for: ； in, Indicates the mark The pixel coordinates of the corner points in the image; S3.

2. Use the pre-calibrated camera parameters to obtain the intrinsic parameter matrix including focal length and principal point offset ; Among them, the internal parameter matrix for: Get the intrinsic matrix from the camera calibration process , which contains the focal length and And the principal point offset and ; ； in, represents the internal parameter matrix; Representing images focal length of direction; Representing images focal length of direction; Indicates that the principal point is in the direction The offset; Indicates that the principal point is in the direction The offset; S3.3, establish a mathematical relationship from 3D world coordinates to 2D image coordinates; Among them, according to the perspective projection model, the 3D world coordinates are established to 2D image coordinates The mathematical relationship between ； in, represents the scaling factor; represents the rotation matrix; represents the translation vector; S3.

4. Solving the rotation matrix using iterative optimization method and translation vectors ; S3.5, calculating the difference between the 3D point after being projected onto the image plane through the currently estimated camera pose and the actual observed 2D point; The process of calculating the difference between the 3D point after being projected onto the image plane by the currently estimated camera pose and the actual observed 2D point is: For each 3D point , using the currently estimated rotation matrix R and translation vector t, calculate its reprojected point in the image ; ； in, represents the reprojected point in the image; Represents a perspective projection operation; Calculate reprojection points The actual observation point The Euclidean distance between them is used as the reprojection error; ； in, represents the reprojection error; S3.

6. Output the rotation matrix of the camera relative to the marker and translation vectors .

7. The method for positioning and displaying an augmented reality image according to claim 1, characterized in that: In S3.4, an iterative optimization method is used to solve the rotation matrix and translation vectors The specific process is: Define the reprojection error function , which represents the difference between the 3D point after being projected onto the image plane by the currently estimated camera pose and the actual observed 2D point; Use an iterative optimization algorithm to minimize the reprojection error ; The resulting expression is: ； in, Indicates the number of points.

8. The method for positioning and displaying an augmented reality image according to claim 1, characterized in that: In S4, the specific steps of converting the 3D model to the world coordinate system using the estimated camera posture parameters are as follows: S4.1, converting vertex coordinates of the 3D model from a model coordinate system to a world coordinate system; S4.

2. Apply the exchange method to adapt the 3D model to the actual environment.

9. The method for positioning and displaying an augmented reality image according to claim 1, characterized in that: In the S5, the specific steps of using the graphics rendering engine Unity to draw them into the current view are: S5.

1. Create a scene in Unity, import the 3D model and set the properties; S5.2, updating the transformation component of the model according to the calculated position of the 3D model in the world coordinate system; S5.3, adjusting camera parameters to match the camera perspective in the real world; S5.

4. Implement AR rendering logic to ensure seamless integration of virtual objects with the real environment.

10. The method for positioning and displaying an augmented reality image according to claim 1, characterized in that: In S6, the specific steps of merging the rendered virtual content with the original video stream to form a real image output are: S6.1, obtaining the original image of each frame from the camera; S6.

2. Render virtual content from the same perspective in Unity; S6.3, using Alpha blending to overlay virtual content onto the original image; S6.

4. Output the AR image to the display screen.

Images