Mixed reality interactive ranging method in large-scale environment

Abstract

The application relates to a mixed reality interactive ranging method in a large-scale environment, which comprises the following steps: arranging a field control point in reality and recording the field control point in a three-dimensional coordinate system of a virtual space; picking up a reference object and determining the space parameters of the reference object; recognizing a control point where a detection object is located and performing three-dimensional registration; recognizing the point position of the detection object; performing difference between the feature point coordinates of the reference object and the point position coordinates of the recognized detection object, calculating each space vector between each point position of the detection object and the feature point position of the reference object, and storing the straight line distance through the space vector according to a specified direction vector; obtaining the Euclidean distance between each point position of the detection object and the feature point position on the reference object through the space vector; and calculating the space vector to obtain the dot product with the specified direction vector. The application has the advantages that the mixed reality can realize space distance measurement in a large-scale environment, the efficiency of the ranging is remarkably improved, and the measurement accuracy is ensured.

Description

Technical Field

[0001] This invention relates to the field of mixed reality technology, and in particular to a mixed reality interactive ranging method in a large-scale environment. Background Technology

[0002] In the fields of mixed reality (MVR) and augmented reality (AR), existing distance measurement techniques based on spatial coordinate systems are limited by the capabilities of SLAM and mesh recognition, resulting in a limited range of reliable measurements, making them more suitable for measuring centimeter-level objects. For fields such as engineering construction, transportation operations, and warehouse management, MVR technology has gradually become an important tool to support related production work. However, distance measurement in such large-scale environments is characterized by large elevation differences, large spans, or large areas, which conventional MVR or AR measurement methods cannot support. Interactive MVR distance measurement methods in large-scale environments represent an important direction in this field and are a pressing technical problem that needs to be solved. Summary of the Invention

[0003] The purpose of this invention is to address the shortcomings of the prior art by providing a mixed reality interactive ranging method in a large-scale environment. Through mixed reality means, multiple position control point registration and mapping are used to obtain the position information and ranging parameters of the reference object. The position of the object is recorded and connected using a virtual space with sufficient accuracy around the detection object. Finally, the spatial distance in the specified direction is calculated under a unified coordinate system, thereby realizing the spatial distance measurement of mixed reality in a large-scale environment.

[0004] The objective of this invention is achieved through the following technical solutions: A mixed reality interactive ranging method in a large-scale environment includes establishing a virtual space with the same scale as the overall real-world scene. The virtual space contains content elements of the mixed reality scene and a three-dimensional coordinate system. The content elements possess coordinate parameters under the three-dimensional coordinate system. The ranging method, based on the virtual space, includes the following steps: In reality, on-site control points are set up, and these on-site control points are recorded in the three-dimensional coordinate system of the virtual space using three-dimensional coordinates; Pick a reference object and determine the spatial parameters of the reference object. When picking the reference object, the reference object includes the point position on the reference object and the specified direction vector based on the point position. For the detection object, the location control point of the detection object is identified and three-dimensional registration is performed to complete the three-dimensional calibration in the virtual space; The detection object is identified by location; The coordinates of the feature points of the reference object are subtracted from the coordinates of each point of the detected object to calculate the spatial vectors between each point of the detected object and the feature points of the reference object, and then stored according to the specified direction vectors. The linear distance is obtained by taking the modulus of the spatial vector to obtain the Euclidean distance between each point of the detected object and the feature points on the reference object; The distance in a specified direction is obtained by calculating the dot product of the spatial vector and the vector in the specified direction.

[0005] The reference object is picked up by either specifying it in the real space or specifying it in the virtual space; wherein, specifying it in the real space means: selecting the reference object on site, and using a mixed reality device to identify the nearby position control points for 3D registration, thereby realizing the 3D calibration of the reference object in the virtual space; and specifying it in the virtual space means: directly selecting the reference object in the virtual space.

[0006] The spatial parameters of the reference object are determined by single-point picking or multi-point fitting.

[0007] When performing single-point picking, a single point Po is picked for the reference object, and a direction vector v is specified based on this point; when performing multi-point fitting, multi-point picking is performed for the reference object O, its feature points Po are calculated and the direction vector v is specified.

[0008] The detection object's location is picked up using single-point picking, and the detection object's location is continuously identified in a single-point picking manner.

[0009] If the object being detected is within the range of the control point where it is identified in this identification, the picking process continues; if the object being detected is not within the range of the control point where it is identified, the control point where the object is identified is re-identified.

[0010] Record the coordinate values ​​of each point of the detected object within the range of the control point in the three-dimensional coordinate system of the virtual space.

[0011] The advantages of this invention are: it enables spatial distance measurement in large-scale environments using mixed reality, significantly improves the efficiency of such distance measurement, and ensures measurement accuracy. Attached Figure Description

[0012] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a schematic diagram illustrating the principle of the method of the present invention. Detailed Implementation

[0013] The features and other related features of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments, so as to facilitate understanding by those skilled in the art: Example: Figure 1 and Figure 2 As shown, the mixed reality interactive ranging method in a large-scale environment in this embodiment includes the following steps: Step 0: Creating a Virtual Space The basic condition for this embodiment is to establish a virtual space with the same scale as the real scene. This space includes the content elements of the mixed reality scene and a three-dimensional coordinate system. That is, any element in the space (including points, lines, surfaces, volumes, etc.) has coordinate parameters under this coordinate system.

[0014] Step 1: On-site control point layout: In reality, on-site control points are set up, and these control points are recorded in the virtual space three-dimensional coordinate system to which step 0 belongs in the form of three-dimensional coordinates.

[0015] Step 2, Reference object picking: Option 1 (Specified Reference Object in Real Space): For the reference object O selected on site, the nearby control points are identified by the mixed reality device for 3D registration, and the 3D calibration of personnel in the virtual space established in step 0 is completed.

[0016] Option 2 (Specify reference object in virtual space): Directly select reference object O in the virtual space established in step 0.

[0017] In this embodiment, Option 1 is suitable for situations where the reference object can be specified on-site. That is, the reference object can be any point that can be seen or marked on-site, representing the most flexible approach and adaptable to any situation. Option 2, compared to Option 1, reduces the work of specifying the reference object on-site. Instead, it directly specifies the point in virtual space. This facilitates multiple comparisons with a pre-selected, fixed object, making it more convenient than Option 1. During use, technicians can choose between the two options based on the actual situation.

[0018] Step 3: Determine based on spatial parameters: Option 1 (Single-point picking): Perform single-point picking on the reference object O, taking the point Po, and specifying the direction vector v based on this point. Option 2 (multi-point fitting): Pick multiple points on the reference object O and fit and calculate its feature points Po (the calculation method is not limited to calculating the average value, calculating based on the probability distribution, etc.), and specify the direction vector v.

[0019] In this embodiment, in Option 1, only one point needs to be selected for comparison, provided that the actual requirements are met; this is simple and quick. However, in Option 2, for situations where the reference object is complex or the environmental conditions are poor, several points can be selected for fitting, improving the accuracy of the reference object.

[0020] Step 4: Identify control points near the detection object: For the detection object Tn (n=1……m), identify the location control point of the detection object, perform three-dimensional registration, and complete the three-dimensional calibration of personnel in the virtual space established in step 0.

[0021] Step 5: Pick up the detection object points: Single-point picking is performed on the detection object Tn, and the selected point is Pn.

[0022] Step 6: Continuously pick up the objects to be detected: Perform the detection object picking until the picking is completed: If the detection object is within the range of the identification control point in this operation, repeat steps 5-6; if the detection object is not within the range of the identification control point in this operation, repeat steps 4-6.

[0023] Step 7: Record the location of the detected object: Record the coordinate values ​​of P1, ..., Pn in the virtual space coordinate system established in step 0.

[0024] Step 8: Parameter preprocessing in virtual space: The Po coordinates obtained in step 3 and the P1, ..., Pn coordinates obtained in step 7 are subtracted to calculate the spatial vectors po1, ..., pon between the detected object points P1, ..., Pn and the reference object point Po. The spatial vectors po1, ..., pon and the direction vector v specified in step 3 are then stored.

[0025] Step 9, Distance Calculation: The straight-line distance can be obtained by taking the modulus of vectors po1, ..., pon to obtain the Euclidean distances Do1, ..., Don between each detection point and the reference point.

[0026] Given a distance (e.g., projected distance, elevation difference, etc.) in a specified direction, calculate the dot product of vectors po1, ..., pon with vector v to obtain Vo1, ..., Von.

[0027] In this embodiment, taking the measurement of the height difference between two points on a large-span object as an example, the steps are as follows: Step 0: Creating a Virtual Space A virtual space with the same scale as the real scene is established. Content elements for mixed reality scene presentation are created in the space as needed. The normal vector of the geoid is used as the z-axis of the three-dimensional coordinate system. The x-axis and y-axis can be orthogonal on the geoid. That is, any element in the space (including points, lines, surfaces, volumes, etc.) has coordinate parameters under this coordinate system.

[0028] Step 1: On-site control point layout: In reality, on-site control points are set up, and these control points are recorded in the virtual space three-dimensional coordinate system to which step 0 belongs in the form of three-dimensional coordinates.

[0029] Step 2, Reference object picking: For the selected reference object O on site, the nearby control points are identified and registered in three dimensions using mixed reality equipment, thus completing the three-dimensional calibration of personnel in the virtual space established in step 0.

[0030] Step 3: Determine based on spatial parameters: Multiple points are picked up for the reference object O. The point formed by the arithmetic mean of the coordinates of the point set is denoted as O'. The distance from each point in the point set to O' is calculated. The distance is controlled to be less than the given control parameter k for initial screening. The arithmetic mean of the point set after initial screening is denoted as point O'', which is used as the feature point coordinates of the reference object. The normal vector of the geoid is used as the direction vector v.

[0031] Step 4: Identify control points near the detection object: For the detection object Tn (n=1……m), identify the location control point of the detection object, perform three-dimensional registration, and complete the three-dimensional calibration of personnel in the virtual space established in step 0.

[0032] Step 5: Pick up the detection object points: Single-point picking is performed on the detection object Tn, and the selected point is Pn.

[0033] Step 6: Continuously pick up the objects to be detected: Perform the detection object picking until the picking is completed: If the detection object is within the range of the identification control point in this operation, repeat steps 5-6; if the detection object is not within the range of the identification control point in this operation, repeat steps 4-6.

[0034] Step 7: Record the location of the detected object: Record the coordinate values ​​of P1, ..., Pn in the virtual space coordinate system established in step 0.

[0035] Step 8: Parameter preprocessing in virtual space: The Po coordinates obtained in step 3 and the P1, ..., Pn coordinates obtained in step 7 are subtracted to calculate the spatial vectors po1, ..., pon between the detected object points P1, ..., Pn and the reference object point Po. The spatial vectors po1, ..., pon and the direction vector v specified in step 3 are then stored.

[0036] Step 9, Distance Calculation: The straight-line distances can be obtained by taking the modulus of vectors po1, ..., pon to obtain the Euclidean distances Do1, ..., Don between each detection point and the reference point. Given a distance in a specified direction (e.g., projected distance, elevation difference, etc.), calculate the dot product of vectors po1, ..., pon with vector v to obtain Vo1, ..., Von. Since the normal vector of the geoid is directly used as the direction vector v, this distance is the elevation difference between the large-span reference object and the detected object.

[0037] Although the above embodiments have described the concept and embodiments of the present invention in detail with reference to the accompanying drawings, those skilled in the art will recognize that various improvements and modifications can still be made to the present invention without departing from the scope of the claims, and therefore will not be elaborated here.

Claims

1. A mixed reality interactive ranging method in a large-scale environment, comprising establishing a virtual space with the same scale as the overall real-world scene, wherein the virtual space includes content elements of a mixed reality scene and a three-dimensional coordinate system, and the content elements possess coordinate parameters under the three-dimensional coordinate system, characterized in that: Based on the virtual space, the ranging method includes the following steps: In reality, on-site control points are set up, and these on-site control points are recorded in the three-dimensional coordinate system of the virtual space using three-dimensional coordinates; A reference object is picked up and its spatial parameters are determined. The reference object is picked up by including the points on the reference object and the specified direction vector based on the points. The spatial parameters of the reference object are determined by single-point picking or multi-point fitting. When single-point picking is performed, a single point is picked up on the reference object and a direction vector is specified based on the point. When multi-point fitting is performed, multiple points are picked up on the reference object, its feature points are fitted and calculated, and a direction vector is specified. For the detection object, the location control point of the detection object is identified and three-dimensional registration is performed to complete the three-dimensional calibration in the virtual space; The detection object is identified by location; The coordinates of the feature points of the reference object are subtracted from the coordinates of each point of the detected object to calculate the spatial vectors between each point of the detected object and the feature points of the reference object, and stored according to the specified direction vectors. The linear distance is obtained by taking the modulus of the spatial vector to obtain the Euclidean distance between each point of the detected object and the feature points on the reference object; The distance in a specified direction is obtained by calculating the dot product of the spatial vector and the vector in the specified direction; The detection object's location is picked up using single-point picking, and the detection object's location is continuously identified in a single-point picking manner; If the object being detected is within the range of the control point where it is identified in this identification, then the single-point picking will continue to be repeated; if the object being detected is not within the range of the control point where it is identified in this identification, then the control point where the object is located will be re-identified.

2. The mixed reality interactive ranging method in a large-scale environment according to claim 1, characterized in that: The reference object is picked up by either specifying it in the real space or specifying it in the virtual space; wherein, specifying it in the real space means: selecting the reference object on site, and using a mixed reality device to identify the nearby position control points for 3D registration, thereby realizing the 3D calibration of the reference object in the virtual space; and specifying it in the virtual space means: directly selecting the reference object in the virtual space.

3. The mixed reality interactive ranging method in a large-scale environment according to claim 1, characterized in that: Record the coordinate values ​​of each point of the detected object within the range of the control point in the three-dimensional coordinate system of the virtual space.

Images