Large-scale mixed reality scene registration and precision optimization method based on multi-point control

Abstract

The invention relates to a large-scale mixed reality scene registration and precision optimization method based on multi-point control, and the method comprises the steps: setting two or more than two position control points in a reality scene, obtaining the coordinates of each position control point through mixed reality equipment in a use process of the mixed reality equipment, taking the coordinates as a first coordinate set, and setting a second coordinate set as a third coordinate set; pre-recording the pre-estimated position of each control point by the mixed reality equipment, and regarding the pre-estimated position as a coordinate set II; for each position control point of the first coordinate set, finding the correspondence of each position control point in the second coordinate set, calculating the rigid transformation of each position control point in two different coordinate sets, namely the first coordinate set and the second coordinate set, and then obtaining the rigid transformation of a model in a virtual space through a weighting algorithm; and carrying out weight calculation and iteration, and checking whether rigid transformation is available or not. The method has the advantages that powerful support is provided for development of the mixed reality technology, and more seamless fusion of virtuality and reality is achieved through accurate point set registration and position optimization.

Description

Technical Field

[0001] The present invention relates to the field of mixed reality technology, and in particular to a large-scale mixed reality scene registration and precision optimization method based on multi-point control. Background Art

[0002] Mixed reality technology creates visual illusions by projecting pre-made or dynamically generated content images onto specific virtual space locations, thereby enhancing users' perception of real scenes. Through interaction, it creates an effect of deep enhancement of real physical world information, and has been gradually applied to the industrial and engineering fields. In the process of production and management business applications, the registration of virtual elements with the physical world will be an important prerequisite for effectively, accurately and continuously enhancing personnel's information perception capabilities.

[0003] Currently, in large-scale scenes (such as construction sites, large-scale industrial plants, narrow and long space structures, etc.), various 3D registration and spatial alignment technologies are subject to factors such as changeable site environments and uneven surfaces. They are unable to achieve precision control of large-scale virtual content scenes, especially in distant areas and spatial angles. Large visual errors occur, which seriously affects the research and development applications in such application scenarios.

[0004] Therefore, large-scale mixed reality scene registration and precision optimization in complex external environments is an important direction in the development of industry technology, and it is also a key issue that needs to be urgently solved for the implementation and promotion of mixed reality technology in a large number of on-site application scenarios. Summary of the invention

[0005] The purpose of the present invention is to provide a large-scale mixed reality scene alignment and precision optimization method based on multi-point control based on the above-mentioned deficiencies of the prior art, adopt multiple deployed control points (hereinafter referred to as position control points, including but not limited to artificial feature identification, spontaneous radio frequency signals and other position information interaction methods), and propose a positioning identification and precision optimization method, so as to finally achieve high-precision alignment and presentation of large-scale mixed reality scenes, while ensuring the on-site alignment speed, improve the stability and accuracy of the position control points, and ensure the precise combination of virtual and real scenes.

[0006] The purpose of the present invention is achieved by the following technical solutions:

[0007] A large-scale mixed reality scene registration and precision optimization method based on multi-point control, characterized in that the method comprises the following steps:

[0008] Two or more position control points are set in a real scene. During the use of the mixed reality device, the mixed reality device obtains the coordinates of each control point, which is regarded as a coordinate set 1, and the mixed reality device pre-records the estimated position of each control point, which is regarded as a coordinate set 2;

[0009] For each control point of the coordinate set 1, find the correspondence of each control point in the coordinate set 2, and calculate the rigid transformation of each control point in two different coordinate sets, namely, the coordinate set 1 and the coordinate set 2, and then obtain a rigid transformation of a model in a virtual space through a weighted algorithm;

[0010] Calculate and iterate the weights and check whether the rigid transformation is applicable.

[0011] The coordinate set 1 is preprocessed, and the coordinates of the control points located in the virtual space provided by the mixed reality device are obtained according to different types of control points.

[0012] The correspondence can be performed by nearest point iteration, that is, finding the nearest point of one control point of the coordinate set in the second coordinate set; or matching and adjusting the initial weight by referring to prior knowledge, that is, different points or planes composed of points have higher weights to affect the final rigid transformation result.

[0013] For each point in the coordinate set 2, a set of transformation matrices is calculated for the corresponding point in the coordinate set 1, and then the error is processed, the weights are changed, and the optimal transformation is calculated.

[0014] Whether the rigid transformation is available is determined by setting the prefabrication; if the condition is met, the rigid transformation is considered to be an available transformation, and the transformation is provided to the mixed reality device to guide its alignment; if the condition is not met, the coordinate set two is set to the coordinate set two that has undergone the optimal transformation, the correspondence between it and the coordinate set one is regained, and the rigid transformation is calculated until the threshold is met.

[0015] The advantages of the present invention are: a complete alignment and optimization framework is constructed; through this framework, the system can cope with the complexity of different scenes and provide efficient and accurate point set processing solutions; in general, the present invention provides strong support for the development of mixed reality technology, and through precise point set alignment and position optimization, a more seamless integration of virtual and reality is achieved, providing a cutting-edge technical path for further expansion and improvement of the application field of mixed reality. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of multi-control point registration of the present invention;

[0017] Figure 2 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0018] The features of the present invention and other related features are further described in detail below through embodiments in conjunction with the accompanying drawings to facilitate understanding by those skilled in the art:

[0019] Example: Figure 1 and Figure 2 As shown, the large-scale mixed reality scene registration and accuracy optimization method based on multi-point control in this embodiment includes the following steps:

[0020] Two or more position control points are set in the scene, and the mixed reality device can obtain the coordinates of these position control points as coordinate set 1 (hereinafter referred to as point set 1). The mixed reality device also has a pre-recorded coordinate set 2 (hereinafter referred to as point set 2).

[0021] Next, for each point in point set 1, the system uses the corresponding point in point set 2. This correspondence is the key to the subsequent optimization process. Through this correspondence, the rigid transformation of the model position is obtained to achieve the purpose of model registration and virtual-real combination. Point set registration is committed to aligning point set data sets collected from different perspectives or times so that they can be compared or fused in the same coordinate system. The entire registration process involves multiple steps, each of which performs specific processing and analysis on the point set data to achieve the best registration effect.

[0022] Step 1: Identification and preprocessing of control points, obtaining the coordinates of the control points in the virtual space, treating them as point set 1, and preprocessing point set 1. Obtaining the coordinates of the control points in the virtual space depends on the type of the control points.

[0023] Taking the most common QR code as an example, after the mixed reality device scans the QR code, the distance from the mixed reality device camera to the QR code can be obtained. When maintaining SLAM, the mixed reality space will always calculate the distance from the device camera to the origin of the space (starting point). At this time, the position of the position control point in the virtual space and its corresponding identification number are obtained. After that, the obtained coordinate-number set is preprocessed to remove unusable coordinates that do not meet the spatial logic, do not meet user expectations, are not registered, etc. Such preprocessing helps to improve the accuracy of subsequent steps and ensure the stability of input data quality.

[0024] Step 2: Matching and rigid transformation of position control points. In this stage, the correspondence of each point in point set 1 in point set 2 is found, the rigid transformation between these points is calculated respectively, and then a rigid transformation of a model in virtual space is obtained through a weighted algorithm. This correspondence can use the common nearest point iteration, that is, find the nearest point of point set 1 in point set 2, or refer to prior knowledge for matching and adjusting the initial weight, that is, different points or planes composed of points have higher weights to affect the final rigid transformation result.

[0025] For example, some mixed reality devices can efficiently provide the ground plane where the user is located and its normal vector, and match it with the ground plane stored in the virtual space to eliminate and correct incorrect point correspondences, and can also calculate the plane's rotation matrix and provide a higher weight. Or for some mixed reality devices, the closer the rigid transformation is to the origin of the coordinate system, the more reliable it is.

[0026] Step 3: Calculation and iteration of weights. In this stage, the main purpose is to verify whether the rigid transformation in step 2 is available. In order to quantify the distance between two point sets, a loss function is designed for different mixed reality devices. This loss function indicates that when its value is minimum, the distance between the point sets is minimum. At this time, the solution at the minimum value of the loss function is used as the weight to calculate the rigid transformation matrix at this time. The availability is determined by judging whether the minimum value of the loss function meets the preset threshold. If available, it is submitted to the mixed reality device to complete the alignment. Otherwise, the point set 2 after the rigid transformation is used as the new click 2, and the matching stage and verification stage are repeated until the preset threshold is met or the maximum number of cycles is exceeded.

[0027] The following loss functions are available:

[0028] 1. Mahalanobis distance:

[0029]

[0030] 2. Error square sum algorithm:

[0031]

[0032] Through the cyclic iteration of multiple control points, the system cleverly uses the above-mentioned point set registration and accuracy optimization methods to achieve the gradual optimization of the position of the overall point set model. The information of each control point is fully utilized, and the system gradually adjusts the point set model to make it more accurately fit the actual scene. The cleverness of this iterative process is that by repeatedly performing the steps of registration, weighting, removing outliers, calculating losses, and minimizing losses, the system can improve the registration accuracy and model accuracy in the mixed reality scene in the process of continuous optimization. This method effectively utilizes the information of multiple control points, allowing the system to more finely adjust the point set model when processing large-scale mixed reality scenes, thereby providing a more realistic and accurate mixed reality experience. The flexibility and comprehensiveness of this iterative process lays a solid foundation for mixed reality technology to create a better user experience in practical applications.

[0033] Although the above embodiments have described in detail the concepts and embodiments of the present invention with reference to the accompanying drawings, ordinary technicians in this field can recognize that various improvements and changes can still be made to the present invention without departing from the scope of the claims, so they are not described one by one here.

Claims

1. A large-scale mixed reality scene registration and accuracy optimization method based on multi-point control, characterized by: The method comprises the following steps: Two or more position control points are set in a real scene. During the use of the mixed reality device, the mixed reality device obtains the coordinates of each control point, which is regarded as a coordinate set 1, and the mixed reality device pre-records the estimated position of each control point, which is regarded as a coordinate set 2; For each control point of the coordinate set 1, find the correspondence of each control point in the coordinate set 2, and calculate the rigid transformation of each control point in two different coordinate sets, namely, the coordinate set 1 and the coordinate set 2, and then obtain a rigid transformation of a model in a virtual space through a weighted algorithm; Calculate and iterate the weights and check whether the rigid transformation is applicable.

2. According to claim 1, a large-scale mixed reality scene registration and precision optimization method based on multi-point control is characterized in that: The coordinate set 1 is preprocessed, and the coordinates of the control points located in the virtual space provided by the mixed reality device are obtained according to different types of control points.

3. According to claim 1, a large-scale mixed reality scene registration and precision optimization method based on multi-point control is characterized in that: The correspondence can be performed by nearest point iteration, that is, finding the nearest point of one control point of the coordinate set in the second coordinate set; or matching and adjusting the initial weight by referring to prior knowledge, that is, different points or planes composed of points have higher weights to affect the final rigid transformation result.

4. The large-scale mixed reality scene registration and precision optimization method based on multi-point control according to claim 1 is characterized in that: For each point in the coordinate set 2, a set of transformation matrices is calculated for the corresponding point in the coordinate set 1, and then the error is processed, the weights are changed, and the optimal transformation is calculated.

5. The large-scale mixed reality scene registration and precision optimization method based on multi-point control according to claim 4 is characterized by: By setting a prefabrication, it is determined whether the rigid transformation is available; if the condition is met, the rigid transformation is considered to be an available transformation, and the transformation is provided to the mixed reality device to guide its registration; If the condition is not met, the coordinate set 2 is set to the coordinate set 2 that has undergone the optimal transformation, the corresponding relationship between it and the coordinate set 1 is regained, and the rigid transformation is calculated until the threshold is met.

Images