AR equipment positioning system and method based on data fusion

Abstract

The invention relates to the technical field of equipment positioning, and discloses an AR equipment positioning system and method based on data fusion, and the system comprises an acquisition module, a conventional positioning module and a small space positioning module. The method corresponds to the system. According to the invention, accurate positioning of the AR device in different scenes is realized; the acquisition module collects various data and provides rich information for positioning; the conventional positioning module fuses the sensing and positioning data to generate a conventional positioning result to meet the positioning requirements of general scenes; the small-space positioning module is used for judging according to the spatial data, fusing a conventional positioning result, infrared data and high-frequency data, and improving the precision and reliability of small-space positioning; the problems that an existing AR positioning system is limited in precision and cannot adapt to complex scenes are solved.

Description

Technical Field

[0001] The present application relates to the field of device positioning technology, and specifically to an AR device positioning system and method based on data fusion. Background Art

[0002] Accurate device positioning is crucial for the application of augmented reality (AR) technology. Currently, existing AR positioning systems suffer from numerous issues. Established AR positioning systems, such as the HoloLens 2 and HTC VIVE XR, have limited accuracy. Their absolute position and rotation errors cannot meet the requirements of high-precision applications in military, aerospace, and medical applications. Domestic AR surgical navigation systems are still in their infancy, primarily developed by combining HoloLens with traditional systems. These systems suffer from insufficient computing power, low positioning accuracy, and short battery life.

[0003] Among the published patents, Chinese patent application publication number CN118887293A discloses a method, system, head-mounted display (HMD), and medium for large-space positioning based on feature extraction. While this method has achieved some success in large-space positioning, it still has significant shortcomings. Specifically, this patent application relies solely on image data collected by a camera and a preset spatial feature library for positioning. The data fusion dimension is single, and it fails to fully utilize multi-source sensor data. Positioning reliability is poor in complex environments. Furthermore, it is only applicable to large-scale scenarios and has poor adaptability to delicate operations such as medical surgery. It also fails to consider the specific needs of different industries and lacks the ability to be applied in multiple fields.

[0004] In response to the various problems of existing technologies, there is an urgent need for a new technical solution for AR device positioning that can integrate multiple data, improve positioning accuracy, and be applicable to multiple industries. Summary of the Invention

[0005] The purpose of this application is to provide an AR device positioning system and method based on data fusion to solve the technical problems raised in the above background technology.

[0006] To achieve the above objectives, this application discloses the following technical solutions: In a first aspect, the present application discloses an AR device positioning system based on data fusion, comprising: The acquisition module is used to collect sensor data and positioning data for conventional positioning; it is also used to collect spatial data for spatial size judgment; and it collects infrared data and high-frequency data for small space positioning; infrared data is near-infrared band light information, and high-frequency data is high-frame-rate digital image data; The conventional positioning module is connected to the acquisition module and is used to fuse sensor data and positioning data to generate conventional positioning results for AR devices; The small space positioning module is connected to the acquisition module and the conventional positioning module. It is used to determine whether small space positioning is needed based on spatial data. When it is determined that small space positioning is needed, it integrates the conventional positioning results and infrared data to generate infrared positioning results, and uses high-frequency data to continuously correct the infrared positioning results to generate small space positioning results for AR devices.

[0007] Preferably, the generation of the conventional positioning result includes: The sensor data includes data collected by a nine-axis sensor, a light sensor, and a distance sensor, and the positioning data includes GPS and Beidou positioning data; Collection time Sensor data and positioning data , based on sensor data Generation time Predictive sensor data , and based on this predicted sensor data Generation time Predictive positioning data , get the time Collected sensor data and positioning data , when predicting positioning data and positioning data When the position deviation is greater than or equal to the preset position deviation threshold, the predicted sensor data is used , sensor data , predictive positioning data and positioning data , generate and output conventional positioning results.

[0008] As a preference, the utilization of predictive sensor data , sensor data , predictive positioning data and positioning data , generate and output conventional positioning results, including: Using conventional position fusion formulas, combined with predicted sensor data , sensor data , predictive positioning data and positioning data , calculate the conventional positioning result, the conventional position fusion formula is: in, The predicted sensor data is calculated and sensor data The data deviation, is the calculated fused conventional positioning result.

[0009] Preferably, the predicted sensor data The generation of includes: Using the preset device motion model, based on the time Sensor data For the moment Predictive positioning data A prediction is made, wherein the device motion model is fitted based on historical sensor data and historical device motion conditions, and the device motion model is updated based on real-time motion data of a wearer of the AR device, and the real-time motion data is synchronously collected using the acquisition module.

[0010] Preferably, the predicted positioning data The generation of includes: Using the preset location correlation model, based on the predicted sensor data and the real-time motion data determines the moment Predictive positioning data , wherein the position association model is fitted based on historical positioning data and corresponding historical sensor data and historical equipment movement conditions.

[0011] Preferably, the generation of the small space positioning result includes: Collection time spatial data and extract corresponding spatial features ,judge Is it true? If so, it is determined that small space positioning is needed. Otherwise, it is determined that small space positioning is not needed. For a preset small space feature set, when small space positioning is required, perform the following steps: A1: Generate the conventional positioning result using the conventional positioning module; A2: Collect and use the infrared data to capture a target object in a small space, update the conventional positioning result based on the target object, and generate an infrared positioning result; A3: Collect and use the high-frequency data to continuously correct the infrared positioning results to generate small-space positioning results.

[0012] As a preference, the collection time spatial data and extract corresponding spatial features ,judge Whether it is established, including: Use the acquisition module to collect time The spatial data is extracted, and the distance information and contour information in the spatial data are extracted to obtain the spatial features. , calculate the spatial characteristics The corresponding rate of change in the time series , based on spatial features and the corresponding rate of change In the small space feature set Compare the features one by one, calculate the similarity between the features, and determine when the preset similarity threshold is met. ,otherwise .

[0013] Preferably, step A2 includes: Using a telephoto low-distortion near-infrared lens to collect time Infrared data in a small space is collected and the corresponding infrared image is obtained. The target object in the small space is identified and captured from the infrared image using the target detection algorithm. Determine the image position of the target object in the infrared image , based on the image position Compared with conventional positioning results Compare and integrate the updated conventional positioning results to generate infrared positioning results , the contrast and fusion is: in, The frequency of updating the calculated conventional positioning results beyond the range; The conventional positioning result exceeds the update frequency range Calculations include: Count the number of updates of regular positioning results using image position ; Count the number of times the deviation between the updated conventional positioning result and the conventional positioning result before the update is greater than or equal to the preset deviation threshold ; Calculate the update frequency of conventional positioning results beyond the range .

[0014] Preferably, step A3 includes: Utilize high frame rate and high integration imaging sensors to continuously capture moments at a set high frequency The high-frequency data of a small space within the corresponding acquisition period is used to obtain a continuous image sequence; Analyze the changes in the motion trajectory of the target object in the image sequence and posture changes , and based on the motion trajectory changes And the posture changes Continue with the infrared positioning results Perform fusion to generate small space positioning results ; wherein the fusion is: in, is the number of frames in the image sequence.

[0015] In a second aspect, the present application discloses an AR device positioning method based on data fusion, which is applicable to the AR device positioning system based on data fusion as described above, and includes the following steps: S1: Collects sensor data and positioning data for conventional positioning; collects spatial data for space size judgment; collects infrared data and high-frequency data for small space positioning; infrared data is near-infrared band light information, and high-frequency data is high-frame-rate digital image data; S2: Fusion of sensor data and positioning data to generate conventional positioning results for AR devices; S3: Determine whether small space positioning is needed based on spatial data. If so, fuse conventional positioning results with infrared data to generate infrared positioning results. Use high-frequency data to continuously correct the infrared positioning results to generate small space positioning results for the AR device.

[0016] Beneficial effects: The AR device positioning system and method based on data fusion of the present application realize the precise positioning of AR devices in different scenarios; the acquisition module collects a variety of data to provide rich information for positioning; the conventional positioning module fuses the sensing and positioning data to generate conventional positioning results to meet the positioning needs of general scenarios; the small space positioning module judges based on spatial data, fuses conventional positioning results, infrared data and high-frequency data, and improves the accuracy and reliability of small space positioning; it solves the problem that the existing AR positioning system has limited accuracy and cannot adapt to complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 This is a structural block diagram of an AR device positioning system based on data fusion provided in an embodiment of the present application; Figure 2 This is a flowchart of the AR device positioning method based on data fusion provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The following is a clear and complete description of the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0020] In this document, the term "comprising" is intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0021] The first aspect of this embodiment discloses Figure 1 The AR device positioning system based on data fusion shown includes: The acquisition module is used to collect sensor data and positioning data for conventional positioning; it is also used to collect spatial data for spatial size judgment; and it collects infrared data and high-frequency data for small space positioning; infrared data is near-infrared band light information, and high-frequency data is high-frame-rate digital image data; The conventional positioning module is connected to the acquisition module and is used to fuse sensor data and positioning data to generate conventional positioning results for AR devices; The small space positioning module is connected to the acquisition module and the conventional positioning module. It is used to determine whether small space positioning is needed based on spatial data. When it is determined that small space positioning is needed, it integrates the conventional positioning results and infrared data to generate infrared positioning results, and uses high-frequency data to continuously correct the infrared positioning results to generate small space positioning results for AR devices.

[0022] Based on the above, this embodiment uses the acquisition module to collect multi-source data, the conventional positioning module to fuse sensor and positioning data, and the small space positioning module to judge and fuse multi-source data based on spatial data, thereby realizing accurate positioning of AR devices in different scenarios. The acquisition module collects a variety of data to provide rich information for positioning. The conventional positioning module fuses sensor and positioning data to generate conventional positioning results to meet the positioning requirements of general scenarios. The small space positioning module judges based on spatial data, fuses conventional positioning results, infrared data and high-frequency data, and improves the accuracy and reliability of small space positioning. It solves the problem that the existing AR positioning system has limited accuracy and cannot adapt to complex scenarios.

[0023] Specifically, the generation of the conventional positioning result includes: The sensor data includes data collected by a nine-axis sensor, a light sensor, and a distance sensor, and the positioning data includes GPS and Beidou positioning data; Collection time Sensor data and positioning data , based on sensor data Generation time Predictive sensor data , and based on this predicted sensor data Generation time Predictive positioning data , get the time Collected sensor data and positioning data , when predicting positioning data and positioning data When the position deviation is greater than or equal to the preset position deviation threshold, the predicted sensor data is used , sensor data , predictive positioning data and positioning data , generate and output conventional positioning results.

[0024] Based on the above, this embodiment uses the sensor data and positioning data collected at different times, combined with the prediction data and the preset deviation threshold judgment method to achieve more accurate and reliable conventional positioning results. Prediction of sensor data and positioning data The system compares the predicted positioning data with the actual positioning data collected at each moment. When the deviation reaches a threshold, it uses both the predicted and actual data to generate a conventional positioning result. This process fully considers the continuity of device motion and the real-time nature of data. Compared with positioning methods that rely solely on currently collected data, it can better respond to changes in device motion state, reduce positioning errors, improve positioning accuracy, and enhance the positioning system's adaptability to complex motion scenarios. This provides strong support for the stable positioning of AR devices in conventional scenarios, ensuring that they can accurately obtain location information in various application scenarios.

[0025] Specifically, the use of predictive sensor data , sensor data , predictive positioning data and positioning data , generate and output conventional positioning results, including: Using conventional position fusion formulas, combined with predicted sensor data , sensor data , predictive positioning data and positioning data , calculate the conventional positioning result, the conventional position fusion formula is: in, The predicted sensor data is calculated and sensor data The data deviation, is the calculated fused conventional positioning result.

[0026] In this embodiment, according to the predicted sensor data and actual sensor data The deviation of the predicted positioning data and actual positioning data Perform weighted calculations. When predicting sensor data Compared with actual sensor data When approaching, If the value of is small, then The value of is larger, and the predicted positioning data In the final result This means that when the predicted sensor data is reliable, the predicted positioning data is more trusted. Based on this, it is possible to avoid excessive deviations in positioning results due to abnormal predicted sensor data, making the final positioning result more consistent with the actual situation.

[0027] In a simple example, in an AR device positioning scenario, a distance sensor is used to collect data to assist positioning. There is a fixed reference point in the scene, and the AR device obtains the distance information from the reference point through the distance sensor. Based on the previous motion trajectory and algorithm predictions, the system predicts that the AR device's distance from the reference point is 50 cm. The actual distance measured by the distance sensor is 60 cm. Based on the predicted sensor data, the system infers that the horizontal coordinate of the AR device's predicted positioning data in the two-dimensional plane coordinate system is 80 cm. Using other positioning methods (such as inertial measurement unit-assisted positioning), the system determines that the horizontal coordinate of the AR device's actual positioning data is 90 cm. Based on this, the calculated conventional positioning result is 81.7 cm. This shows that using data collected by the distance sensor and combining it with the formula can combine predicted and actual sensor and positioning data to obtain a relatively more accurate positioning result.

[0028] Based on the above, this embodiment uses a conventional position fusion formula, combined with predicted and actual sensor and positioning data, to achieve accurate calculation of conventional positioning results. This formula performs weighted calculation on the predicted positioning data and the actual positioning data based on the deviation between the predicted sensor data and the actual sensor data. When the deviation is small, the predicted positioning data has a higher weight; when the deviation is large, the actual positioning data has an even higher weight. This dynamic weighted calculation method can flexibly adjust the fusion results according to the reliability of the data, fully utilize the advantages of different data, effectively reduce the impact of abnormal data on the positioning results, improve the accuracy and stability of positioning, and enable conventional positioning results to more accurately reflect the actual position of the AR device, meeting the strict requirements for the positioning accuracy of the AR device in different scenarios.

[0029] Specifically, the predicted sensor data The generation of includes: Using the preset device motion model, based on the time Sensor data For the moment Predictive positioning data A prediction is made, wherein the device motion model is fitted based on historical sensor data and historical device motion conditions, and the device motion model is updated based on real-time motion data of a wearer of the AR device, and the real-time motion data is synchronously collected using the acquisition module.

[0030] Based on the above, this embodiment uses existing deep learning technology to build a device motion model based on historical data fitting, combined with real-time motion data updates, to achieve effective generation of predicted sensor data. The device motion model is built based on historical sensor data and device motion conditions, and can reflect the device motion law. As the real-time motion data of the AR device wearer is collected, the model is continuously updated to ensure its timeliness and accuracy. Based on this, according to the time Sensor data prediction time The predicted sensor data can better fit the actual movement trend of the device, provide reliable preliminary data support for the prediction of subsequent positioning data and the generation of conventional positioning results, improve the entire positioning system's ability to track changes in the device's motion state, and ensure the timeliness and accuracy of positioning.

[0031] Specifically, the predicted positioning data The generation of includes: Using the preset location correlation model, based on the predicted sensor data and the real-time motion data determines the moment Predictive positioning data , wherein the position association model is fitted based on historical positioning data and corresponding historical sensor data and historical equipment movement conditions.

[0032] Through the above, this embodiment utilizes existing deep learning technology, constructs a position association model based on historical data fitting, and combines predicted sensor data and real-time motion data to achieve accurate determination of predicted positioning data. The position association model is fitted based on historical positioning, sensing and device motion data, revealing the intrinsic connection between sensor data and positioning data. Through this model, predicted positioning data is calculated based on predicted sensor data and real-time motion data, fully considering the impact of device motion state and environmental factors on positioning. This method can more accurately estimate the position of the device at the next moment, provide a more reliable reference for the generation of conventional positioning results, enhance the foresight and accuracy of the positioning system, and enable AR devices to achieve high-precision positioning in complex environments and dynamic scenes.

[0033] Specifically, the generation of the small space positioning result includes: Collection time spatial data and extract corresponding spatial features ,judge Is it true? If so, it is determined that small space positioning is needed. Otherwise, it is determined that small space positioning is not needed. For a preset small space feature set, when small space positioning is required, perform the following steps: A1: Generate the conventional positioning result using the conventional positioning module; A2: Collect and use the infrared data to capture a target object in a small space, update the conventional positioning result based on the target object, and generate an infrared positioning result; A3: Collect and use the high-frequency data to continuously correct the infrared positioning results to generate small-space positioning results.

[0034] Through the above, this embodiment utilizes the collection of spatial data, extraction of features and comparison with the preset feature set, combined with the fusion processing of conventional positioning results, infrared data and high-frequency data, to achieve accurate positioning of AR devices in small space scenes. By collecting spatial data to determine whether to enter a small space scene, in the small space, the conventional positioning results are used to provide initial positioning, and the infrared data is combined to capture the target object to update the positioning, and then the high-frequency data is used for continuous correction. The advantages of multi-source data are integrated to effectively improve the accuracy and real-time performance of small space positioning. This solves the shortcomings of existing technologies in small space positioning, enables AR devices to work accurately in small space high-precision positioning scenarios such as medical surgery and precision instrument operation, expands the application scope of AR technology, and promotes technological development in related fields.

[0035] Specifically, the collection time spatial data and extract corresponding spatial features ,judge Whether it is established, including: Use the acquisition module to collect time The spatial data is extracted, and the distance information and contour information in the spatial data are extracted to obtain the spatial features. , calculate the spatial characteristics The corresponding rate of change in the time series , based on spatial features and the corresponding rate of change In the small space feature set Compare the features one by one, calculate the similarity between the features, and determine when the preset similarity threshold is met. ,otherwise .

[0036] Through the above, this embodiment uses the method of collecting spatial data to extract distance and contour information, calculating the feature change rate and comparing the similarity with the preset feature set to achieve accurate judgment of small space scenes. After the acquisition module obtains the spatial data, it extracts key features and analyzes their change rate, and calculates the similarity of this information with the preset small space feature set. Compared with simple spatial data judgment, this multi-dimensional analysis and judgment method can more accurately identify small space scenes and reduce misjudgment. Accurate scene judgment provides a reliable basis for the subsequent work of the small space positioning module, ensuring that the AR device can switch the positioning mode in time when entering a small space scene, improving the intelligence and adaptability of the positioning system, and ensuring the stable positioning of the AR device in different spatial scenes.

[0037] Specifically, step A2 includes: Using a telephoto low-distortion near-infrared lens to collect time Infrared data in a small space is collected and the corresponding infrared image is obtained. The target object in the small space is identified and captured from the infrared image using the target detection algorithm. Determine the image position of the target object in the infrared image , based on the image position Compared with conventional positioning results Compare and integrate the updated conventional positioning results to generate infrared positioning results , the contrast and fusion is: in, The frequency of updating the calculated conventional positioning results beyond the range; The conventional positioning result exceeds the update frequency range Calculations include: Count the number of updates of regular positioning results using image position ; Count the number of times the deviation between the updated conventional positioning result and the conventional positioning result before the update is greater than or equal to the preset deviation threshold ; Calculate the update frequency of conventional positioning results beyond the range .

[0038] It should be noted that the telephoto low-distortion near-infrared lens of this embodiment is a lens of a telephoto low-distortion near-infrared optical system independently developed and produced by the Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences. It is an optical system for fixed wavelength, size, and imaging range markers, which can simplify the algorithm and improve the accuracy of the back-end processing algorithm. While ensuring a large field of view at the optical end, it has low distortion and reduces the target deformation. It also uses Fast glass broadband anti-reflection coating to improve transmittance and ensure data accuracy.

[0039] In the specific application of this embodiment, weighted fusion is used to determine the reliability of the conventional positioning results (the frequency of the conventional positioning results exceeding the range is updated). (embodied in Figure 2), perform weighted summation of infrared image position information and conventional positioning results. When it approaches 1, it means that the conventional positioning results are frequently updated beyond the range and the reliability is low. The proportion in the fusion result is larger; when When it approaches 0, it means that the conventional positioning result is relatively stable and reliable. It dominates the fusion results. Through this dynamic weighting method, the advantages of the two data can be adaptively combined to improve the accuracy of the positioning results. It is the number of times the conventional positioning result is updated using the image position, reflecting the update activity of the infrared image position information to the conventional positioning result. It is the number of times that the deviation between the conventional positioning result after the update and the conventional positioning result before the update is greater than or equal to the preset deviation threshold, reflecting the situation where the conventional positioning result fluctuates greatly after being updated by the infrared image position information. , can measure the stability of conventional positioning results in the process of updating infrared image position information. Relative update times If there are more, it means that the conventional positioning results have changed a lot after the update and the stability is poor. If the value is large, the fusion will rely more on the infrared image position information; conversely, if the number of out-of-range times is small, it will rely more on the conventional positioning results.

[0040] Based on the above, this embodiment utilizes the existing telephoto low-distortion near-infrared lens to collect infrared data, combines the existing target detection algorithm and conventional positioning results, and fuses them by calculating the out-of-range update frequency of the conventional positioning results, thereby achieving accurate generation of infrared positioning results. The telephoto low-distortion near-infrared lens acquires a clear infrared image, the target detection algorithm identifies the position of the target object, and the image position and the conventional positioning result are weightedly fused according to the out-of-range update frequency of the conventional positioning results. This method integrates the infrared image information and the conventional positioning results, dynamically adjusts the fusion weight according to the stability of the positioning results, effectively improves the accuracy of infrared positioning, provides more accurate initial positioning data for small space positioning, enhances the performance of the small space positioning module, and meets the high-precision requirements of small space positioning.

[0041] Specifically, step A3 includes: Utilize high frame rate and high integration imaging sensors to continuously capture moments at a set high frequency The high-frequency data of a small space within the corresponding acquisition period is used to obtain a continuous image sequence; Analyze the changes in the motion trajectory of the target object in the image sequence and posture changes , and based on the motion trajectory changes And the posture changes Continue with the infrared positioning results Perform fusion to generate small space positioning results ; wherein the fusion is: in, is the number of frames in the image sequence.

[0042] It should be noted that the high-frame-rate, highly integrated imaging sensor of this embodiment is a high-frame-rate, highly integrated CMOS imaging system developed by Changguang Chenxin, a subsidiary of the Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences. Its signal strength is greater than the noise intensity, ensuring that the effective signal can be clearly imaged. It has the characteristics of high frame rate and high integration, ensuring data update rate, and achieving miniaturization and lightweight.

[0043] In the specific application of this embodiment, based on the analysis of high frame rate imaging data and the fusion with infrared positioning results, the positioning accuracy of AR devices in small spaces is improved as follows: a high frame rate and high integration imaging sensor is used to collect high-frequency data in a small space at a higher frequency to form a continuous image sequence. The high frame rate feature enables the image sequence to more precisely capture the dynamic changes of the target object in a small space; by analyzing these image sequences, the changes in the target object's motion trajectory are obtained. and posture changes ,These change information can reflect the real-time motion state of the target object in a small space. is the infrared positioning result obtained in the previous step, It is the accumulation of the target object motion trajectory changes in all frames in the image sequence. It is the accumulation of the target object posture changes in all frames in the image sequence. The infrared positioning results serve as the basic positioning information, upon which the target object's motion trajectory and posture changes, obtained from analyzing high-frame-rate image sequences, are superimposed. Because high-frame-rate image sequences can accurately and in detail reflect the dynamics of the target object in real time, accumulating these dynamic changes and combining them with the infrared positioning results allows for continuous correction and improvement of the positioning results, enabling small-space positioning results to more accurately reflect the target object's actual position and state within the small space. This design fully leverages the advantages of high-frame-rate data, effectively improving the accuracy and real-time performance of small-space positioning.

[0044] Through the above, this embodiment uses the existing high-frame-rate and high-integration imaging sensor to collect high-frequency data, analyze the motion trajectory and posture changes of the target object, and integrate it with the infrared positioning results, thereby realizing the continuous optimization of the small space positioning results. The high-frame-rate and high-integration imaging sensor collects image sequences in a small space at a high frequency, from which the motion and posture changes of the target object are analyzed. These change information are integrated with the infrared positioning results, and the positioning results are continuously corrected as the number of image frames increases. This enables the small space positioning results to reflect the dynamic changes of the target object in real time, greatly improving the accuracy and real-time performance of small space positioning, ensuring that AR devices can achieve accurate positioning in complex motion scenes in small spaces, and meeting the stringent requirements of medical, industrial and other fields for high-precision positioning in small spaces.

[0045] The second aspect of this embodiment discloses Figure 2 A data fusion-based AR device positioning method is shown, which is applicable to the data fusion-based AR device positioning system described above, and includes the following steps: S1: Collects sensor data and positioning data for conventional positioning; collects spatial data for space size judgment; collects infrared data and high-frequency data for small space positioning; infrared data is near-infrared band light information, and high-frequency data is high-frame-rate digital image data; S2: Fusion of sensor data and positioning data to generate conventional positioning results for AR devices; S3: Determine whether small space positioning is needed based on spatial data. If so, fuse conventional positioning results with infrared data to generate infrared positioning results. Use high-frequency data to continuously correct the infrared positioning results to generate small space positioning results for the AR device.

[0046] It should be noted that the AR device positioning method based on data fusion in this embodiment corresponds to the aforementioned AR device positioning system based on data fusion. Therefore, the contents not specifically described in the AR device positioning method based on data fusion in this embodiment may include but are not limited to functional definitions, working principles, and technical effects, etc., and may all refer to the records in the aforementioned AR device positioning system based on data fusion. This text will not elaborate on them here.

[0047] In summary, the AR device positioning system and method based on data fusion in this embodiment realizes the precise positioning of AR devices in different scenarios; the acquisition module collects a variety of data to provide rich information for positioning; the conventional positioning module fuses the sensing and positioning data to generate conventional positioning results to meet the positioning requirements of general scenarios; the small space positioning module judges based on spatial data, fuses conventional positioning results, infrared data and high-frequency data, and improves the accuracy and reliability of small space positioning; it solves the problem that the existing AR positioning system has limited accuracy and cannot adapt to complex scenarios.

[0048] In the embodiments provided herein, it should be understood that the embodiments described herein can be implemented using hardware, software, firmware, middleware, code, or any appropriate combination thereof. For hardware implementation, the processor may be implemented in one or more of the following: an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a processor, a controller, a microcontroller, a microprocessor, or other electronic units designed to implement the functionality described herein, or any combination thereof. For software implementation, some or all of the processes of the embodiments may be performed by a computer program instructing the relevant hardware. During implementation, the program may be stored in a computer-readable storage medium or transmitted as one or more instructions or codes on a computer-readable storage medium. Computer-readable storage media include computer storage media and communication media, wherein communication media includes any medium that facilitates the transmission of a computer program from one location to another. The storage medium may be any available medium that can be accessed by a computer. Computer-readable storage media may include, but are not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing the desired program code in the form of instructions or data structures and accessible by a computer.

[0049] Finally, it should be noted that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or make equivalent replacements for some of the technical features therein. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. An AR device positioning system based on data fusion, characterized in that: include: An acquisition module is used to acquire sensor data and positioning data for conventional positioning; it is also used to acquire spatial data for spatial dimension judgment; And collect infrared data and high-frequency data for small space positioning; infrared data is near-infrared band light information, and high-frequency data is high-frame-rate digital image data; The conventional positioning module is connected to the acquisition module and is used to fuse sensor data and positioning data to generate conventional positioning results for AR devices; The small space positioning module is connected to the acquisition module and the conventional positioning module. It is used to determine whether small space positioning is needed based on spatial data. When it is determined that small space positioning is needed, it integrates the conventional positioning results and infrared data to generate infrared positioning results, and uses high-frequency data to continuously correct the infrared positioning results to generate small space positioning results for AR devices.

2. The AR device positioning system based on data fusion according to claim 1, characterized in that: The generation of the conventional positioning result includes: The sensor data includes data collected by a nine-axis sensor, a light sensor, and a distance sensor, and the positioning data includes GPS and Beidou positioning data; Collection time Sensor data and positioning data , based on sensor data Generation time Predictive sensor data , and based on this predicted sensor data Generation time Predictive positioning data , get the time Collected sensor data and positioning data , when predicting positioning data and positioning data When the position deviation is greater than or equal to the preset position deviation threshold, the predicted sensor data is used , sensor data , predictive positioning data and positioning data , generate and output conventional positioning results.

3. The AR device positioning system based on data fusion according to claim 2, characterized in that: The use of predictive sensor data , sensor data , predictive positioning data and positioning data , generate and output conventional positioning results, including: Using conventional position fusion formulas, combined with predictive sensor data , sensor data , predictive positioning data and positioning data , calculate the conventional positioning result, the conventional position fusion formula is: in, The predicted sensor data is calculated and sensor data The data deviation, is the conventional positioning result after calculation and fusion.

4. The AR device positioning system based on data fusion according to claim 2, characterized in that: The predicted sensor data The generation of includes: Using the preset device motion model, based on the time Sensor data For the moment Predictive positioning data A prediction is performed, wherein the device motion model is fitted based on historical sensor data and historical device motion conditions, and the device motion model is updated based on real-time motion data of a wearer of the AR device, and the real-time motion data is synchronously collected using the collection module.

5. The AR device positioning system based on data fusion according to claim 4, characterized in that: The predicted positioning data The generation of includes: Using the preset location correlation model, based on the predicted sensor data and the real-time motion data determines the moment Predictive positioning data , wherein the position association model is fitted based on historical positioning data and corresponding historical sensor data and historical equipment movement conditions.

6. The AR device positioning system based on data fusion according to claim 1, characterized in that: The generation of the small space positioning result includes: Collection time spatial data and extract corresponding spatial features ,judge Is it true? If so, it is determined that small space positioning is needed. Otherwise, it is determined that small space positioning is not needed. For a preset small space feature set, when small space positioning is required, perform the following steps: A1: Generate the conventional positioning result using the conventional positioning module; A2: Collect and use the infrared data to capture a target object in a small space, update the conventional positioning result based on the target object, and generate an infrared positioning result; A3: Collect and use the high-frequency data to continuously correct the infrared positioning results to generate small-space positioning results.

7. The AR device positioning system based on data fusion according to claim 6, characterized in that: The collection time spatial data and extract corresponding spatial features ,judge Whether it is established, including: Use the acquisition module to collect time The spatial data is extracted, and the distance information and contour information in the spatial data are extracted to obtain the spatial features. , calculate the spatial characteristics The corresponding rate of change in the time series , based on spatial features and the corresponding rate of change In the small space feature set Compare the features one by one, calculate the similarity between the features, and determine when the preset similarity threshold is met. ,otherwise .

8. The AR device positioning system based on data fusion according to claim 6, characterized in that: Step A2 includes: Using a telephoto low-distortion near-infrared lens to collect time Infrared data in a small space is collected and the corresponding infrared image is obtained. The target object in the small space is identified and captured from the infrared image using the target detection algorithm. Determine the image position of the target object in the infrared image , based on the image position Compared with conventional positioning results Compare and integrate the updated conventional positioning results to generate infrared positioning results , the contrast and fusion is: in, The frequency of updating the calculated conventional positioning results beyond the range; The conventional positioning result exceeds the update frequency range Calculations include: Count the number of updates of regular positioning results using image position ; Count the number of times the deviation between the updated conventional positioning result and the conventional positioning result before the update is greater than or equal to the preset deviation threshold ; Calculate the update frequency of conventional positioning results beyond the range .

9. The AR device positioning system based on data fusion according to claim 6, characterized in that: Step A3 includes: Utilize high frame rate and high integration imaging sensors to continuously capture moments at a set high frequency The high-frequency data of a small space within the corresponding acquisition period is used to obtain a continuous image sequence; Analyze the changes in the motion trajectory of the target object in the image sequence and posture changes , and based on the motion trajectory changes And the posture changes Continue with the infrared positioning results Perform fusion to generate small space positioning results ; wherein the fusion is: in, is the number of frames in the image sequence.

10. A data fusion-based AR device positioning method, the method being applicable to the data fusion-based AR device positioning system according to any one of claims 1 to 9, characterized in that: The following steps are involved: S1: Collects sensor data and positioning data for conventional positioning; collects spatial data for space size judgment; collects infrared data and high-frequency data for small space positioning; infrared data is near-infrared band light information, and high-frequency data is high-frame-rate digital image data; S2: Fusion of sensor data and positioning data to generate conventional positioning results for AR devices; S3: Determine whether small space positioning is needed based on spatial data. If so, fuse conventional positioning results with infrared data to generate infrared positioning results. Use high-frequency data to continuously correct the infrared positioning results to generate small space positioning results for the AR device.

Images