Data fusion positioning method, device, mobile device, chip system and medium

Abstract

The present disclosure provides a data fusion positioning method, device, mobile device, chip system and medium. The method includes: generating fused positioning pose points according to the repositioning pose points and sensor pose points of the mobile device; the repositioning pose points and / or the sensor pose points are obtained by correction of abnormal pose points; determining the target positioning pose points according to the fused positioning pose points and the inertial measurement unit IMU information of the mobile device, and the target positioning pose points are used to represent the current position of the mobile device. In this embodiment, by eliminating abnormal pose points in the repositioning pose points and / or sensor pose points, the accuracy of the input data can be improved, which is conducive to ensuring the robustness of the positioning result; and in this embodiment, generating fused positioning pose points from the repositioning pose points and the sensor pose points can reduce the number and quantity of input data, which is conducive to ensuring the smoothness of the positioning result.

Description

Technical Field

[0001] The present disclosure relates to the field of control technology, and in particular to a data fusion positioning method, device, mobile device, chip system and medium. Background Art

[0002] The robot needs to locate itself in real time during autonomous walking to accurately reach its destination. During the positioning process, it is necessary to obtain the navigation positioning information and inertial navigation information of the machine as input data to predict the robot's position. When the input data is abnormal, positioning failure is likely to occur, causing the robot to deviate from its walking position. Summary of the invention

[0003] The present disclosure provides a data fusion positioning method, device, mobile device, chip system and medium to solve the above technical problems.

[0004] According to a first aspect of the present disclosure, a data fusion positioning method is provided, the method comprising:

[0005] Generate fused positioning pose points according to the repositioning pose points and sensor pose points of the mobile device; the repositioning pose points and / or the sensor pose points are obtained by correcting abnormal pose points;

[0006] A target positioning pose point is determined according to the fused positioning pose point and the inertial measurement unit (IMU) information of the mobile device, and the target positioning pose point is used to represent the current position of the mobile device.

[0007] Optionally, correcting abnormal posture points in the relocated posture points of the mobile device includes:

[0008] According to the sensor pose point of the mobile device and the best conversion relationship at the last moment, a first mapping point of the sensor pose point in the world coordinate system is obtained; the best conversion relationship at the last moment is used to represent the conversion relationship from the sensor pose point at the last moment to the best repositioning pose point at the same moment; the best repositioning pose point refers to the repositioning pose point with the smallest offset distance from the sensor pose point mapped from the visual coordinate system to the world coordinate system;

[0009] Determining whether the relocated pose point is an abnormal pose point according to the distance between the first mapping point and the relocated pose point;

[0010] If the repositioned pose point is determined to be an abnormal pose point, the abnormal pose point is removed from the repositioned pose points of the mobile device.

[0011] Optionally, judging whether the relocated pose point is an abnormal pose point according to the distance between the first mapping point and the relocated pose point includes:

[0012] Obtaining a distance between the first mapping point and the repositioning pose point;

[0013] Determine the relocated pose point whose distance is greater than the first distance threshold as an abnormal pose point.

[0014] Optionally, the method further comprises:

[0015] Determine the relocated pose point whose distance is less than or equal to the first distance threshold as a normal pose point.

[0016] Optionally, the mobile device further includes an odometer, and the method further includes correcting abnormal pose points in the sensor pose points of the mobile device according to the odometer information, including:

[0017] Align odometer information and sensor pose points based on time information;

[0018] Obtaining the ratio of the odometer information change to the sensor pose point change within a preset time length to obtain a distance proportional factor; the odometer information change refers to the difference between the odometer information at the start time and the odometer information at the end time within the preset time length; the sensor pose point change refers to the difference between the sensor pose point at the start time and the sensor pose point at the end time within the preset time length;

[0019] The sensor pose points whose distance scale factors are outside a preset value range are determined as abnormal pose points, and the abnormal pose points are removed from the sensor pose points of the mobile device.

[0020] Optionally, generating a fused positioning pose point according to the repositioning pose point of the mobile device and the sensor pose point includes:

[0021] Determine the best repositioning posture point among a first preset number of normal posture points; the first preset number of normal posture points is composed of the repositioning posture point at the current moment, the best repositioning posture point at the previous moment, and other normal posture points;

[0022] Obtaining an optimal transformation relationship corresponding to the optimal repositioning posture point;

[0023] The sensor pose points are mapped to a world coordinate system according to the optimal conversion relationship, and the mapping points of the sensor pose points in the world system are used as fused positioning pose points.

[0024] Optionally, determining the best repositioning pose point among a first preset number of normal pose points comprises:

[0025] Obtaining a conversion relationship corresponding to each normal posture point in a first preset number of normal posture points; the conversion relationship is used to represent a conversion relationship between the sensor posture point at the same moment of the normal posture point and the normal posture point in the world coordinate system;

[0026] According to the conversion relationship corresponding to each normal posture point, the distance between each normal posture point and its mapping point is obtained to obtain a first preset number of distances corresponding to each normal posture point;

[0027] Obtaining an average value of a first preset number of distances of each normal displacement point to obtain an average value of the first preset number of distances;

[0028] The minimum value of the first preset number of distance average values ​​is obtained, and the normal displacement point corresponding to the minimum value is used as the optimal repositioning posture point at the current moment.

[0029] Optionally, the method further comprises:

[0030] If it is determined that the minimum value of the first preset number of distance averages is greater than or equal to a first preset distance threshold, the first preset number of normal posture points are adjusted to a second preset number of normal posture points and then the best repositioning posture points are reacquired, and the first preset number is less than the second preset number.

[0031] Optionally, the method further includes obtaining a starting transformation relationship of a first optimal repositioning pose point, including:

[0032] Control the mobile device to rotate in three directions to obtain three repositioning points;

[0033] Obtaining the distance between any two repositioning pose points among the three repositioning pose points;

[0034] If it is determined that the distance between any two repositioning pose points is less than or equal to the second preset distance threshold, the transformation relationship between the current repositioning pose point and the sensor pose point is obtained as the starting transformation relationship for transforming the sensor pose point from the visual coordinate system to the mapping point in the world coordinate system.

[0035] According to a second aspect of the present disclosure, a data fusion positioning device is provided, the device comprising:

[0036] A fused positioning pose point generation module is used to generate fused positioning pose points according to the repositioning pose points and sensor pose points of the mobile device; the repositioning pose points and / or the sensor pose points are obtained by correcting abnormal pose points;

[0037] The target positioning pose point determination module is used to determine the target positioning pose point according to the fused positioning pose point and the inertial measurement unit IMU information of the mobile device, wherein the target positioning pose point is used to represent the current position of the mobile device.

[0038] Optionally, the device further comprises an abnormal posture point correction module, which is used to correct abnormal posture points in the relocated posture points of the mobile device; the abnormal posture point correction module comprises:

[0039] A mapping point acquisition submodule, used for acquiring a first mapping point of the sensor pose point in the world coordinate system according to the sensor pose point of the mobile device and the best conversion relationship at the last moment; the best conversion relationship at the last moment is used to represent the conversion relationship from the sensor pose point at the last moment to the best repositioning pose point at the same moment; the best repositioning pose point refers to the repositioning pose point with the smallest offset distance when the sensor pose point is mapped from the visual coordinate system to the world coordinate system;

[0040] A posture point judgment submodule, used for judging whether the relocated posture point is an abnormal posture point according to the distance between the first mapping point and the relocated posture point;

[0041] The abnormal posture point elimination submodule is used to determine that the repositioned posture point is an abnormal posture point, and then eliminate the abnormal posture point from the repositioned posture points of the mobile device.

[0042] Optionally, the posture point judgment submodule includes:

[0043] A mapping point distance acquisition unit, used to acquire the distance between the first mapping point and the repositioning pose point;

[0044] The abnormal posture point determination unit is used to determine the relocated posture point whose distance is greater than the first distance threshold as an abnormal posture point.

[0045] Optionally, the posture point judgment submodule further includes:

[0046] The normal posture point determination unit is used to determine the repositioned posture point whose distance is less than or equal to the first distance threshold as a normal posture point.

[0047] Optionally, the mobile device further includes an odometer, and the apparatus further includes an abnormal posture point correction module, which is used to correct abnormal posture points in the sensor posture points of the mobile device according to the odometer information; the abnormal posture point correction module includes:

[0048] The sensor pose point alignment submodule is used to align the odometer information and sensor pose points according to the time information;

[0049] The distance scale factor acquisition submodule is used to obtain the ratio of the odometer information change amount to the sensor posture point change amount within a preset time length to obtain the distance scale factor; the odometer information change amount refers to the difference between the odometer information at the start time and the odometer information at the end time within the preset time length; the sensor posture point change amount refers to the difference between the sensor posture point at the start time and the sensor posture point at the end time within the preset time length;

[0050] The abnormal posture point determination submodule is used to determine the sensor posture point whose distance scale factor is outside the preset value range as an abnormal posture point, and remove the abnormal posture point from the sensor posture points of the mobile device.

[0051] Optionally, the fusion positioning pose point generation module includes:

[0052] The best posture point determination submodule is used to determine the best repositioning posture point among a first preset number of normal posture points; the first preset number of normal posture points is composed of the repositioning posture point at the current moment, the best repositioning posture point at the previous moment and other normal posture points;

[0053] An optimal transformation relationship acquisition submodule is used to obtain the optimal transformation relationship corresponding to the optimal repositioning posture point;

[0054] The fusion pose point determination submodule is used to map the sensor pose points to a world coordinate system according to the optimal conversion relationship, and use the mapping points of the sensor pose points in the world system as fusion positioning pose points.

[0055] Optionally, the optimal posture point determination submodule includes:

[0056] A conversion relationship acquisition unit, used to acquire a conversion relationship corresponding to each normal posture point in a first preset number of normal posture points; the conversion relationship is used to represent a conversion relationship from a sensor posture point at the same moment of the normal posture point to the normal posture point in a world coordinate system;

[0057] A posture point distance acquisition unit, for acquiring the distance between each normal posture point and its mapping point according to the conversion relationship corresponding to each normal posture point, and obtaining a first preset number of distances corresponding to each normal posture point;

[0058] A distance average value obtaining unit, used to obtain an average value of a first preset number of distances of each normal displacement point, to obtain a first preset number of distance average values;

[0059] The optimal repositioning posture point acquisition unit is used to obtain the minimum value of the first preset number of distance average values, and use the normal displacement point corresponding to the minimum value as the optimal repositioning posture point at the current moment.

[0060] Optionally, the device further comprises:

[0061] The module for adjusting the number of posture points is used to determine that the minimum value of the average value of the first preset number of distances is greater than or equal to the first preset distance threshold, and then adjust the first preset number of normal posture points to the second preset number of normal posture points and re-acquire the best repositioning posture points, and the first preset number is less than the second preset number.

[0062] Optionally, the device further comprises a starting transformation relationship acquisition module, which is used to acquire a starting transformation relationship of a first optimal repositioning posture point; the starting transformation relationship acquisition module comprises:

[0063] The repositioning posture point acquisition submodule is used to control the mobile device to rotate in three directions on the spot to obtain three repositioning posture points;

[0064] A posture point distance acquisition submodule, used to obtain the distance between any two repositioning posture points among the three repositioning posture points;

[0065] The starting transformation relationship determination submodule is used to determine that the distance between any two repositioning pose points is less than or equal to a second preset distance threshold, and then obtain the transformation relationship between the current repositioning pose point and the sensor pose point as the starting transformation relationship for converting the sensor pose point from the visual coordinate system to the mapping point in the world coordinate system.

[0066] According to a third aspect of the present disclosure, there is provided a mobile device, comprising: a sensor, a processor, and a memory;

[0067] The sensor is used to collect relocation pose points, sensor pose points, IMU information and odometer information of the mobile device and send them to the processor;

[0068] The memory is used to store a computer program executable by the processor;

[0069] The processor is configured to execute the computer program in the memory to implement the method as described in any one of the first aspects.

[0070] According to a fourth aspect of the present disclosure, a chip system is provided, comprising a processing unit and an interface circuit, wherein the processing unit obtains program instructions through the interface circuit, the program instructions are executed by the processing unit, and the processing unit is used to execute the method as described in any one of the first aspects.

[0071] According to a fifth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, and when an executable computer program in the storage medium is executed by a processor, the method described in any one of the first aspects can be implemented.

[0072] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0073] The solution provided in this embodiment can generate fused positioning pose points based on the repositioning pose points and sensor pose points of the mobile device; the repositioning pose points and / or the sensor pose points are obtained by correcting the abnormal pose points; the target positioning pose points are determined based on the fused positioning pose points and the inertial measurement unit IMU information of the mobile device, and the target positioning pose points are used to represent the current position of the mobile device. In this way, in this embodiment, by eliminating abnormal pose points in the repositioning pose points and / or sensor pose points, the accuracy of the input data can be improved, which is conducive to ensuring the robustness of the positioning results; and in this embodiment, the repositioning pose points and the sensor pose points are generated into fused positioning pose points, which can reduce the number and quantity of input data, which is conducive to ensuring the smoothness of the positioning results.

[0074] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 A block diagram of a mobile device including sensor types according to an embodiment of the present disclosure is provided.

[0076] Figure 2 The present invention is a flowchart of a data fusion positioning method according to an embodiment of the present invention.

[0077] Figure 3 The present invention is a flowchart of obtaining abnormal posture points in repositioning posture points according to an embodiment of the present invention.

[0078] Figure 4 The present invention is a flowchart of obtaining abnormal posture points in sensor posture points according to an embodiment of the present invention.

[0079] Figure 5 The present invention is a flowchart of obtaining fused positioning pose points according to an embodiment of the present invention.

[0080] Figure 6 The present invention is a flowchart of obtaining the best repositioning pose point according to an embodiment of the present invention.

[0081] Figure 7 The present invention is a schematic diagram of a repositioning posture point according to an embodiment of the present invention.

[0082] Figure 8 The present invention is a flowchart of obtaining a starting conversion relationship according to an embodiment of the present invention.

[0083] Fig. 9 The present invention is a flowchart of obtaining target positioning points according to an embodiment of the present invention.

[0084] Fig.10 The present invention is a block diagram of a data fusion positioning device according to an embodiment of the present invention.

[0085] Fig.11 The block diagram of a chip system according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0086] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices consistent with some aspects of the present disclosure as detailed in the appended claims.

[0087] To solve the above technical problems, the embodiments of the present disclosure provide a data fusion positioning method, apparatus, mobile device, chip system and medium. The data fusion positioning method can be applied to the scenario of mobile device positioning and navigation. Figure 1 The mobile device may include multiple sensors, such as a camera module and / or a radar module, an IMU sensor or an inertial measurement unit (including a gyroscope and an accelerometer), an odometer, and a relocation service in the mobile device. The camera module and / or the radar module may collect sensor pose points T i v , which can be understood as the position point located in the visual coordinate system, used to characterize the position and posture of the mobile device in the visual coordinate system. The IMU sensor can collect IMU information (including angular velocity and linear acceleration) to represent the angular velocity and linear acceleration of the mobile device at the sampling time; the odometer can collect odometer information odom to represent the displacement of the mobile device over a period of time; the relocation service can obtain the relocation position point T i w , used to represent the position point of the mobile device at the sampling time under the navigation map, and the navigation map is implemented using the world coordinate system (world). Those skilled in the art can set the type and number of sensors according to specific scenarios, and the corresponding solutions fall within the protection scope of this disclosure.

[0088] It is understandable that, considering that the data collected by each sensor has different frame rates, information synchronization and data interpolation can be performed according to the time information of each type of sensor data, so as to have the effect of the same frame rate.

[0089] Then, the processor of the mobile device can execute a data fusion positioning method to obtain a target positioning pose point representing the current position of the mobile device; then, the above target positioning pose point can be output to the navigation module of the mobile device, and the navigation module controls the mobile device to move to the destination.

[0090] A data fusion positioning method provided by the embodiment of the present disclosure is shown in Figure 2 , including steps 21 to 22.

[0091] In step 21, a fused positioning pose point is generated according to the repositioning pose point and the sensor pose point of the mobile device.

[0092] In this step, the processor of the mobile device can obtain the repositioning pose points and the sensor pose points, and then fuse the repositioning pose points and the sensor pose points to generate fused positioning pose points.

[0093] Considering that the sampling frequency of the relocation pose points of the mobile device is lower than that of the sensor pose points, and the relocation pose points are annotated in the world coordinate system while the sensor pose points are annotated in the visual coordinate system, this step is to convert the sensor pose points in the visual coordinate system to the mapping points in the world coordinate system as the relocation pose points, i.e., the fused positioning pose points. It can be seen that the frequency of the fused positioning pose points is greater than the frequency of the original relocation pose points.

[0094] It should be noted that a mapping relationship, ie, an optimal conversion relationship in subsequent embodiments, is required when the sensor pose points are converted to the world coordinate system, and the optimal conversion relationship is related to the repositioning pose points and the sensor pose points.

[0095] Taking into account that the repositioning pose points and / or sensor pose points may drift, which may cause subsequent results to drift, in this step, the abnormal pose points in the repositioning pose points and / or sensor pose points can be corrected, or in other words, the repositioning pose points and / or sensor pose points used in step 21 are obtained after correction, and when the accuracy of the input data is improved, the accuracy of the fused positioning pose points can be improved.

[0096] In step 22, a target positioning pose point is determined according to the fused positioning pose point and the IMU information of the mobile device, and the target positioning pose point is used to represent the current position of the mobile device.

[0097] In this step, the processor can determine the target positioning pose point based on the fused positioning pose point and the IMU information of the mobile device. For example, the fused positioning pose point and the IMU information of the mobile device can be input into the extended Kalman filter algorithm (ESKF), and the ESKF algorithm calculates the target positioning pose point at the current moment, and the target positioning pose point is used to represent the current position of the mobile device. The processor can output the above target positioning pose point to the navigation module of the mobile device, and the navigation module navigates the route of the mobile device.

[0098] It should be noted that the above-mentioned current time refers to the sampling time corresponding to the data processed by the processor, which is different from the time on the mobile device clock. For example, the sampling time of the repositioning posture point and the sensor posture point is T1, and the calculation time of the processor calculating the target positioning posture point is T2, then T1 is regarded as the current time instead of T2.

[0099] At this point, in this embodiment, by eliminating abnormal posture points in the repositioning posture points and / or sensor posture points, the accuracy of the input data can be improved, which is beneficial to ensuring the robustness of the positioning results; and, in this embodiment, the repositioning posture points and the sensor posture points are generated into fused positioning posture points, which can reduce the number and quantity of input data, which is beneficial to ensuring the smoothness of the positioning results.

[0100] In step 33, if the relocated pose point is determined to be an abnormal pose point, the abnormal pose point is removed from the relocated pose points of the mobile device.

[0101] exist Figure 2 Based on the data fusion positioning method illustrated in the example, the embodiment of the present disclosure also provides another data fusion positioning method, which, in addition to step 21 and step 22, also includes the step of correcting abnormal posture points in the relocated posture points of the mobile device, see Figure 3 , including steps 31 to 33.

[0102] In step 31, a first mapping point of the sensor pose point in the world coordinate system is obtained according to the sensor pose point of the mobile device and the best conversion relationship at the previous moment; the best conversion relationship at the previous moment is used to represent the conversion relationship from the sensor pose point at the previous moment to the best repositioning pose point at the same moment; the best repositioning pose point refers to the repositioning pose point with the smallest offset distance when the sensor pose point is mapped from the visual coordinate system to the world coordinate system.

[0103] In this step, the processor can obtain the sensor pose points And the best conversion relationship at the last moment. Among them, the last best conversion relationship is used to represent the best repositioning pose point at the last moment and the sensor pose point at the same time The conversion relationship from the visual coordinate system (visual) to the mapping point in the world coordinate system (world) can be understood as the conversion relationship between the sensor pose point at the same moment and the optimal repositioning pose point.

[0104] Considering that the subsequent embodiments will describe the scheme of the best conversion relationship of the best repositioning posture point at the current moment, and the best conversion relationship at the current moment will become the best conversion relationship at the previous moment in this step at the next moment, you can refer to the content of the subsequent embodiments, and will not be described here. That is to say, the best conversion relationship at the current moment can be used at the current moment and can be called the current best conversion relationship, and when used at the next moment, it is called the best conversion relationship at the previous moment.

[0105] In one example, during the positioning and navigation process of a mobile device, the position point is relocated is the pose point in the map coordinate system, which has global consistency. Considering the relocation pose point The frequency is 1Hz, and the sensor position point With 30Hz, repositioning of pose points through time alignment and sensor pose points The following conversion relationship needs to be satisfied, as shown in formula (1).

[0106]

[0107] By transforming equation (1), we can get equation (2).

[0108]

[0109] In this step, the processor can calculate the position and posture of the sensor according to formula (1). The best conversion relationship with the previous moment Get the current mapping point of the current sensor pose point in the world coordinate system

[0110] In step 32, it is determined whether the relocated pose point is an abnormal pose point according to the distance between the first mapping point and the relocated pose point.

[0111] It is understandable that the first mapping point after the sensor pose point is converted and the repositioned pose point at the current moment Theoretically, at the same location or with a small difference in distance, a first distance threshold may be set in this step, and the value range of the first distance threshold is [1, 10] meters. In an example, the value of the first distance threshold is 2 meters.

[0112] The processor can obtain the first mapping point and repositioning pose points The distance Distance1 is as shown in formula (3).

[0113]

[0114] In formula (3), Indicates the last best conversion relationship, that is, the best conversion relationship obtained in the last control cycle. represents the sensor pose point, Indicates the current relocation pose point. The first value of the starting conversion relationship Starting conversion relationship The method of obtaining is described in the subsequent embodiments and will not be described here.

[0115] The processor can obtain the first distance threshold and compare the distance Distance1 with the first distance threshold. When the distance Distance1 is less than or equal to the first distance threshold, the processor can determine the positioning pose point at the current time. is a normal pose point, which can be retained and used in subsequent processes; when the distance Distance1 is greater than the first distance threshold, the processor can determine the current repositioning pose point It is an abnormal posture point.

[0116] In step 33, if the relocated pose point is determined to be an abnormal pose point, the abnormal pose point is removed from the relocated pose points of the mobile device.

[0117] In this step, the abnormal posture points in the repositioned posture points of the mobile device are eliminated. Then, the process returns to step 21 to continue to obtain the repositioned posture points at the next moment. After the above operation, the normal posture points of multiple repositioned posture points can be obtained.

[0118] In this way, in this embodiment, by eliminating abnormal posture points in the repositioned posture points and retaining normal posture points, it is beneficial to improve the accuracy of the repositioned posture points, and further to improve the accuracy of the subsequent fused positioning posture points.

[0119] exist Figure 2 or Figure 3 Based on the data fusion positioning method illustrated in the example, the embodiment of the present disclosure also provides another data fusion positioning method, which, in addition to steps 21 to 22, or steps 31 to 33, also includes the step of correcting abnormal posture points in the sensor posture points of the mobile device according to the odometer information, see Figure 4 , including steps 41 to 43.

[0120] In step 41, the odometer information and the sensor pose points are aligned according to the time information.

[0121] In this step, both the odometer information and the sensor pose points include time information, and the odometer information and the sensor pose points can be aligned according to the time information. It is understandable that when the frequencies of the odometer information and the sensor pose points are different, data interpolation can be performed on the data with a lower frequency, or data interpolation can be performed on both the odometer information and the sensor pose points to make the frequencies of the two the same.

[0122] In step 42, the ratio of the change in odometer information to the change in sensor posture point within a preset time length is obtained to obtain a distance proportional factor; the change in odometer information refers to the difference between the odometer information at the start time and the odometer information at the end time within the preset time length; the change in sensor posture point refers to the difference between the sensor posture point at the start time and the sensor posture point at the end time within the preset time length.

[0123] In this step, the processor can obtain the change in odometer information, that is, obtain the difference odom1-odom2 between the odometer information at the start time and the odometer information at the end time within the preset time. In addition, obtain the difference between the sensor pose point at the start time and the sensor pose point at the end time within the preset time. Then, the processor can obtain the ratio of the change in odometer information to the change in sensor pose point within a preset time period to obtain the distance proportional factor e, as shown in formula (4).

[0124]

[0125] In step 43, the sensor pose points whose distance scale factors are outside the preset value range are determined to be abnormal pose points, and the abnormal pose points are removed from the sensor pose points of the mobile device.

[0126] In this step, the processor can obtain a preset value range of the distance scale factor, which is [0.5, 1.5] and can be adjusted according to the specific scenario. Then, the processor can determine whether the distance scale factor is within the preset value range. If it is determined that the distance scale factor is within the preset value range, the processor can determine that the sensor pose point is a normal pose point. If it is determined that the distance scale factor is outside the preset value range, the processor can determine that the sensor pose point is an abnormal pose point.

[0127] When it is determined that the sensor posture point is an abnormal posture point, the processor can remove the abnormal posture point, so that the sensor posture points are all normal posture points.

[0128] It should be noted that in this example, the sensor posture point correction and the repositioning posture point correction are combined. At this time, the sensor posture points are corrected first to obtain accurate sensor posture points, and then the repositioning posture points are corrected to obtain accurate repositioning posture points. The corresponding scheme falls within the protection scope of this disclosure.

[0129] At this point, in this embodiment, by eliminating abnormal posture points in the sensor posture points, the accuracy of the input data can be improved, and then the accuracy of the fused positioning posture points can be improved, which is conducive to ensuring the robustness of the positioning results.

[0130] exist Figure 2 , Figure 3 or Figure 4 Based on the data fusion positioning method illustrated above, the present disclosure also provides another data fusion positioning method, see Figure 5 , generating fused positioning pose points according to the repositioning pose points of the mobile device and the sensor pose points, including steps 51 to 53.

[0131] In step 51, the best repositioning posture point among a first preset number of normal posture points is determined; the first preset number of normal posture points is composed of the repositioning posture point at the current moment, the best repositioning posture point at the previous moment and other normal posture points.

[0132] In this step, it is determined that the repositioning posture point is a normal posture point, and the processor can obtain the first preset number of normal posture points based on the normal posture point. The first preset number of normal posture points is composed of the repositioning posture point at the current moment, the best repositioning posture point at the previous moment, and other normal posture points. In one example, the first preset number is 3, and the processor can obtain the repositioning posture point at the current moment and the best repositioning posture point at the previous moment, as well as a normal repositioning posture point between the current repositioning posture point and the best repositioning posture point at the previous moment, or a previous repositioning posture point earlier than the best repositioning posture point at the previous moment.

[0133] Then, the processor may obtain the best repositioning posture point among the first preset number of normal posture points as the best repositioning posture point at the current moment, and the subsequent conversion relationship calculation process is based on the best repositioning posture point at the current moment.

[0134] In one example, the processor may determine the best repositioned pose point among a first predetermined number of normal pose points, see Figure 6 , including steps 61 to 64.

[0135] In step 61, a transformation relationship corresponding to each normal posture point in a first preset number of normal posture points is obtained; the transformation relationship is used to represent the transformation relationship from the sensor posture point at the same moment of the normal posture point to the normal posture point in the world coordinate system.

[0136] In this step, after obtaining the first preset number of normal posture points, the processor can obtain the conversion relationship corresponding to each normal posture point according to formula (2), that is, each normal posture point in the first preset number of normal posture points corresponds to a conversion relationship. The conversion relationship is used to represent the conversion relationship from the sensor posture point at the same time of the normal posture point to the normal posture point in the world coordinate system.

[0137] See also Figure 7 , taking the first preset number as 3 as an example, the three normal posture points can be 2, 3 and 4, and the normal posture point 4 is the repositioning posture point at the current moment (determined as the normal posture point in the above process), and the normal posture point 3 is the last best repositioning posture point. The normal posture points 2, 3 and 4 correspond to the sensor posture points and Combining formula (2), the corresponding transformation relationship of normal posture points 2, 3 and 4 can be calculated:

[0138] In step 62, for the conversion relationship corresponding to each normal posture point, the distance between each normal posture point and its mapping point is obtained to obtain a first preset number of distances corresponding to each normal posture point.

[0139] It is understandable that, considering that each normal posture point may become the best repositioning posture point, each normal posture point may be assumed as the best repositioning posture point in this step, and the distances between the first preset number of normal posture points and the sensor posture point are calculated. In this way, the processor can obtain the first preset number of distances corresponding to each normal posture point.

[0140] For each normal posture point, the distance between a first preset number of normal posture points and the sensor posture point is calculated, as shown in formula (5).

[0141]

[0142] In formula (5), n is 2, 3 or 4, that is, the second, third or fourth normal posture point.

[0143] Assuming that the normal pose point 2 is the best repositioned pose point, equation (5) can be transformed into equation (6).

[0144]

[0145] In this step, the three distances Distance22, Distance23 and Distance24 corresponding to the normal posture point 2 can be obtained through formula (6).

[0146] Assuming that the normal pose point 3 is the best repositioned pose point, equation (5) can be transformed into equation (7).

[0147]

[0148] In this step, the three distances Distance32, Distance33 and Distance34 corresponding to the normal posture point 3 can be obtained through formula (7).

[0149] Assuming that the normal pose point 4 is the best repositioned pose point, equation (5) can be transformed into equation (8).

[0150]

[0151] In this step, three distances Distance42, Distance43 and Distance44 corresponding to the normal posture point 4 can be obtained through formula (8).

[0152] In step 63, an average value of a first preset number of distances of each normal displacement point is obtained to obtain an average value of a first preset number of distances.

[0153] Considering that when the first preset number of normal displacement points are each used as the optimal repositioning posture point, the corresponding first preset number of distances are different, in this step, the average value of the first preset number of distances of each normal displacement point can be obtained to obtain the average value of the first preset number of distances, as shown in formula (9).

[0154]

[0155] In this step, the processor can obtain the average distance values ​​corresponding to the normal posture points 2, 3 and 4, such as AverageDistance 2, AverageDistance 3 and AverageDistance 4.

[0156] In step 64, the minimum value of the first preset number of distance average values ​​is obtained, and the normal displacement point corresponding to the minimum value is used as the optimal repositioning posture point at the current moment.

[0157] Considering that the best repositioning posture point can make the offset distance of each normal posture point smaller, the minimum value of the first preset number of distance averages, such as min(AverageDistanceN), can be obtained in this step, and the normal posture point corresponding to the minimum value is used as the best repositioning posture point among the above-mentioned first preset number of normal displacement points, that is, the best repositioning posture point at the current moment is obtained. In addition, the processor can use the conversion relationship corresponding to the best repositioning posture point as the best conversion relationship at the current moment. It is understandable that the best conversion relationship at the current moment can be used as the conversion relationship between the current moment and the next moment to convert the sensor posture point to the world coordinate system for use.

[0158] Understandably, according to Figure 6 In the example scenario, the processor can exclude Figure 7 The abnormal pose points 6, 7 and 8 in the image are used to obtain the normal pose points 1, 2, 3, 4, 5, 9 and 10. And according to the above normal pose points, the best relocation pose points and their best transformation relationship are obtained.

[0159] Taking into account the best repositioning posture points selected from the first preset number of normal posture points, there is a very small probability that the offset distance of each normal posture point will be large. In one example, a distance threshold, namely the first preset distance threshold, can be set in advance. The value range of the first preset distance threshold can be [0.5-2]. In this example, the first preset distance threshold is 1 meter.

[0160] After obtaining the minimum value of the first preset number of distance average values, the processor can compare the minimum value with the first preset distance threshold. When the minimum value is less than the first preset distance threshold, the normal posture point corresponding to the minimum value is determined as the best repositioning posture point among the first preset number of normal displacement points.

[0161] When the minimum value is greater than or equal to the first preset distance threshold, the processor may adjust the first preset number to a second preset number, and the first preset number is less than the second preset number, for example, the first preset number is 3 and the second preset number is 5. Then, the second preset number of normal pose points is used to obtain the best repositioning pose point at the current moment.

[0162] By expanding the range of normal pose points to determine the best relocation pose point at the current moment, it is beneficial to improve the accuracy of the best relocation pose point and avoid selecting abnormal pose points as the best relocation pose point.

[0163] In step 52, the best transformation relationship corresponding to the best repositioning pose point is obtained.

[0164] In this step, after determining the best repositioning posture point within the first preset number of normal posture points, the processor can obtain the transformation relationship corresponding to the best repositioning posture point, that is, the best transformation relationship, according to equations (1) and (2).

[0165] Considering that obtaining the best repositioning posture point requires a first preset number of normal posture points, for the first group of the first preset number of normal posture points at the beginning of the walking stage of the mobile device or when the normal posture points are less than the first preset number, the above scheme cannot be used to obtain the best repositioning posture point and its best conversion relationship. Therefore, in this example, the first best conversion relationship is assigned as the starting conversion relationship The starting conversion relationship For how to obtain Figure 8 , including steps 81 to 83.

[0166] In step 81, the mobile device is controlled to rotate in three directions on the spot to obtain three repositioning posture points.

[0167] In this step, the mobile device can be rotated in three directions before navigation, and the processor can obtain three repositioning pose points.

[0168] In step 82, the distance between any two repositioning pose points among the three repositioning pose points is obtained.

[0169] In this step, the processor can obtain the distance between any two repositioning pose points among the three repositioning pose points, as shown in formula (10).

[0170]

[0171] In step 83, if the distance between any two repositioning pose points is less than or equal to the second preset distance threshold (0.3 m), the transformation relationship between the current repositioning pose point and the sensor pose point is obtained as the starting transformation relationship for transforming the sensor pose point from the visual coordinate system to the mapping point in the world coordinate system.

[0172] Considering that the position offset of the mobile device is small when it rotates in place, a distance threshold, namely the second preset distance threshold, can be preset. The value range of the second preset distance threshold is [0.1, 1] meters. In one example, the second preset distance threshold is 0.3 meters. The processor can compare the three distances with the second preset distance threshold respectively, as shown in formula (11).

[0173]

[0174] When it is determined that the distances between any two relocation pose points are less than or equal to the second preset distance threshold (0.3m), it can be determined that the drift of the relocation pose point of the mobile device meets the requirements. At this time, the processor can use any one of the above three relocation pose points as the current relocation pose point, and calculate the transformation relationship between the current relocation pose point and the sensor pose point according to formula (2) as the starting transformation relationship for transforming the current sensor pose point from the visual coordinate system to the mapping point in the world coordinate system.

[0175] It is understandable that, based on the above analysis process, the first preset number of optimal transformation matrices when the number of normal posture points is less than the first preset number are The values ​​of are all starting conversion relations Until the first preset number of normal pose points is met, it is updated to the best transformation relationship corresponding to the best repositioning pose point. It can ensure accurate positioning of mobile equipment while walking and reduce deviation.

[0176] It should be noted that in this example, the sensor posture point correction and the repositioning posture point correction are combined. At this time, the sensor posture points are corrected first to obtain accurate sensor posture points, and then the repositioning posture points are corrected to obtain accurate repositioning posture points. The corresponding scheme falls within the protection scope of this disclosure.

[0177] In step 53, the sensor pose points are mapped to a world coordinate system according to the optimal conversion relationship, and the mapping points of the sensor pose points in the world system are used as fused positioning pose points.

[0178] The processor may generate a fused positioning pose point using the best repositioning pose point determined in step 64 and its corresponding best transformation relationship and the sensor pose point. Figure 7 , assuming that the current best repositioning pose point is the normal pose point 3, then the best transformation relationship at that time is

[0179] In this example, the sensor pose points can be transformed into the world coordinate system, that is, the sensor pose points and the optimal transformation relationship Multiply to get the mapping point As the fused positioning pose point in the world coordinate system. In this way, considering that the frequency of sensor pose points is higher than that of repositioning pose points, converting sensor pose points into mapping points is equivalent to increasing the frequency of repositioning pose points. In addition, the repositioning pose points and sensor pose points are fused into one data.

[0180] In one example, the above-mentioned fused positioning pose point can also obtain the current repositioning pose point by reading the QR code information, and then generate the fused positioning pose point according to the sensor pose point at the same time, which can be suitable for scenes with poor GPS signals such as basements and parking lots.

[0181] At this point, in this embodiment, by obtaining the best repositioning posture points of the first preset number of normal posture points, it is a further selection of the normal posture points to improve the accuracy of the normal posture points; then, the best conversion relationship corresponding to the best repositioning posture points is used to convert the sensor posture points, which can improve the accuracy of the fused positioning posture points.

[0182] exist Figure 2 Based on the data fusion positioning method illustrated above, the present disclosure also provides another data fusion positioning method, see Fig. 9The processor can provide the fused positioning pose point and IMU information as input data to the extended Kalman filter algorithm (ESKF), and the ESKF algorithm calculates the target positioning pose point at the current moment, which is used to represent the current position of the mobile device. The processor can output the above target positioning pose point to the navigation module of the mobile device, and the navigation module navigates the route of the mobile device.

[0183] To this end, in one example, the ESKF Kalman filter algorithm includes:

[0184] The error state variable is shown in formula (12).

[0185]

[0186] In formula (12), p represents translation; v represents velocity; g represents gravity; R represents rotation; ba and bg represent the zero bias of the accelerometer and gyroscope respectively; F represents the linearized Jacobian matrix; cov represents the error state covariance; K represents the Kalman gain; δ represents the symbol of the error state variable, such as δp, δR, δb g ,δb a , δg represent the errors of displacement, rotation, gyroscope bias, acceleration bias and gravity velocity respectively; p t , R t 、b gt 、b at , g t They represent the true values ​​of displacement, rotation, gyroscope bias, acceleration bias and gravity velocity respectively.

[0187] By taking the derivative of each error variable in equation (12), we can obtain the derivative of the error variable in the motion equation, as shown in equation (13).

[0188]

[0189] In formula (13), They represent displacement, velocity, angle, gyroscope bias, acceleration bias and derivative of gravity velocity respectively; They represent the linear acceleration measurement value of the accelerometer and the angular velocity measurement value of the IMU sensor respectively; n g They represent the noise of gyroscope zero bias, linear acceleration zero bias and gravity acceleration respectively.

[0190] The recursive form of the discrete-time state equation of the nominal state variable is shown in equation (14).

[0191]

[0192] In formula (14), t and Δt represent time and time variation respectively; Exp represents an operation of converting a vector into a matrix.

[0193] It can be understood that the first expression in formula (14) is the displacement (or distance) p(t) that obtains the new displacement p(t+Δt) after time Δt; similarly, the second to sixth expressions are update formulas for the parameters IMU velocity value, rotation amount gyroscope bias, accelerometer bias and gravity acceleration value.

[0194] According to the derivative formula and the recursive formula, the error state recursive formula can be obtained, as shown in formula (15).

[0195]

[0196] In formula (15), the first expression represents the recursive solution of the error state of the displacement. After a time Δt, the error state of the displacement at the time t+Δt is solved; similarly, the second to sixth expressions represent the solution of the error state of the velocity, the error state of the rotation, the bias error state of the gyroscope, the error state of the accelerometer, and the error state of the gravity acceleration.

[0197] Based on the above formula (15), the Jacobian matrix can be obtained as shown in formula (16).

[0198]

[0199] In formula (16), I represents the unit matrix.

[0200] It should be noted that the rows and columns of the Jacobian matrix represent the derivatives of each parameter in equation (12).

[0201] Among them, the recursive prediction process of the error state covariance is shown in formula (17).

[0202] cov new =F*cov*F T +Q (17)

[0203] In formula (17), Q represents the noise covariance.

[0204] Visual observations are updated as shown in equation (18).

[0205] y=h(x t )+v,v~N(0,V) (18)

[0206] In formula (18), y represents the observed data, h(x t ) is calculated based on the nominal state (measured value or original data value), and v represents the observation noise, that is:

[0207]

[0208] In formula (19), Represents the predicted value of the position and posture of the IMU sensor to the world coordinate system; The observed value representing the displacement of the IMU sensor to the world coordinate system, that is, the fused positioning pose point obtained in the above steps.

[0209] Considering that the displacement includes translation p and rotation R, transforming equation (19) yields and The observation formula is shown in formula (20).

[0210]

[0211] Formula (20) is a transformation expression of T in formula (19) converted into (translation) displacement P and rotation R.

[0212] It can be seen that the partial derivative of the nominal state with respect to the error state satisfies equation (21).

[0213]

[0214] In formula (21), x t It represents the set of parameters on the left side of each equation in equation (12).

[0215] Then, the error derivatives of t, R with respect to p, v, R, bg, ba, and g are obtained, as shown in formula (22).

[0216]

[0217] Furthermore, the global visual observation sets the noise (weight) as shown in equations (23)-(27).

[0218]

[0219] K=cov new *H T *(H*cov new *H T +V) -1 (twenty four)

[0220] δx=K(yh(x new )) (25)

[0221] x=x new +δx (26)

[0222] cov = (1-KH)cov new (27)

[0223] In equations (23)-(27), V represents the covariance matrix of the observation noise, H represents the Jacobian matrix relative to the error state, cov represents the covariance matrix of the error state of the latest prediction, and cov new represents the corrected error state covariance matrix.

[0224] In this example, the Kalman filter algorithm (ESKF) can combine various formulas to obtain the predicted value, and then use the input observation value (i.e., the fused positioning pose point, the value on the left side of formula (19)) and the predicted value (i.e., the recursive value, the value on the right side of formula (19)) to perform correction (such as formula (20)) based on the theoretical equality principle (such as formula (19)), and obtain the error state (such as formulas (20)-(27)), obtain the current error state for use in the next control cycle, and obtain the predicted value of the current control cycle, i.e., the target positioning pose point (x in formula (26)). Finally, the processor can output the above target positioning pose point to the navigation module to achieve positioning.

[0225] Thus, in this embodiment, the accuracy of the output result can be guaranteed by setting the ESKF algorithm. Considering that the number of input data paths is reduced and the frequency is increased, the smoothness of the positioning result can be guaranteed, and the situation where abnormal posture points cause large drift in positioning and fail to position normally can be avoided.

[0226] Based on a data fusion positioning method provided in an embodiment of the present disclosure, an embodiment of the present disclosure further provides a data fusion positioning device, see Fig.10 ,include:

[0227] The fused positioning pose point generating module 101 is used to generate fused positioning pose points according to the repositioning pose points and sensor pose points of the mobile device; the repositioning pose points and / or the sensor pose points are obtained by correcting abnormal pose points;

[0228] The target positioning pose point determination module 102 is used to determine the target positioning pose point according to the fused positioning pose point and the inertial measurement unit IMU information of the mobile device, and the target positioning pose point is used to represent the current position of the mobile device.

[0229] Optionally, the device further comprises an abnormal posture point correction module, which is used to correct abnormal posture points in the relocated posture points of the mobile device; the abnormal posture point correction module comprises:

[0230] A mapping point acquisition submodule, used for acquiring a first mapping point of the sensor pose point in the world coordinate system according to the sensor pose point of the mobile device and the best conversion relationship at the last moment; the best conversion relationship at the last moment is used to represent the conversion relationship from the sensor pose point at the last moment to the best repositioning pose point at the same moment; the best repositioning pose point refers to the repositioning pose point with the smallest offset distance when the sensor pose point is mapped from the visual coordinate system to the world coordinate system;

[0231] A posture point judgment submodule, used for judging whether the relocated posture point is an abnormal posture point according to the distance between the first mapping point and the relocated posture point;

[0232] The abnormal posture point elimination submodule is used to determine that the repositioned posture point is an abnormal posture point, and then eliminate the abnormal posture point from the repositioned posture points of the mobile device.

[0233] Optionally, the posture point judgment submodule includes:

[0234] A mapping point distance acquisition unit, used to acquire the distance between the first mapping point and the repositioning pose point;

[0235] The abnormal posture point determination unit is used to determine the relocated posture point whose distance is greater than the first distance threshold as an abnormal posture point.

[0236] Optionally, the posture point judgment submodule further includes:

[0237] The normal posture point determination unit is used to determine the repositioned posture point whose distance is less than or equal to the first distance threshold as a normal posture point.

[0238] Optionally, the mobile device further includes an odometer, and the apparatus further includes an abnormal posture point correction module, which is used to correct abnormal posture points in the sensor posture points of the mobile device according to the odometer information; the abnormal posture point correction module includes:

[0239] The sensor pose point alignment submodule is used to align the odometer information and sensor pose points according to the time information;

[0240] The distance scale factor acquisition submodule is used to obtain the ratio of the odometer information change amount to the sensor posture point change amount within a preset time length to obtain the distance scale factor; the odometer information change amount refers to the difference between the odometer information at the start time and the odometer information at the end time within the preset time length; the sensor posture point change amount refers to the difference between the sensor posture point at the start time and the sensor posture point at the end time within the preset time length;

[0241] The abnormal posture point determination submodule is used to determine the sensor posture point whose distance scale factor is outside the preset value range as an abnormal posture point, and remove the abnormal posture point from the sensor posture points of the mobile device.

[0242] Optionally, the fusion positioning pose point generation module includes:

[0243] The best posture point determination submodule is used to determine the best repositioning posture point among a first preset number of normal posture points; the first preset number of normal posture points is composed of the repositioning posture point at the current moment, the best repositioning posture point at the previous moment and other normal posture points;

[0244] An optimal transformation relationship acquisition submodule is used to obtain the optimal transformation relationship corresponding to the optimal repositioning posture point;

[0245] The fusion pose point determination submodule is used to map the sensor pose points to a world coordinate system according to the optimal conversion relationship, and use the mapping points of the sensor pose points in the world system as fusion positioning pose points.

[0246] Optionally, the optimal posture point determination submodule includes:

[0247] A conversion relationship acquisition unit, used to acquire a conversion relationship corresponding to each normal posture point in a first preset number of normal posture points; the conversion relationship is used to represent a conversion relationship from a sensor posture point at the same moment of the normal posture point to the normal posture point in a world coordinate system;

[0248] A posture point distance acquisition unit, for acquiring the distance between each normal posture point and its mapping point according to the conversion relationship corresponding to each normal posture point, and obtaining a first preset number of distances corresponding to each normal posture point;

[0249] A distance average value obtaining unit, used to obtain an average value of a first preset number of distances of each normal displacement point, to obtain a first preset number of distance average values;

[0250] The optimal repositioning posture point acquisition unit is used to obtain the minimum value of the first preset number of distance average values, and use the normal displacement point corresponding to the minimum value as the optimal repositioning posture point at the current moment.

[0251] Optionally, the device further comprises:

[0252] The module for adjusting the number of posture points is used to determine that the minimum value of the average value of the first preset number of distances is greater than or equal to the first preset distance threshold, and then adjust the first preset number of normal posture points to the second preset number of normal posture points and re-acquire the best repositioning posture points, and the first preset number is less than the second preset number.

[0253] Optionally, the device further comprises a starting transformation relationship acquisition module, which is used to acquire a starting transformation relationship of a first optimal repositioning posture point; the starting transformation relationship acquisition module comprises:

[0254] The repositioning posture point acquisition submodule is used to control the mobile device to rotate in three directions on the spot to obtain three repositioning posture points;

[0255] A posture point distance acquisition submodule, used to obtain the distance between any two repositioning posture points among the three repositioning posture points;

[0256] The starting transformation relationship determination submodule is used to determine that the distance between any two repositioning pose points is less than or equal to a second preset distance threshold, and then obtain the transformation relationship between the current repositioning pose point and the sensor pose point as the starting transformation relationship for converting the sensor pose point from the visual coordinate system to the mapping point in the world coordinate system.

[0257] It should be noted that the device embodiment provided in this embodiment corresponds to the above method embodiment. For details, please refer to the contents of the above method embodiments, which will not be repeated here.

[0258] In some possible embodiments, a chip system is also provided, which is used as a processor of the above mobile device, see Fig.11 , the chip system includes at least one processing unit 111 and at least one interface circuit 112. The processing unit 111 and the interface circuit 112 can be interconnected through lines. For example, the interface circuit 112 can be used to receive signals from other devices (such as the memory of an electronic device). For another example, the interface circuit 112 can be used to send signals to other devices (such as the processing unit 111). Exemplarily, the interface circuit 112 can read the instructions stored in the memory and send the instructions to the processing unit 111. When the instructions are executed by the processing unit 111, the alignment device can perform the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, and some embodiments of the present disclosure are not specifically limited to this.

[0259] In one example, the interface circuit 112 can obtain data, program instructions and / or information in the internal storage area of ​​the chip system; it can also obtain data, program instructions and / or information from outside the chip system. In one example, the chip system also includes a memory 113, which is used to store necessary computer programs and data. It is understandable that those skilled in the art can also understand that the various illustrative logical blocks and steps listed in the embodiments of the present disclosure can be implemented by electronic hardware, computer software, or a combination of the two. Whether these functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art can use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present disclosure.

[0260] In some possible embodiments, a mobile device is further provided, including:

[0261] Sensors, processors, and memory;

[0262] The sensor is used to collect relocation pose points, sensor pose points, IMU information and odometer information of the mobile device and send them to the processor;

[0263] The memory is used to store a computer program sensor, processor and memory executable by the processor;

[0264] The processor is configured to execute the computer program in the memory to implement the above method.

[0265] In an exemplary embodiment, a non-transitory computer-readable storage medium is also provided. When an executable computer program in the storage medium is executed by a processor, the above method can be implemented.

[0266] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the disclosure disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0267] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A data fusion positioning method, characterized in that: The method comprises: Generate fused positioning pose points according to the repositioning pose points and sensor pose points of the mobile device; the repositioning pose points and / or the sensor pose points are obtained by correcting abnormal pose points; A target positioning pose point is determined according to the fused positioning pose point and the inertial measurement unit (IMU) information of the mobile device, and the target positioning pose point is used to represent the current position of the mobile device.

2. The method according to claim 1, characterized in that Correcting abnormal posture points in the relocated posture points of the mobile device includes: According to the sensor pose point of the mobile device and the best conversion relationship at the last moment, a first mapping point of the sensor pose point in the world coordinate system is obtained; the best conversion relationship at the last moment is used to represent the conversion relationship from the sensor pose point at the last moment to the best repositioning pose point at the same moment; the best repositioning pose point refers to the repositioning pose point with the smallest offset distance from the sensor pose point mapped from the visual coordinate system to the world coordinate system; Determining whether the relocated pose point is an abnormal pose point according to the distance between the first mapping point and the relocated pose point; If the repositioned pose point is determined to be an abnormal pose point, the abnormal pose point is removed from the repositioned pose points of the mobile device.

3. The method according to claim 2, characterized in that Judging whether the repositioning pose point is an abnormal pose point according to the distance between the first mapping point and the repositioning pose point includes: Obtaining a distance between the first mapping point and the repositioning pose point; Determine the relocated pose point whose distance is greater than the first distance threshold as an abnormal pose point.

4. The method according to claim 2, characterized in that: The method further comprises: Determine the relocated pose point whose distance is less than or equal to the first distance threshold as a normal pose point.

5. The method according to claim 1, characterized in that The mobile device further includes an odometer, and the method further includes correcting abnormal pose points in sensor pose points of the mobile device according to odometer information, including: Align odometer information and sensor pose points based on time information; Obtaining the ratio of the odometer information change to the sensor pose point change within a preset time length to obtain a distance proportional factor; the odometer information change refers to the difference between the odometer information at the start time and the odometer information at the end time within the preset time length; the sensor pose point change refers to the difference between the sensor pose point at the start time and the sensor pose point at the end time within the preset time length; The sensor pose points whose distance scale factors are outside a preset value range are determined as abnormal pose points, and the abnormal pose points are removed from the sensor pose points of the mobile device.

6. The method according to any one of claims 1 to 5, characterized in that: Generate fused positioning pose points based on the repositioning pose points of the mobile device and the sensor pose points, including: Determine the best repositioning posture point among a first preset number of normal posture points; the first preset number of normal posture points is composed of the repositioning posture point at the current moment, the best repositioning posture point at the previous moment, and other normal posture points; Obtaining an optimal transformation relationship corresponding to the optimal repositioning posture point; The sensor pose points are mapped to a world coordinate system according to the optimal conversion relationship, and the mapping points of the sensor pose points in the world system are used as fused positioning pose points.

7. The method according to claim 6, characterized in that Determining the best repositioned pose point among a first preset number of normal pose points includes: Obtaining a conversion relationship corresponding to each normal posture point in a first preset number of normal posture points; the conversion relationship is used to represent a conversion relationship between the sensor posture point at the same moment of the normal posture point and the normal posture point in the world coordinate system; According to the conversion relationship corresponding to each normal posture point, the distance between each normal posture point and its mapping point is obtained to obtain a first preset number of distances corresponding to each normal posture point; Obtaining an average value of a first preset number of distances of each normal displacement point to obtain an average value of the first preset number of distances; The minimum value of the first preset number of distance average values ​​is obtained, and the normal displacement point corresponding to the minimum value is used as the optimal repositioning posture point at the current moment.

8. The method according to claim 7, characterized in that The method further comprises: If it is determined that the minimum value of the first preset number of distance averages is greater than or equal to a first preset distance threshold, the first preset number of normal posture points are adjusted to a second preset number of normal posture points and then the best repositioning posture points are reacquired, and the first preset number is less than the second preset number.

9. The method according to claim 6, characterized in that The method further includes obtaining a starting transformation relationship of a first optimal repositioning pose point, including: Control the mobile device to rotate in three directions to obtain three repositioning points; Obtaining the distance between any two repositioning pose points among the three repositioning pose points; If it is determined that the distance between any two repositioning pose points is less than or equal to the second preset distance threshold, the transformation relationship between the current repositioning pose point and the sensor pose point is obtained as the starting transformation relationship for transforming the sensor pose point from the visual coordinate system to the mapping point in the world coordinate system.

10. A data fusion positioning device, characterized in that: The device comprises: A fused positioning pose point generation module is used to generate fused positioning pose points according to the repositioning pose points and sensor pose points of the mobile device; the repositioning pose points and / or the sensor pose points are obtained by correcting abnormal pose points; The target positioning pose point determination module is used to determine the target positioning pose point according to the fused positioning pose point and the inertial measurement unit IMU information of the mobile device, wherein the target positioning pose point is used to represent the current position of the mobile device.

11. A mobile device, characterized in that: include: Sensors, processors, and memory; The sensor is used to collect relocation pose points, sensor pose points, IMU information and odometer information of the mobile device and send them to the processor; The memory is used to store a computer program executable by the processor; The processor is configured to execute the computer program in the memory to implement the method according to any one of claims 1 to 9.

12. A chip system, characterized in that: The method comprises a processing unit and an interface circuit, wherein the processing unit obtains program instructions through the interface circuit, and the program instructions are executed by the processing unit, and the processing unit is used to execute the method according to any one of claims 1 to 9.

13. A non-transitory computer-readable storage medium, characterized in that: When the executable computer program in the storage medium is executed by a processor, the method according to any one of claims 1 to 9 can be implemented.

Images