Fusion positioning device comprising a plurality of positioning sensors of inertial sensors

Abstract

The application relates to a fusion positioning device of a plurality of positioning sensors comprising an inertial sensor, wherein the fusion positioning device comprises at least one inertial sensor, a first positioning sensor and a second positioning sensor, which are used to respectively collect inertial data of a target and obtain first and second positioning poses of the target; in addition, a fusion positioning component is used to predict a current state quantity according to the inertial data, obtain a current predicted state quantity based on the first positioning pose and / or the second positioning pose, calculate a Kalman gain through a Kalman filtering principle, further obtain a correction value of the current predicted state quantity, and generate a real current state quantity according to the current predicted state quantity and the correction value, so as to obtain a target fusion positioning result. Thus, the problems that, in the related art, when a marker point is shielded for a long time, fusion positioning is prone to deviation, robustness is poor, positioning precision in a large field is low, optical motion capture cannot avoid abnormal mutation, positioning stability is poor and the like are solved.

Description

Technical Field

[0001] This application relates to the field of multi-sensor fusion positioning technology, and in particular to a fusion positioning device for multiple positioning sensors including an inertial sensor. Background Technology

[0002] Optical motion capture is a commonly used positioning technology for indoor rigid body tracking and positioning. An optical motion capture system can quickly and accurately locate actively emitting or passively reflecting markers, providing their coordinates. If three fixed markers form a non-isosceles triangle, or if three or more fixed markers form an asymmetrical structure, they can form a rigid body, and the optical motion capture system can calculate the rigid body's pose. Typically, a rigid body with fixed markers can be considered a positioning sensor, which can be used to locate any device with a fixed positioning sensor, such as drones, VR (Virtual Reality) headsets, and XR (Extended Reality) cameras. However, the problem of sensor occlusion is unavoidable, leading to positioning failure.

[0003] Currently, related technologies can be developed by introducing inertial sensors, or inertial measurement units. These inertial sensors are intrinsic sensors, unaffected by obstructions or the size of the workspace, and their measurement accuracy depends only on the accuracy of the sensor itself. By binding the inertial sensor with an optical positioning sensor and fusing the positioning data from the two sensors through a certain data fusion algorithm, some shortcomings of optical positioning, such as obstruction problems, can be overcome.

[0004] However, when the marker points are occluded for a long time, the fusion positioning of the relevant technologies is prone to deviation, has poor robustness, and has low positioning accuracy in large areas. It is also unable to avoid abnormal changes in optical motion capture, resulting in poor positioning stability, which urgently needs to be addressed. Summary of the Invention

[0005] This application provides a fusion positioning device for multiple positioning sensors, including an inertial sensor, to solve the problems of fusion positioning being prone to deviation when the marker point is occluded for a long time, having poor robustness, low positioning accuracy in large areas, and being unable to avoid abnormal changes in optical motion capture, resulting in poor positioning stability.

[0006] A first aspect of this application provides a fusion positioning device for multiple positioning sensors including an inertial sensor, comprising: at least one inertial sensor for acquiring inertial data of a target based on a sampling frame rate of the at least one inertial sensor; at least one first positioning sensor for obtaining a first positioning pose of the target based on a positioning update frame rate of the at least one first positioning sensor; at least one second positioning sensor for obtaining a second positioning pose of the target based on a positioning update frame rate of the at least one second positioning sensor, wherein the first positioning sensor and the second positioning sensor are of different types; and a fusion positioning component for performing fusion calculations based on the inertial data and the first positioning pose and / or the second positioning pose.

[0007] Optionally, in one embodiment of this application, the fusion positioning component is specifically used to predict the current state quantity based on the inertial data, obtain the current observation quantity based on the first positioning pose and / or the second positioning pose, and obtain the fusion positioning result of the target through weighted calculation.

[0008] Optionally, in one embodiment of this application, the fusion localization component is further used to calculate the fusion localization result of the target based on a Kalman filter.

[0009] Optionally, in one embodiment of this application, the fusion positioning component is further configured to determine whether the at least one first positioning sensor and / or at least one second positioning sensor meets a preset abnormal fluctuation condition before generating the true state value at the current moment based on the current predicted state value and the correction value, and to prohibit the correction of the current predicted state value when the at least one first positioning sensor and / or the at least one second positioning sensor meets the preset abnormal fluctuation condition.

[0010] Optionally, in one embodiment of this application, the fusion positioning component is further configured to determine whether the preset abnormal fluctuation condition is met based on the relative attitude between the current inertial sensor world coordinate system and the positioning sensor world coordinate system of the at least one first positioning sensor and the positioning sensor world coordinate system of the at least one second positioning sensor.

[0011] Optionally, in one embodiment of this application, it further includes: a power supply component, which supplies power to the at least one inertial sensor, the at least one first positioning sensor and / or the at least one second positioning sensor.

[0012] Optionally, in one embodiment of this application, it further includes: a communication component, configured to send operating instructions of the fusion positioning component to the at least one inertial sensor, the at least one first positioning sensor, and the at least one second positioning sensor, and to receive inertial data from the at least one inertial sensor and positioning poses from the at least one first positioning sensor and the at least one second positioning sensor, so as to send the inertial data and the positioning poses to the fusion positioning component.

[0013] Optionally, in one embodiment of this application, the at least one first positioning sensor is either an optical positioning sensor or a visual positioning sensor.

[0014] Optionally, in one embodiment of this application, the at least one second positioning sensor is either a mechanical rocker arm positioning sensor or a visual positioning sensor.

[0015] Optionally, in one embodiment of this application, it further includes a display component for displaying the current operating status of the at least one inertial sensor.

[0016] A second aspect of this application provides a fusion positioning method, comprising the following steps: obtaining a first positioning pose of a target based on the positioning update frame rate of at least one first positioning sensor; obtaining the first positioning pose of the target based on the positioning update frame rate of at least one first positioning sensor; obtaining a second positioning pose of the target based on the positioning update frame rate of at least one second positioning sensor, wherein the first positioning sensor and the second positioning sensor are of different types; and performing fusion calculation based on the inertial data and the first positioning pose and / or the second positioning pose.

[0017] Optionally, in one embodiment of this application, the current state quantity is predicted based on the inertial data, and the current observation quantity is obtained based on the first positioning pose and / or the second positioning pose. The fusion positioning result of the target is obtained by weighted calculation.

[0018] Optionally, in one embodiment of this application, the fusion localization result of the target is calculated based on a Kalman filter.

[0019] Optionally, in one embodiment of this application, before generating the true state value at the current moment based on the current predicted state value and the correction value, the method further includes: determining whether the at least one first positioning sensor and / or the at least one second positioning sensor meets a preset abnormal fluctuation condition; and prohibiting the correction of the current predicted state value when the at least one first positioning sensor and / or the at least one second positioning sensor meets the preset abnormal fluctuation condition.

[0020] Optionally, in one embodiment of this application, determining whether the at least one first positioning sensor and / or at least one second positioning sensor meets the preset abnormal fluctuation condition includes: determining whether the preset abnormal fluctuation condition is met based on the relative attitude between the current inertial sensor world coordinate system and the positioning sensor world coordinate system of the at least one first positioning sensor and the positioning sensor world coordinate system of the at least one second positioning sensor.

[0021] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the fusion positioning method as described in the above embodiments.

[0022] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described fusion positioning method.

[0023] Therefore, the embodiments of this application have the following beneficial effects:

[0024] Embodiments of this application may include at least one inertial sensor for acquiring inertial data of the target based on the sampling frame rate of the at least one inertial sensor; at least one first positioning sensor for obtaining a first positioning pose of the target based on the positioning update frame rate of the at least one first positioning sensor; at least one second positioning sensor for obtaining a second positioning pose of the target based on the positioning update frame rate of the at least one second positioning sensor, wherein the first positioning sensor and the second positioning sensor are of different types; and a fusion positioning component for performing fusion calculations based on the inertial data and the first and / or second positioning poses. This application achieves fusion positioning by combining inertial sensors with optical positioning sensors and mechanical rocker arm positioning sensors, continuously generating rigid body positioning data, thereby effectively improving the accuracy and anti-interference capability of rigid body positioning and avoiding the loss of rigid body motion positioning data during long-term optical motion capture. This solves the problems of related technologies, such as easy deviation in fusion positioning when marker points are occluded for a long time, poor robustness, low positioning accuracy in large areas, inability to avoid abnormal changes in optical motion capture, and poor positioning stability.

[0025] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0026] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0027] Figure 1 This is an example diagram of a fusion positioning device comprising multiple positioning sensors including an inertial sensor, according to an embodiment of this application.

[0028] Figure 2 This is a flowchart of a fusion positioning method provided according to an embodiment of this application;

[0029] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0030] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0031] The following describes a fusion positioning device for multiple positioning sensors, including an inertial sensor, according to embodiments of this application, with reference to the accompanying drawings. Addressing the problems mentioned in the background section, this application provides a fusion positioning device for multiple positioning sensors, including an inertial sensor, comprising: at least one inertial sensor for acquiring inertial data of a target based on the sampling frame rate of the at least one inertial sensor; at least one first positioning sensor for obtaining a first positioning pose of the target based on the positioning update frame rate of the at least one first positioning sensor; at least one second positioning sensor for obtaining a second positioning pose of the target based on the positioning update frame rate of the at least one second positioning sensor, wherein the first and second positioning sensors are of different types; and a fusion positioning component for performing fusion calculations based on the inertial data and the first and / or second positioning poses. This application achieves fusion positioning by combining inertial sensors with optical positioning sensors and mechanical rocker arm positioning sensors, continuously generating rigid body positioning data, thereby effectively improving the accuracy and anti-interference capability of rigid body positioning and avoiding the loss of rigid body motion positioning data during long-term optical motion capture. This solves the problems of poor robustness, low positioning accuracy in large areas, inability to avoid abnormal changes in optical motion capture, and poor positioning stability caused by the fusion positioning when the marker point is occluded for a long time.

[0032] Specifically, Figure 1 This is a block diagram of a fusion positioning device for multiple positioning sensors, including an inertial sensor, according to an embodiment of this application.

[0033] like Figure 1As shown, the fusion positioning device 10, which includes multiple positioning sensors including an inertial sensor, comprises: an inertial sensor 101, a first positioning sensor 102, a second positioning sensor 103, and a fusion positioning component 104.

[0034] Among them, at least one inertial sensor 101 is used to acquire inertial data of the target based on the sampling frame rate of at least one inertial sensor.

[0035] In the embodiments of this application, the accelerometer and gyroscope of the inertial sensor can be used to obtain the current three-axis acceleration data and three-axis angular velocity data based on the sampling frequency. The sampling frequency can reach thousands of hertz, which is much greater than the positioning data frequency of the positioning sensor. The sampling frequency of the inertial sensor in the embodiments of this application can generally be set to 400Hz.

[0036] Therefore, by introducing an inertial sensor, this application provides reliable hardware support for the subsequent realization of optical-inertial fusion positioning.

[0037] Optionally, in one embodiment of this application, the fusion positioning device 10 of multiple positioning sensors including an inertial sensor further includes: a display component for displaying the current operating status of at least one inertial sensor.

[0038] The embodiments of this application can display the working status of the inertial sensor through a display component, thereby enabling users to more intuitively understand the real-time status of the inertial sensor and improving the user-friendliness and reliability of the fusion positioning device of multiple positioning sensors.

[0039] At least one first positioning sensor 102 is used to obtain the first positioning pose of the target based on the positioning update frame rate of the at least one first positioning sensor.

[0040] The embodiments of this application can use the positioning update frame rate of the first positioning sensor to calculate and obtain the positioning pose data of the first positioning sensor at the current moment, thereby providing data basis and guidance for multi-sensor fusion.

[0041] Optionally, in one embodiment of this application, at least one first positioning sensor is either an optical positioning sensor or a visual positioning sensor.

[0042] It should be noted that the embodiments of this application can use an optical motion capture positioning sensor as the first positioning sensor to achieve sensor fusion positioning, which can obtain the target's pose data through a certain positioning frequency (default is 50Hz).

[0043] For example, embodiments of this application can utilize the optical positioning sensor of an optical motion capture system. By binding no fewer than three active emitting or passive reflective marker units to a rigid body, and the passive marker units do not form a symmetrical structure, such as spherical active emitting points or passive reflective spheres, the positioning sensor composed of multiple camera positioning marker units in space can track and locate the rigid body marker unit, thereby obtaining rigid body positioning pose data. Thus, embodiments of this application, by setting appropriate positioning sensors, can obtain reliable rigid body pose and other data, effectively ensuring the realization of sensor fusion positioning.

[0044] It should be noted that in actual implementation, in addition to the optical motion capture and positioning sensors mentioned above, those skilled in the art may also use other devices that can provide rigid body pose as the first positioning sensor, such as camera sensors in SLAM (Simultaneous Localization and Mapping) systems, depending on the actual situation, without making specific restrictions.

[0045] At least one second positioning sensor 103 is used to obtain the second positioning pose of the target based on the positioning update frame rate of the at least one second positioning sensor, wherein the first positioning sensor and the second positioning sensor are of different types.

[0046] Furthermore, embodiments of this application can also utilize the positioning update frame rate of the second positioning sensor (default is 50Hz) to calculate and obtain the positioning pose data of the second positioning sensor at the current moment, thereby further ensuring the realization of the multi-sensor fusion positioning function.

[0047] Optionally, in one embodiment of this application, at least one second positioning sensor is either a mechanical rocker arm positioning sensor or a visual positioning sensor.

[0048] It should be noted that, in the embodiments of this application, the mechanical rocker arm positioning sensor can be used as the second positioning sensor, thereby obtaining the rotation angle of the mechanical rocker arm based on the built-in mechanical encoder, and then calculating the position of the rocker arm end and the position and attitude of the camera fixed at the end; in addition, the above-mentioned second positioning sensor can also be other visual positioning sensors, and those skilled in the art can select them according to the actual situation in the specific implementation process, without making specific limitations.

[0049] For example, in an XR scenario, an XR camera can be fixed to a mechanical rocker arm, and a positioning device including an inertial sensor can be fixed to the camera. At the same time, an optical positioning sensor is fixed to the camera, so that real-time positioning data of three sensors attached to the XR camera can be obtained in real time, namely the acceleration and angular velocity data of the inertial sensor, the optical positioning data of the first positioning sensor, and the mechanical rocker arm positioning data of the second positioning sensor.

[0050] Therefore, the embodiments of this application obtain the rigid body pose mounted on the rocker arm through the mechanical rocker arm positioning sensor, thereby meeting the raw data requirements for subsequent sensor fusion positioning and ensuring the smooth realization of multi-sensor fusion positioning.

[0051] The fusion positioning component 104 is used to perform fusion calculations based on inertial data and a first positioning pose and / or a second positioning pose.

[0052] After acquiring inertial data such as acceleration, angular velocity, and the position, attitude, and velocity of the inertial sensor at the previous moment through the inertial sensor, the embodiments of this application can further perform fusion calculations based on the inertial data acquired by the inertial sensor and the first and second positioning poses of the target to obtain reliable target positioning data.

[0053] Optionally, in one embodiment of this application, the fusion positioning component is specifically used to predict the current state quantity based on inertial data, obtain the current observation quantity based on the first positioning pose and / or the second positioning pose, and obtain the fusion positioning result of the target through weighted calculation.

[0054] Specifically, embodiments of this application can substitute the state variables of the inertial sensor at the previous moment and the acceleration and angular velocity at the current moment into the dynamic equation for calculation. The relationship between acceleration and angular velocity and the position, attitude, and velocity of the inertial sensor is established through the dynamic equation. That is, the velocity is obtained by integrating the acceleration once, the position data is obtained by integrating it again, and the rotation angle is obtained by integrating the angular velocity once, so as to obtain the predicted values ​​of the current position, attitude, velocity and other state variables and the covariance of the current state. When the positioning sensor data is not available, embodiments of this application can continue to calculate and predict the current state variables through the dynamic equation.

[0055] Furthermore, embodiments of this application can obtain the current predicted state quantities of the first positioning sensor and the first positioning sensor, including the zero bias of the inertial sensor's own acceleration and angular velocity, the relative pose of the inertial sensor and the first positioning sensor, the relative attitude of the inertial sensor's world coordinate system and the first positioning sensor's world coordinate system, the scale factor of the inertial sensor and the first positioning sensor, the relative pose of the inertial sensor and the second positioning sensor, the relative attitude of the inertial sensor's world coordinate system and the second positioning sensor's world coordinate system, and the scale factor of the inertial sensor and the second positioning sensor, etc.; wherein, the obtained current predicted state quantity can be used to obtain the current predicted pose of the first positioning sensor through the relative pose of the inertial sensor and the first positioning sensor; the obtained current predicted state quantity can be used to obtain the current predicted pose of the second positioning sensor through the relative pose of the inertial sensor and the second positioning sensor.

[0056] Therefore, the embodiments of this application further improve the reliability and robustness of the positioning data by combining inertial data, the first positioning pose and the second positioning pose to make predictions, obtaining the current state quantity and observation, and using weighted calculation to obtain the target fusion positioning result.

[0057] Optionally, in one embodiment of this application, the fusion localization component is further used to calculate the fusion localization result of the target based on a Kalman filter.

[0058] The embodiments of this application can utilize the Kalman filtering principle through the fusion positioning component, that is, by using the linear system state equation and the system input and output observation data, to make an optimal estimate of the system state, calculate the Kalman gain, obtain the correction of the current predicted state quantity, and obtain the estimate of the true state quantity at the current moment based on the correction of the predicted state quantity.

[0059] It should be noted that those skilled in the art can select appropriate Kalman filtering techniques based on the actual situation during the specific implementation process, such as extended Kalman filtering, unscented Kalman filtering, etc., without making specific restrictions.

[0060] Therefore, the embodiments of this application, through the fusion positioning component, utilize the Kalman filtering principle to achieve fusion positioning of the inertial sensor, the first positioning sensor, and the second positioning sensor, thereby improving the stability and positioning accuracy of the positioning data.

[0061] Optionally, in one embodiment of this application, the fusion positioning component is further configured to determine whether a preset abnormal fluctuation condition is met based on the relative attitude between the current inertial sensor world coordinate system and the positioning sensor world coordinate systems of at least one first positioning sensor and at least one second positioning sensor.

[0062] It should be noted that the embodiments of this application can obtain the relative attitude of the inertial sensor world coordinate system with the first positioning sensor world coordinate system and the relative attitude with the second positioning sensor world coordinate system in the state variables through the fusion positioning component, and save the data of a certain time domain window size. Ideally, the data remains unchanged.

[0063] Subsequently, the median filtering result of the relative attitude between the inertial sensor world coordinate system, the first positioning sensor world coordinate system, and the second positioning sensor world coordinate system is obtained in the time domain window.

[0064] Next, the relative attitudes of the inertial sensor world coordinate system and the world coordinate systems of the first and second positioning sensors at the current moment are obtained respectively. The inverse of the above median filtering result is taken. Ideally, the inverse result is an identity matrix. The comparison between the inverse result and the threshold value is analyzed to determine whether the current positioning sensor has abnormal fluctuations.

[0065] Therefore, the embodiments of this application can effectively solve the problem of long-term occlusion of optical motion capture positioning sensors, avoid the impact of abnormal changes in optical motion capture on fusion positioning performance, and improve the positioning accuracy and robustness of optical motion capture positioning in large areas.

[0066] Optionally, in one embodiment of this application, the fusion positioning component is further configured to determine whether at least one first positioning sensor and / or at least one second positioning sensor meets a preset abnormal fluctuation condition before generating the true state value at the current moment based on the current predicted state value and the correction value, and to prohibit the correction of the current predicted state value when at least one first positioning sensor and / or at least one second positioning sensor meets the preset abnormal fluctuation condition.

[0067] Before generating the true state value at the current moment based on the current predicted state value and the correction value, embodiments of this application can also use the fusion positioning component to determine whether the first positioning sensor and the second positioning sensor have abnormal fluctuations based on the fluctuations of the fused state value. If either positioning sensor has abnormal fluctuations, the state value will not be corrected, the original result will be maintained, and abnormal fluctuations in fusion will be avoided.

[0068] It should be noted that when the positioning sensor experiences abnormal fluctuations, the following state variables are not updated: the acceleration and angular velocity zero bias of the inertial sensor itself, the relative pose between the inertial sensor and the first positioning sensor, the relative attitude between the inertial sensor's world coordinate system and the first positioning sensor's world coordinate system, the scale factor between the inertial sensor and the first positioning sensor, the relative pose between the inertial sensor and the second positioning sensor, the relative attitude between the inertial sensor's world coordinate system and the second positioning sensor's world coordinate system, and the scale factor between the inertial sensor and the second positioning sensor.

[0069] Therefore, the embodiments of this application use a fusion positioning component to detect abnormal fluctuations in the first and second positioning sensors, so that even if the optical positioning sensor data is lost for a long time, the fused data can still be output stably, improving the stability and static accuracy of abnormal changes and greatly improving the performance of fusion positioning.

[0070] Optionally, in one embodiment of this application, the fusion positioning device 10 of multiple positioning sensors including an inertial sensor further includes: a power supply component, which supplies power to at least one inertial sensor, at least one first positioning sensor and / or at least one second positioning sensor.

[0071] In the embodiments of this application, the inertial sensor, the second positioning sensor, and the second positioning sensor can be continuously powered by a wired or wireless power supply component, thereby ensuring that each sensor can work normally and stably.

[0072] Optionally, in one embodiment of this application, the fusion positioning device 10 of the multiple positioning sensors including the inertial sensor further includes: a communication component, configured to send the working instructions of the fusion positioning component to at least one inertial sensor, at least one first positioning sensor and at least one second positioning sensor, and receive the inertial data of at least one inertial sensor and the positioning pose of at least one first positioning sensor and at least one second positioning sensor, so as to send the inertial data and positioning pose to the fusion positioning component.

[0073] It should be noted that the embodiments of this application can also transmit the working instructions of the above-mentioned fusion positioning component, the inertial data of the inertial sensor, and the positioning pose data of the first and second positioning sensors accurately and in real time through wired or wireless communication components, thereby ensuring the real-time performance and reliability of the fusion positioning.

[0074] Furthermore, the embodiments of this application also fix inertial sensors, various positioning sensors, power supply components, and communication components in the basic structural unit, and provide connection interfaces that can be connected and fixed to other positioning devices such as positioning cameras in SLAM systems. This achieves the integration of multiple sensors and other devices, including inertial sensors, ensures the stability of positioning data, improves the versatility of the device, and avoids exposing inertial sensors and other devices, thus providing good protection.

[0075] The fusion positioning device for multiple positioning sensors, including an inertial sensor, proposed according to embodiments of this application includes at least one inertial sensor, a first positioning sensor, and a second positioning sensor to respectively collect inertial data of the target and obtain the first and second positioning poses of the target. Furthermore, a fusion positioning component is used to predict the current state quantity based on the inertial data, and obtain the current predicted state quantity based on the first and / or second positioning poses. A Kalman gain is calculated using the Kalman filtering principle to obtain a correction value for the current predicted state quantity. Finally, the true state quantity at the current moment is generated based on the current predicted state quantity and the correction value, resulting in a target fusion positioning result. This application achieves the fusion of inertial sensor optical positioning and mechanical positioning through the Kalman filtering principle, and avoids abrupt changes in optical motion capture through an anomaly detection algorithm, thereby improving positioning robustness and positioning accuracy for large-area optical motion capture.

[0076] Next, the fusion positioning method proposed according to the embodiments of this application is described with reference to the accompanying drawings.

[0077] Figure 2 This is a flowchart of the fusion positioning method according to an embodiment of this application.

[0078] like Figure 2 As shown, the fusion localization method includes the following steps:

[0079] In step S201, the first positioning pose of the target is obtained based on the positioning update frame rate of at least one first positioning sensor.

[0080] In step S202, the first positioning pose of the target is obtained based on the positioning update frame rate of at least one first positioning sensor.

[0081] In step S203, the second positioning pose of the target is obtained based on the positioning update frame rate of at least one second positioning sensor, wherein the first positioning sensor and the second positioning sensor are of different types.

[0082] In step S204, a fusion calculation is performed based on the inertial data and the first and / or second positioning poses.

[0083] Optionally, in one embodiment of this application, the current state quantity is predicted based on inertial data, and the current observation quantity is obtained based on the first positioning pose and / or the second positioning pose. The fusion positioning result of the target is obtained by weighted calculation.

[0084] Optionally, in one embodiment of this application, the fusion localization result of the target is calculated based on a Kalman filter.

[0085] Optionally, in one embodiment of this application, before generating the true state value at the current moment based on the current predicted state value and the correction value, the method further includes: determining whether at least one first positioning sensor and / or at least one second positioning sensor meets a preset abnormal fluctuation condition; and prohibiting the correction of the current predicted state value when at least one first positioning sensor and / or at least one second positioning sensor meets the preset abnormal fluctuation condition.

[0086] Optionally, in one embodiment of this application, determining whether at least one first positioning sensor and / or at least one second positioning sensor meets a preset abnormal fluctuation condition includes: determining whether the preset abnormal fluctuation condition is met based on the relative attitude between the current inertial sensor world coordinate system and the positioning sensor world coordinate systems of at least one first positioning sensor and at least one second positioning sensor.

[0087] It should be noted that the foregoing explanation of the embodiment of the fusion positioning device including multiple positioning sensors with inertial sensors also applies to the fusion positioning method of this embodiment, and will not be repeated here.

[0088] According to the fusion positioning method proposed in the embodiments of this application, a first positioning pose of the target is obtained based on the positioning update frame rate of at least one first positioning sensor; the first positioning pose of the target is obtained based on the positioning update frame rate of at least one first positioning sensor; and a second positioning pose of the target is obtained based on the positioning update frame rate of at least one second positioning sensor, wherein the first positioning sensor and the second positioning sensor are of different types; fusion calculation is performed based on inertial data and the first positioning pose and / or the second positioning pose, thereby effectively solving the problem of abnormal abrupt changes in optical motion capture, greatly improving the positioning robustness and the accuracy of optical motion capture positioning in large areas.

[0089] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0090] The memory 301, the processor 302, and the computer program stored on the memory 301 and capable of running on the processor 302.

[0091] When the processor 302 executes the program, it implements the fusion positioning method provided in the above embodiments.

[0092] Furthermore, electronic devices also include:

[0093] Communication interface 303 is used for communication between memory 301 and processor 302.

[0094] The memory 301 is used to store computer programs that can run on the processor 302.

[0095] The memory 301 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0096] If the memory 301, processor 302, and communication interface 303 are implemented independently, then the communication interface 303, memory 301, and processor 302 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0097] Optionally, in a specific implementation, if the memory 301, processor 302, and communication interface 303 are integrated on a single chip, then the memory 301, processor 302, and communication interface 303 can communicate with each other through an internal interface.

[0098] Processor 302 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0099] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described fusion positioning method.

[0100] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0101] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0102] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0103] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0104] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0105] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0106] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0107] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A fusion positioning device comprising multiple positioning sensors including an inertial sensor, characterized in that, include: At least one inertial sensor is used to acquire inertial data of the target based on the sampling frame rate of the at least one inertial sensor; At least one first positioning sensor is used to obtain the first positioning pose of the target based on the positioning update frame rate of the at least one first positioning sensor; At least one second positioning sensor is used to acquire a second positioning pose of the target based on a positioning update frame rate of the at least one second positioning sensor, wherein the first positioning sensor and the second positioning sensor are of different types; and A fusion positioning component is used to perform fusion calculations based on the inertial data and the first positioning pose and / or the second positioning pose. The fusion positioning component is further used to determine whether the at least one first positioning sensor and / or at least one second positioning sensor meets a preset abnormal fluctuation condition before generating the true state value at the current moment based on the current predicted state value and the correction value. If the at least one first positioning sensor and / or at least one second positioning sensor meets the preset abnormal fluctuation condition, the correction of the current predicted state value is prohibited. The fusion positioning component is further used to compare the current inertial sensor world coordinate system with the positioning sensor world coordinate system of the at least one first positioning sensor and the positioning sensor world coordinate system of the at least one second positioning sensor. The relative attitude between the world coordinate systems of the positioning sensors is determined to meet the preset abnormal fluctuation conditions. The fusion positioning component acquires the relative attitude between the inertial sensor world coordinate system and the first positioning sensor world coordinate system, and the relative attitude between the inertial sensor world coordinate system and the second positioning sensor world coordinate system in the state variables. It saves data of a certain time domain window size, acquires the median filtering result of the relative attitude between the inertial sensor world coordinate system and the first positioning sensor world coordinate system and the second positioning sensor world coordinate system in the time domain window, acquires the relative attitude between the inertial sensor world coordinate system and the first positioning sensor and the second positioning sensor world coordinate system at the current time, respectively, and inverses the above median filtering result. The inverse result is analyzed and compared with the threshold value to determine whether the current positioning sensor has abnormal fluctuations.

2. The apparatus according to claim 1, characterized in that, The fusion positioning component is specifically used to predict the current state quantity based on the inertial data, obtain the current observation quantity based on the first positioning pose and / or the second positioning pose, and obtain the fusion positioning result of the target through weighted calculation.

3. The apparatus according to claim 2, characterized in that, The fusion localization component is further used to calculate the fusion localization result of the target based on a Kalman filter.

4. The apparatus according to claim 1, characterized in that, Also includes: A power supply component that supplies power to the at least one inertial sensor, the at least one first positioning sensor, and / or the at least one second positioning sensor.

5. The apparatus according to claim 1, characterized in that, Also includes: A communication component is configured to send operating instructions of the fusion positioning component to the at least one inertial sensor, the at least one first positioning sensor, and the at least one second positioning sensor, and to receive inertial data from the at least one inertial sensor and positioning poses from the at least one first positioning sensor and the at least one second positioning sensor, so as to send the inertial data and the positioning poses to the fusion positioning component.

6. The apparatus according to claim 1, characterized in that, The at least one first positioning sensor is either an optical positioning sensor or a visual positioning sensor.

7. The apparatus according to claim 1, characterized in that, The at least one second positioning sensor is either a mechanical rocker arm positioning sensor or a visual positioning sensor.

8. The apparatus according to claim 1, characterized in that, Also includes: A display component is provided for displaying the current operating status of the at least one inertial sensor.

9. A fusion positioning method, characterized in that, A fusion positioning device comprising multiple positioning sensors including an inertial sensor, as described in any one of claims 1-8, wherein the method includes the following steps: Based on the sampling frame rate of the at least one inertial sensor, the inertial data of the target is acquired; Based on the positioning update frame rate of the at least one first positioning sensor, the first positioning pose of the target is obtained; Based on the positioning update frame rate of the at least one second positioning sensor, the second positioning pose of the target is obtained, wherein the first positioning sensor and the second positioning sensor are of different types; and The calculation is performed based on the inertial data and the first and / or second positioning poses; before generating the true state value at the current moment based on the current predicted state value and the correction value, the method further includes: Determine whether the at least one first positioning sensor and / or at least one second positioning sensor meet the preset abnormal fluctuation conditions; When at least one first positioning sensor and / or at least one second positioning sensor meets the preset abnormal fluctuation condition, the correction of the current predicted state value is prohibited. The determination of whether the at least one first positioning sensor and / or at least one second positioning sensor meets the preset abnormal fluctuation condition includes: The relative attitude between the current inertial sensor world coordinate system and the world coordinate systems of at least one first positioning sensor and at least one second positioning sensor is used to determine whether the preset abnormal fluctuation condition is met. The fusion positioning component acquires the relative attitude between the inertial sensor world coordinate system and the world coordinate system of the first positioning sensor, and the relative attitude between the inertial sensor world coordinate system and the world coordinate system of the second positioning sensor in the state variables, saves data of a certain time domain window size, acquires the median filtering result of the relative attitude between the inertial sensor world coordinate system and the world coordinate systems of the first and second positioning sensors in the time domain window, acquires the relative attitude between the inertial sensor world coordinate system and the world coordinate systems of the first and second positioning sensors at the current time, takes the inverse of the above median filtering result, and analyzes the comparison result between the inverse result and the threshold value to determine whether the current positioning sensor has abnormal fluctuations.

10. The method according to claim 9, characterized in that, The current state quantity is predicted based on the inertial data, and the current observation quantity is obtained based on the first positioning pose and / or the second positioning pose. The fusion positioning result of the target is obtained by weighted calculation.

11. The method according to claim 10, characterized in that, The fused localization result of the target is calculated based on the Kalman filter.

12. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the fusion positioning method as described in any one of claims 9-11.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the fusion positioning method as described in any one of claims 9-11.

Images