Target positioning method, device, storage medium and terminal

Abstract

A target positioning method, device, storage medium and terminal, the method obtains inertial measurement data and air pressure data of the target to be positioned and the positioning reference object; calculates the height difference between the target to be positioned and the positioning reference object according to the inertial measurement data and the air pressure data; determines an image acquisition device capable of capturing the image of the target to be positioned based on the height difference, the image acquisition device is loaded in the positioning reference object; obtains the target image of the target to be positioned acquired by the image acquisition device, and locates the target to be positioned based on the target image and the height difference. In this way, the present application can greatly improve the positioning accuracy of the object to be positioned by using inertial measurement data, air pressure data and optical tracking data acquired by image acquisition to perform multi-source perception positioning on the object to be positioned.

Description

Technical Field

[0001] The present application relates to the field of positioning technology, and in particular to a target positioning method, device, storage medium and terminal. Background Art

[0002] Virtual Reality (VR) is a brand-new practical technology developed in the 20th century. Virtual reality technology includes computers, electronic information, and simulation technology. Its basic implementation method is to use computers to simulate virtual environments to give people a sense of environmental immersion. With the continuous development of social productivity and science and technology, the demand for VR technology in all walks of life is growing. VR technology has also made great progress and has gradually become a new field of science and technology.

[0003] The application of virtual reality technology in games has greatly enhanced people's gaming experience. Virtual reality games use computer simulation to create a three-dimensional virtual world, providing users with simulations of vision, hearing, touch and other senses, allowing users to experience an immersive experience and interact freely with things in the space. In virtual reality games, players generally need to use a game helmet and two six-degrees-of-freedom (6DoF) game controllers to perform various game operations. During VR games, the 6DoF controller needs to be positioned in real time, and the current positioning technology is not accurate enough for positioning the 6DoF controller. Summary of the invention

[0004] The present application provides a target positioning method, device, storage medium and terminal, which can improve the accuracy of 6DoF handle positioning.

[0005] In a first aspect, the present application provides a target positioning method, the method comprising:

[0006] Acquire inertial measurement data of the target to be positioned and the positioning reference object, as well as air pressure data;

[0007] Calculating a height difference between the target to be positioned and the positioning reference object according to the inertial measurement data and the air pressure data;

[0008] Based on the height difference, determining an image acquisition device capable of capturing the image of the target to be positioned, wherein the image acquisition device is loaded in the positioning reference object;

[0009] The target image of the target to be located acquired by the image acquisition device is acquired, and the target to be located is located based on the target image and the height difference.

[0010] Accordingly, a second aspect of the present application provides a target positioning device, the device comprising:

[0011] An acquisition module, used to acquire inertial measurement data of the target to be positioned and the positioning reference object, as well as air pressure data;

[0012] A calculation module, used for calculating the height difference between the target to be located and the positioning reference object according to the inertial measurement data and the air pressure data;

[0013] A determination module, configured to determine, based on the height difference, an image acquisition device capable of capturing the image of the target to be positioned, wherein the image acquisition device is loaded in the positioning reference object;

[0014] The positioning module is used to obtain the target image of the target to be positioned acquired by the image acquisition device, and to position the target to be positioned based on the target image and the height difference.

[0015] In a third aspect, the present application provides a storage medium having a computer program stored thereon, which, when loaded by a processor of an electronic device, executes the steps in any target positioning method provided in the present application.

[0016] In a fourth aspect, the present application further provides an electronic device, which includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the steps of any target positioning method provided in the present application by loading the computer program stored in the memory.

[0017] The technical solution provided by the present application is adopted, that is, obtaining the inertial measurement data and air pressure data of the target to be located and the positioning reference object; calculating the height difference between the target to be located and the positioning reference object based on the inertial measurement data and the air pressure data; determining an image acquisition device capable of capturing the image of the target to be located based on the height difference, the image acquisition device being loaded in the positioning reference object; obtaining the target image of the target to be located acquired by the image acquisition device, and locating the target to be located based on the target image and the height difference. In this way, the present application can greatly improve the positioning accuracy of the target to be located by using inertial measurement data, air pressure data and optical tracking data acquired by image acquisition to perform multi-source perception positioning on the target to be located. When the positioning solution provided by the present application is applied to a 6DoF handle, the positioning accuracy of the 6DoF handle can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 It is a flow chart of a target positioning method provided in an embodiment of the present application.

[0020] Figure 2 It is a schematic diagram of the principle of judging whether the handle is within the imaging range of the camera.

[0021] Figure 3 It is another principle schematic diagram for judging whether the handle is within the imaging range of the camera.

[0022] Figure 4 It is a structural block diagram of the target positioning device provided in an embodiment of the present application.

[0023] Figure 5 It is a structural block diagram of the terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] It should be noted that the terms "first", "second" and "third" in this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not limited to the listed steps or modules, but some embodiments also include steps or modules that are not listed, or some embodiments also include other steps or modules inherent to these processes, methods, products or devices.

[0025] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0026] The embodiments of the present application provide a target positioning method, device, storage medium and terminal, wherein the execution subject of the target positioning method may be the target positioning device provided in the embodiments of the present application, or an electronic device integrating the target positioning device, wherein the target positioning device may be implemented in hardware or software. Among them, the electronic device may be a mobile terminal. Among them, the mobile terminal may be a smart phone, a tablet, a vehicle-mounted terminal or a smart wearable device such as VR glasses, VR helmets, etc.

[0027] Please refer to Figure 1 , Figure 1 A flow chart of a target positioning method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the process of the target positioning method provided in the embodiment of the present application can be as follows:

[0028] In 110 , inertial measurement data of the target to be positioned and the positioning reference object, as well as air pressure data are acquired.

[0029] Among them, the target positioning method provided in the present application can be specifically applied to a target positioning device, and the target positioning device can be specifically integrated in a terminal.

[0030] Among them, the target positioning method provided by the present application can be introduced by taking the positioning of the 6DoF handle as an example. The 6DoF handle can be used in conjunction with a VR helmet to convert the user's operation of the 6DoF handle in the real world to the virtual world displayed by the VR helmet. For example, the user can use the 6DoF handle to control the virtual character in the virtual world. The current mainstream 6DoF handle positioning and tracking solutions include outside-in tracking / lighthouse tracking and inside-out tracking. 6DoF means six degrees of freedom. An object has six degrees of freedom in space, namely the freedom of movement along the three rectangular coordinate axes of x, y, and z and the freedom of rotation around these three coordinate axes. Therefore, to fully determine the position of an object, these six degrees of freedom must be clear. The outside-in tracking solution can be implemented using technical means such as laser and infrared, and the inside-out tracking solution can be implemented using technical means such as optics and electromagnetics. Specifically, the outside-in tracking solution requires the installation of at least two base stations in the use area. These base stations are used as reference points for glasses and handles. Each base station contains an infrared beacon and two laser emitters, while each controller has multiple infrared light sensors. The base station's beacon emits synchronized pulses 60 times per second, while one of the lasers emits a beam that sweeps across the room. Once the controllers detect a pulse, they start counting until one of the sensors is hit by the laser beam. Based on the laser's flight time, the system detects the controller's position. In the inside-out tracking scheme, each controller has a set of infrared LEDs located on the controller ring. The camera on the glasses detects the above LEDs and continuously takes images of them. Based on these images, the position of the controller in space is triangulated. In order to improve the positioning and tracking performance of 6DoF controllers, some outside-in tracking solutions and inside-out tracking solutions optimize positioning and tracking by loosely or tightly combining with inertial measurement unit (IMU) information. Loose combination refers to independent positioning and tracking using outside-in tracking or inside-out tracking solutions, and tight combination refers to tracking solutions that combine outside-in tracking solutions and inside-out tracking solutions.

[0031] Both the existing outside-in tracking and inside-out tracking solutions have certain problems. Even if the IMU information is added to optimize the positioning and tracking results, the space and duration of accuracy optimization are limited. In particular, the average accuracy of existing VR devices and their accompanying 6DoF controllers is already very high, but the high average accuracy of 6DoF does not mean a good user experience. It is necessary to improve the robustness of the 6DoF controller in situations such as high dynamic motion, poor visibility conditions, and environmental interference.

[0032] The advantages of the outside-in tracking solution are higher positioning and tracking accuracy and lightweight data transmission; the advantages of the inside-out tracking solution are lower cost, no need to purchase, deploy, and set up external base stations, and more convenient to use. With the advancement of VR software and hardware and the evolution of inside-out tracking technology, there is a lot of room for improvement in the accuracy and performance of inside-out tracking.

[0033] The disadvantage of the ultrasonic positioning solution is that the greater the distance, the faster the ultrasonic signal attenuates, which in turn causes a large change in 6DoF accuracy. In addition, although ultrasonic waves are not affected by ambient light, they are affected by ultrasonic-related interference.

[0034] The advantage of the electromagnetic positioning solution is that it has no field of view (FOV) restrictions and can perform all-round tracking and positioning interactions. However, the higher the accuracy requirements, the greater the power consumption of the electromagnetic sensor, causing the controller to heat up and the hands to sweat. In addition, electromagnetic positioning will be interfered by external magnetic fields, and there will also be interference between devices, so multiple devices cannot work in the same space at the same time.

[0035] Both ultrasonic solutions and electromagnetic solutions are outside-in solutions, which require the deployment of special positioning base stations, increase the cost of the final product, and reduce ease of use.

[0036] The optical solution can reuse the helmet's camera sensor to lower the product cost and reduce the power consumption of the handle. When the camera FOV is designed reasonably, it can basically guarantee the positioning and tracking accuracy. Therefore, the optical handle positioning and tracking solution is better than other solutions in the large FOV tracking range.

[0037] Although the optical-based solution is the current trend, there are still many problems that need to be solved and improved. When actually using VR equipment, there may be interference from the external environment, the movement of the handle may be very violent, and there is often a poor or even no line of sight between the camera on the helmet and the handle LED feature points, resulting in large optical feature point tracking errors or even tracking failures. IMU can maintain posture accuracy for a long time but has poor position accuracy maintenance capabilities. Therefore, in some VR applications, abnormal 6DoF results of the handle often occur, affecting the user experience. In this regard, the present application provides a target positioning method for improving the problem of inaccurate 6DoF handle positioning in the above situation. The target positioning method provided by the present application is described in detail below.

[0038] First, the inertial measurement data of the target to be positioned and the positioning reference object can be obtained, as well as the air pressure data of the target to be positioned and the positioning reference object. The target to be positioned can be a 6DoF controller in a VR game, and the positioning reference object can be a VR helmet in a VR game. The task of positioning the target to be positioned can specifically be determining the position data of the target to be positioned relative to the positioning reference object, that is, determining the position of the 6DoF controller in the VR game relative to the VR helmet.

[0039] The inertial measurement data of the target to be located and the positioning reference object can specifically be data measured by an inertial measurement unit IMU, including acceleration data and three-axis attitude angle (angular velocity) of the target to be located and the positioning reference object, wherein the acceleration data can be measured by an accelerometer, and the angular velocity can be measured by a gyroscope. In actual use, the measurement accuracy of the accelerometer and the gyroscope will be affected by various interference factors and cause deviations. Therefore, in an embodiment of the present application, the air pressure data of the target to be located and the positioning reference object can be further obtained, so as to obtain a more accurate positioning result by fusing the air pressure data and the inertial measurement data. Among them, the air pressure data of the target to be located and the positioning reference object can be measured by a barometer.

[0040] In 120 , the height difference between the target to be positioned and the positioning reference object is calculated according to the inertial measurement data and the air pressure data.

[0041] After the inertial measurement data and the air pressure data of the target to be positioned and the positioning reference object are acquired, the height difference between the target to be positioned and the positioning reference object can be calculated according to the inertial measurement data and the air pressure data.

[0042] In some embodiments, calculating the height difference between the target to be positioned and the positioning reference object according to the inertial measurement data and the air pressure data includes:

[0043] 1. Determine the first height difference between the target to be positioned and the positioning reference object based on the inertial measurement data;

[0044] 2. Determine the second height difference between the target to be located and the positioning reference object based on the air pressure data;

[0045] 3. Filter and fuse the first height difference and the second height difference to obtain the height difference between the target to be located and the positioning reference object.

[0046] In an embodiment of the present application, after determining the inertial measurement data and air pressure data of the target to be positioned and the positioning reference object (i.e., the VR helmet and the 6DoF handle), the inertial fusion differential air pressure altitude algorithm provided by this solution can be used to calculate the height difference between the helmet and the handle. Specifically, the first height difference between the helmet and the 6DoF handle can be calculated based on the inertial measurement data, which can be calculated using the IMU mechanical arrangement method. The accelerometer and gyroscope in the IMU measure the original data as velocity increments and angle increments, and the IMU original output needs to be converted into attitude, velocity, and position information through mechanical arrangement. Then, the height difference between the two is determined based on the difference between the position information to obtain the first height difference.

[0047] Furthermore, after determining the air pressure data of the helmet and the 6DoF handle, the air pressure data of the two can be used to calculate the second height difference between the two. Specifically, the air pressure difference algorithm can be used to calculate the height difference between the helmet and the 6DoF handle based on the air pressure data obtained by the barometer in the helmet and the 6DoF handle. In order to ensure user safety, VR games are generally played in indoor environments, and the straight-line distance between the helmet and the handle generally does not exceed 1.5m, so that the environmental difference between the helmet and the 6DoF handle is very small, that is, the error of the air pressure height difference will not increase due to changes in the indoor environment. The basic principle of the differential air pressure height algorithm is to use the air pressure difference method to correct the deviation caused by the influence of environmental factors encountered in the atmospheric pressure height measurement method, so as to improve the accuracy and reliability of the air pressure height measurement method.

[0048] In the earth's gravity field, atmospheric pressure has the characteristic of changing with altitude, that is, atmospheric pressure decreases with increasing altitude. Therefore, by measuring the atmospheric pressure at the location of the sensor, the atmospheric pressure altitude at the location of the sensor can be indirectly measured based on the relationship between air pressure and altitude. The International Organization for Standardization proposed the "International Standard Atmosphere", which stipulates that the relationship between atmospheric parameters and altitude within the altitude range of -2km to 80km satisfies the following formula:

[0049]

[0050] Where H is the pressure altitude, P H is the measured value of air pressure, g is the acceleration of gravity, β is the temperature gradient coefficient of the space atmosphere, T b , P b , H b When the altitude is between 0 and 11 km, β = -0.00065 K / m, T b =288.15K, P b =101.325Kpa, H b =0.

[0051] The commonly used differential pressure height difference formula is as follows:

[0052]

[0053] Where h is the air pressure height of the handle, h0 is the air pressure height of the helmet, R d is the gas constant of dry air, g is the acceleration due to gravity, T m is the ambient temperature, P is the air pressure of the handle, and P0 is the air pressure of the helmet.

[0054] Unlike the application of differential pressure altitude algorithm in other fields, in the embodiment of the present application, since the distance between the VR helmet and the 6DoF handle is very close, the external environment of the VR helmet and the 6DoF handle is more similar. Therefore, the differential pressure altitude algorithm in the method has a better correction effect on the deviation caused by environmental factors, that is, it has higher accuracy and reliability; the accuracy requirement of differential pressure altitude in indoor positioning applications is relatively low, and it is sufficient to distinguish floors. It is generally believed that an accuracy of about 1m can meet the requirements, but in this patent scheme, the elevation accuracy requirement is higher. The current relative accuracy of consumer-grade barometers is about 2.5Pa (Pascal), and the noise is about 0.65Pa. After correcting the errors caused by environmental factors and the barometer error, the elevation difference accuracy is about decimeter level, which cannot meet the application requirements well. Therefore, it is necessary to improve the accuracy of the estimated altitude difference through inertial fusion pressure altitude algorithm. In general, this method uses the error state Kalman filter to fuse inertial and pressure altitude. The state vector of the filter is altitude. The inertial navigation mechanical arrangement is used to predict the handle altitude. The pressure altitude is used to update the altitude. The initial value of the helmet state is provided by the helmet 6DoF, and the state initialization of the handle is provided by the handle 6DoF obtained by the helmet camera optically tracking the handle LED feature points, thereby ensuring that the height information of the helmet and the handle is in the same world coordinate system. After the state vector is initialized, the state estimate can be obtained from the measurement vector, and the corresponding error covariance matrix describes the statistical characteristics of the observation noise. The observation model, like the system model, is also established based on the known system model, describing the functional relationship between the measurement vector and the state vector in the absence of observation noise. The filter model obtains the optimal estimate of the state vector in real time through the measurement vector, observation model, and system model in a "prediction-update" mode.

[0055] In some embodiments, the target positioning method provided by the present application further includes:

[0056] A. performing error detection on the inertial measurement data to obtain an error detection result;

[0057] B. correcting the first height difference based on the error detection result to obtain a corrected height difference;

[0058] The filtering and fusing the first height difference and the second height difference to obtain the height difference between the target to be located and the positioning reference object includes:

[0059] C. Filter and fuse the corrected height difference and the second height difference to obtain a height difference between the target to be located and the positioning reference object.

[0060] In an embodiment of the present application, in order to reduce the influence of IMU error on the integral error generated in the fusion positioning process, the IMU error can be estimated before calculation to obtain an error detection result. Then, the error detection result can be used to correct the first height difference calculated using inertial measurement data to obtain a corrected height difference. Finally, the corrected height difference is filtered and fused with the second height difference calculated using the air pressure data to obtain the height difference between the VR helmet and the 6DoF handle. This method avoids the influence of the accumulation of IMU errors on the fusion positioning results and further improves the accuracy of the 6DoF handle positioning.

[0061] In some embodiments, performing error detection on the inertial measurement data to obtain an error detection result includes:

[0062] A1. Acquire first multi-degree-of-freedom data of the calibration object obtained by mechanical arrangement based on inertial measurement data of the calibration object;

[0063] A2. Acquire second multi-DOF data of the calibration object based on the optical tracking result of the calibration object;

[0064] A3. Determine an error detection result according to the difference between the first multi-degree-of-freedom data and the second multi-degree-of-freedom data.

[0065] Among them, in an embodiment of the present application, the IMU error can be determined in the initialization stage by combining the measures in the outside-in scheme and the inside-out scheme. Specifically, a calibration object can be arbitrarily determined, for example, a 6DoF handle is used as the calibration object. Then, the IMU mechanical arrangement method is used to obtain the first multi-degree-of-freedom data of the handle, that is, the first 6DoF result of the handle is obtained; then the optical tracking method in the outside-in method can be used to obtain the second multi-degree-of-freedom data of the handle, that is, the second 6DoF result of the handle is obtained. Furthermore, the IMU error can be determined based on the handle 6DoF result obtained by optical tracking and the handle 6DoF result obtained by IMU mechanical arrangement, so as to achieve the error detection result of the IMU data.

[0066] In 130 , an image acquisition device capable of capturing an image of the target to be positioned is determined based on the height difference, and the image acquisition device is loaded in the positioning reference object.

[0067] Furthermore, in an embodiment of the present application, the positioning reference object may also be loaded with an image acquisition device, that is, may be loaded with a camera. Specifically, a camera may be loaded in the VR helmet, and specifically one or more cameras may be loaded. The camera may be used to collect image information of the 6DoF handle. By setting the field of view of the camera loaded in the VR helmet, the image acquisition range of the camera may include various positions where the handle may appear. However, there are differences between the image acquisition areas of different cameras.

[0068] After calculating the height difference between the target to be located and the positioning reference object, it is possible to further estimate in which camera or cameras the handle may appear in the image acquisition range according to the height difference between the target to be located and the positioning reference object.

[0069] In some embodiments, an image acquisition device capable of capturing an image of a target to be positioned is determined based on the height difference, and the image acquisition device is loaded in a positioning reference object, including:

[0070] 1. Obtain imaging range information of each image acquisition device loaded in the positioning reference object;

[0071] 2. Determine an image acquisition device that can capture an image corresponding to the target to be located based on the height difference and the horizontal distance constraint between the target to be located and the positioning reference object.

[0072] Specifically, in the embodiment of the present application, when the height difference between the VR helmet and the 6DoF handle is determined, it is necessary to further obtain the imaging range information of each camera installed in the VR helmet, that is, to obtain the FOV information of each camera. Then, according to the height difference and the horizontal distance constraint between the VR helmet and the 6DoF handle, the camera that can capture the 6DoF handle image is determined, and the camera can be one or more. Furthermore, the range information where the handle may appear in the photo taken by the camera can also be determined.

[0073] In some embodiments, in order to further ensure the accuracy of positioning, the difference between the 6DoF data of the VR helmet and the 6DoF data of each camera needs to be considered in the above process. Specifically, the conversion can be performed based on the external parameters between the camera and the IMU. The pre-acquired helmet 6DoF data is converted to the camera 6DoF data with the camera as the origin. Then, based on the 6DoF data of each camera, the FOV of the camera, and the height difference data from the 6DoF handle to each camera, it is determined whether the 6DoF may appear within the FOV range of the camera and the possible position.

[0074] The factors determining whether the 6DoF handle may be within the camera's FOV and its possible positions include the height difference between the 6DoF handle and the camera, the horizontal distance between the 6DoF handle and the camera, the angle of the camera, etc. During actual use, the ranges of the height difference and the horizontal distance are restricted by the arm length, so empirical values (such as 1.2 m) are used for the restriction. In addition, in extreme cases such as optical tracking failure, the handle cannot obtain horizontal displacement information and can only obtain elevation and attitude information. Therefore, to make the method of the present application more universal, only the influence of the camera's pitch angle on the imaging of the handle is analyzed to determine the imaging range of the handle in the V direction of the image. The schematic diagram of the principle is as Figure 2 and Figure 3 shown. As Figure 2 shown, the handle 20 is outside the FOV of the camera 10, and the handle 30 is within the FOV of the camera 10. The method for determining whether the 6DoF handle is within the camera's imaging range is as follows: 1) Determine the imaging range of the camera, and determine the height range that the camera can image through the camera's FOV parameter (vertical FOV) and the horizontal angle (assuming the maximum distance from the handle to the camera is 1.5 m); 2) Determine whether the 6DoF handle can be imaged in the camera through the height difference information between the 6DoF handle and the camera. Considering that there is a certain error in the height difference, a certain redundancy needs to be reserved when determining whether it is possible to be imaged in the camera. When the following conditions are met, it is considered that the handle may be within the camera's imaging range:

[0075] -dH2 - d1 < dH < dH1 + d2. Wherein, dH is the height difference between the camera and the 6DoF handle (the height of the 6DoF handle - the height of the camera), and dH2 and dH1 are defined as Figure 3 shown, and d1 and d2 are the height difference redundancies, which depend on the height difference accuracy and the size of the 6DoF handle.

[0076] In 140, obtain the target image of the target to be located collected by the image acquisition device, and locate the target to be located based on the target image and the height difference.

[0077] Further, after determining the image acquisition devices that can collect the images of the target to be located, the target images of the target to be located collected by these image acquisition devices can be further obtained, and then the target to be located can be further located based on the target image and the aforementioned height difference.

[0078] Specifically, the images of the 6DoF handle captured in the VR headset can be obtained, and then the handle can be located according to these images and the height difference between the headset and the 6DoF handle.

[0079] In some embodiments, locating the target to be located based on the target image and the height difference includes:

[0080] 1. Identify the target to be located in the target image and obtain the position information of the target to be located in the target image;

[0081] 2. Calculate the horizontal distance and rotation angle information between the target to be located and the positioning reference object based on the position information;

[0082] 3. Determine the relative position relationship between the target to be located and the positioning reference object based on the horizontal distance, rotation angle information and height difference.

[0083] Among them, in an embodiment of the present application, after determining the camera information that can capture the 6DoF handle, the images of the 6DoF handle captured by these cameras are obtained. The image can then be preprocessed, including distortion correction, Gaussian filtering, binarization, region of interest (ROI) extraction, etc. Among them, ROI extraction can be extracted within the imaging range of the aforementioned determined handle in the image. In some embodiments, in order to facilitate the determination of the position of the 6DoF handle in the image, an LED light ring can be set in the 6DoF handle, thereby speeding up the determination of the feature area of ​​interest and the recognition and matching of feature points, and avoiding the impact of environmental interference in other areas on the visual tracking system. Then, the horizontal distance and rotation angle data between the 6DoF handle and the VR helmet can be calculated based on the position of the 6DoF handle in the image, and the relative position relationship between the 6DoF handle and the VR helmet can be further comprehensively determined in combination with the aforementioned height difference.

[0084] In some embodiments, the target positioning method provided by the present application may also include:

[0085] A. Obtain multi-degree-of-freedom data of the positioning reference object;

[0086] B. Determine the target multi-degree-of-freedom data of the target to be located based on the multi-degree-of-freedom data and the positional relationship.

[0087] Among them, in the above method, the relative position relationship between the 6DoF handle and the VR helmet is determined. Further, the 6DoF data of the VR helmet can be obtained, and then the final 6DoF data of the 6DoF handle can be determined according to the relative position relationship between the 6DoF handle and the VR helmet, that is, the VR helmet and the 6DoF handle are unified in a world coordinate system for positioning, and more universal positioning data is obtained.

[0088] In some embodiments, when the optical tracking effect is poor or fails, the IMU, barometer and motion model constraints can be used to utilize the height difference information to suppress the divergence of the height difference direction error of the inertial navigation mechanical arrangement, and the human hand motion range information can be used to constrain the horizontal error range of the inertial navigation mechanical arrangement to reduce the occurrence of handle tracking failure.

[0089] According to the above description, the target positioning method provided by the present application obtains inertial measurement data and air pressure data of the target to be positioned and the positioning reference object; calculates the height difference between the target to be positioned and the positioning reference object based on the inertial measurement data and air pressure data; determines an image acquisition device capable of capturing the image of the target to be positioned based on the height difference, and the image acquisition device is loaded in the positioning reference object; obtains the target image of the target to be positioned captured by the image acquisition device, and locates the target to be positioned based on the target image and the height difference. In this way, the present application uses inertial measurement data, air pressure data, and optical tracking data acquired by the image to perform multi-source perception positioning on the target to be positioned, thereby greatly improving the positioning accuracy of the target to be positioned.

[0090] Specifically, the target positioning method proposed in this application can improve the efficiency, accuracy and reliability of the handle's 6DoF positioning tracking. Specifically, reliable height difference information can be obtained by fusing the IMU with the differential pressure altitude algorithm. On the one hand, it can assist in determining whether the handle is within the camera's imaging range and the imaging area, thereby speeding up image processing. On the other hand, the barometer, IMU and motion constraint model can be fused to improve the accuracy and reliability of the handle's 6DOF when optical tracking fails, thereby improving the robustness of the system.

[0091] Please refer to Figure 4 , Figure 4 Schematic diagram of a target positioning device 200 provided in an embodiment of the present application. The target positioning device 200 is applied to the electronic device provided in the present application. Figure 4 As shown, the target positioning device 200 may include:

[0092] An acquisition module 210 is used to acquire inertial measurement data of the target to be positioned and the positioning reference object, as well as air pressure data;

[0093] A calculation module 220 is used to calculate the height difference between the target to be located and the positioning reference object according to the inertial measurement data and the air pressure data;

[0094] A determination module 230, for determining an image acquisition device capable of capturing an image of a target to be positioned based on the height difference, the image acquisition device being loaded in the positioning reference object;

[0095] The positioning module 240 is used to obtain the target image of the target to be positioned acquired by the image acquisition device, and to locate the target to be positioned based on the target image and the height difference.

[0096] Optionally, in some embodiments, the computing module includes:

[0097] A first determination submodule is used to determine a first height difference between a target to be positioned and a positioning reference object according to inertial measurement data;

[0098] A second determination submodule, used to determine a second height difference between the target to be positioned and the positioning reference object according to the air pressure data;

[0099] The fusion submodule is used to filter and fuse the first height difference and the second height difference to obtain the height difference between the target to be located and the positioning reference object.

[0100] Optionally, in some embodiments, the target positioning device provided by the present application further includes:

[0101] The detection submodule is used to perform error detection on the inertial measurement data and obtain error detection results;

[0102] A correction submodule, used for correcting the first height difference based on the error detection result to obtain a corrected height difference;

[0103] The fusion submodule is also used for:

[0104] The corrected height difference and the second height difference are filtered and fused to obtain the height difference between the target to be positioned and the positioning reference object.

[0105] Optionally, in some embodiments, the detection submodule includes:

[0106] A first acquisition unit, used for acquiring first multi-degree-of-freedom data of the calibration object obtained by mechanical arrangement based on inertial measurement data of the calibration object;

[0107] A second acquisition unit, used for acquiring second multi-degree-of-freedom data of the calibration object based on the optical tracking result of the calibration object;

[0108] The determination unit is used to determine the error detection result according to the difference between the first multi-degree-of-freedom data and the second multi-degree-of-freedom data.

[0109] Optionally, in some embodiments, the determination module includes:

[0110] A first acquisition submodule is used to acquire imaging range information of each image acquisition device loaded in the positioning reference object;

[0111] The third determination submodule is used to determine an image acquisition device that can capture an image corresponding to the target to be located based on the height difference and the horizontal distance constraint between the target to be located and the positioning reference object.

[0112] Optionally, in some embodiments, the positioning module includes:

[0113] The second acquisition submodule is used to acquire the target image of the target to be located acquired by the image acquisition device;

[0114] The recognition submodule is used to recognize the target to be located in the target image and obtain the position information of the target to be located in the target image;

[0115] A calculation submodule, used to calculate the horizontal distance and rotation angle information between the target to be located and the positioning reference object based on the position information;

[0116] The positioning submodule is used to determine the relative position relationship between the target to be positioned and the positioning reference object based on the horizontal distance, rotation angle information and height difference.

[0117] Optionally, in some embodiments, the target positioning device provided by the present application further includes:

[0118] A third acquisition submodule is used to acquire multi-degree-of-freedom data of a positioning reference object;

[0119] The fourth determination submodule is used to determine the target multi-degree-of-freedom data of the target to be positioned according to the multi-degree-of-freedom data and the desired positional relationship.

[0120] It should be noted that the target positioning device 200 provided in the embodiment of the present application is different from the target positioning device 200 in the above embodiment. Figure 1 The target positioning method shown belongs to the same concept, and its specific implementation process is detailed in the above related embodiments, which will not be repeated here.

[0121] According to the above description, the target positioning device provided by the present application obtains the inertial measurement data and air pressure data of the target to be positioned and the positioning reference object through the acquisition module 210; the calculation module 220 calculates the height difference between the target to be positioned and the positioning reference object based on the inertial measurement data and the air pressure data; the determination module 230 determines the image acquisition device that can capture the image of the target to be positioned based on the height difference, and the image acquisition device is loaded in the positioning reference object; the positioning module 240 obtains the target image of the target to be positioned acquired by the image acquisition device, and locates the target to be positioned based on the target image and the height difference. In this way, the present application can greatly improve the positioning accuracy of the object to be positioned by using inertial measurement data, air pressure data and optical tracking data acquired by the image to perform multi-source perception positioning on the object to be positioned.

[0122] The present application also provides a storage medium on which a computer program is stored. When the computer program stored therein is executed on a processor of an electronic device provided in the present application, the processor of the electronic device executes any of the steps in the above target positioning method suitable for an electronic device. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0123] This application also provides a terminal, please refer to Figure 5 , the terminal 300 includes a processor 310 and a memory 320 .

[0124] The processor 310 in the embodiment of the present application may be a general-purpose processor, such as an ARM architecture processor. The memory 320 stores a computer program, which may be a high-speed random access memory, or a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 320 may also include a memory controller to provide the processor 301 with access to the memory 320. The processor 310 executes any of the above target positioning methods by executing the computer program in the memory 320, such as:

[0125] Acquire inertial measurement data and air pressure data of the target to be located and the positioning reference object; calculate the height difference between the target to be located and the positioning reference object based on the inertial measurement data and the air pressure data; determine an image acquisition device capable of capturing an image of the target to be located based on the height difference, the image acquisition device being loaded in the positioning reference object; acquire a target image of the target to be located acquired by the image acquisition device, and locate the target to be located based on the target image and the height difference.

[0126] The above is a detailed introduction to a target positioning method, device, storage medium and terminal provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A target positioning method, characterized in that: The method comprises: Acquire inertial measurement data of the target to be positioned and the positioning reference object, as well as air pressure data; Calculating a height difference between the target to be positioned and the positioning reference object according to the inertial measurement data and the air pressure data; Based on the height difference, determining an image acquisition device capable of capturing an image of the target to be located, wherein the image acquisition device is loaded in the positioning reference object; the target to be located is within a limited range of a field of view angle of the image acquisition device; Acquire a target image of the target to be located acquired by the image acquisition device, and locate the target to be located based on the target image and the height difference; The positioning of the target to be positioned based on the target image and the height difference includes: Identifying the target to be located in the target image, and obtaining position information of the target to be located in the target image; Based on the position information, calculate the horizontal distance and rotation angle information between the target to be positioned and the positioning reference object; Determine the positional relationship between the target to be located and the positioning reference object based on the horizontal distance, the rotation angle information and the height difference; The step of calculating the height difference between the target to be positioned and the positioning reference object according to the inertial measurement data and the air pressure data comprises: Determining a first height difference between the target to be positioned and the positioning reference object according to the inertial measurement data; Determining a second height difference between the target to be positioned and the positioning reference object according to the air pressure data; The first height difference and the second height difference are filtered and fused to obtain a height difference between the target to be located and the positioning reference object.

2. The method according to claim 1, characterized in that The method further comprises: Performing error detection on the inertial measurement data to obtain an error detection result; Correcting the first height difference based on the error detection result to obtain a corrected height difference; The filtering and fusing the first height difference and the second height difference to obtain the height difference between the target to be located and the positioning reference object includes: The corrected height difference and the second height difference are filtered and fused to obtain a height difference between the target to be located and the positioning reference object.

3. The method according to claim 2, characterized in that The performing error detection on the inertial measurement data to obtain an error detection result includes: Acquire first multi-degree-of-freedom data of the calibration object obtained by mechanical arrangement based on inertial measurement data of the calibration object; Acquire second multi-degree-of-freedom data of the calibration object based on the optical tracking result of the calibration object; An error detection result is determined based on a difference between the first multi-degree-of-freedom data and the second multi-degree-of-freedom data.

4. The method according to claim 1, characterized in that: The method of determining an image acquisition device capable of capturing the image of the target to be positioned based on the height difference, wherein the image acquisition device is loaded in the positioning reference object, comprises: Acquiring imaging range information of each image acquisition device loaded in the positioning reference object; Based on the height difference and the horizontal distance constraint between the target to be positioned and the positioning reference object, an image acquisition device capable of capturing an image corresponding to the target to be positioned is determined.

5. The method according to claim 1, characterized in that: The method further comprises: Acquiring multi-degree-of-freedom data of the positioning reference object; The target multi-degree-of-freedom data of the target to be positioned is determined according to the multi-degree-of-freedom data and the positional relationship.

6. A target positioning device, characterized in that: The device comprises: An acquisition module, used to acquire inertial measurement data of the target to be positioned and the positioning reference object, as well as air pressure data; A calculation module, used for calculating the height difference between the target to be located and the positioning reference object according to the inertial measurement data and the air pressure data; The calculation module is specifically used to determine a first height difference between the target to be located and the positioning reference object according to the inertial measurement data; determine a second height difference between the target to be located and the positioning reference object according to the air pressure data; and filter and fuse the first height difference and the second height difference to obtain a height difference between the target to be located and the positioning reference object; A determination module, configured to determine, based on the height difference, an image acquisition device capable of capturing an image of the target to be located, wherein the image acquisition device is loaded in the positioning reference object; and the target to be located is within a field of view angle limit range of the image acquisition device; A positioning module, used to obtain the target image of the target to be positioned acquired by the image acquisition device, and to position the target to be positioned based on the target image and the height difference; The positioning module is specifically used to identify the target to be positioned in the target image and obtain the position information of the target to be positioned in the target image; based on the position information, calculate the horizontal distance and rotation angle information between the target to be positioned and the positioning reference object; based on the horizontal distance, the rotation angle information and the height difference, determine the position relationship between the target to be positioned and the positioning reference object.

7. A storage medium having a computer program stored thereon, characterized in that: When the computer program is loaded by a processor of an electronic device, the steps of the target positioning method according to any one of claims 1 to 5 are executed.

8. A terminal comprising a processor and a memory, wherein the memory stores a computer program, characterized in that: The processor executes the steps in the target positioning method according to any one of claims 1 to 5 by loading the computer program.

Images