A positioning method, device and system, and computer-readable storage medium
By integrating visual positioning and ultrasonic positioning technology, using the data of the camera module and ultrasonic module, the shortcomings in accuracy and computing power of virtual reality hand positioning are solved, and more efficient positioning accuracy and lower computing power requirements are achieved.
Patent Information
- Application Number
- CN202110011592.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-01-06
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-01-06
AI Technical Summary
The existing virtual reality hand positioning technology has shortcomings in terms of accuracy and computing power requirements. Ultrasonic positioning is accurate in the depth dimension but poor in the x and y dimensions, while visual positioning is accurate in the x and y dimensions but in the x and y dimensions. Poor accuracy in depth dimension.
By combining the data of the camera module and ultrasonic module, visual positioning and ultrasonic positioning technology can improve positioning accuracy, reduce the number of optical marking points, reduce computing power requirements, and improve environmental adaptability.
It improves the accuracy of hand positioning in the x and y dimensions and the accuracy of depth dimensions, reduces the computing power requirement, and improves the environmental adaptability of the positioning algorithm.
Smart Images

Figure CN114690120B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to but are not limited to the field of virtual reality technology, and in particular to a positioning method, device and system, and a computer-readable storage medium. Background Art
[0002] Virtual Reality (VR) technology is an advanced, digital human-computer interface technology. Its characteristic is that it uses computers to create an artificial virtual environment that is mainly visual, including hearing and touch. People can perceive the virtual world created by computers through multiple sensory channels such as vision, hearing, and touch, and can also interact with the virtual world in the most natural way such as movement, voice, expression, gesture and sight, thus creating an immersive experience. Virtual reality technology is a comprehensive development of multiple technologies such as computer technology, sensor technology, human-computer interaction technology, and artificial intelligence technology. It has been applied in many fields such as military, medicine, education, entertainment, manufacturing, and engineering training. It is considered to be one of the important technologies that affect people's lives now and in the future.
[0003] In addition to watching videos, existing VR devices also have interactive functions. To achieve interaction, hand positioning is required. Current hand positioning technologies include laser positioning, visible light positioning, ultrasonic positioning, electromagnetic positioning, and visual positioning. Among them, ultrasonic positioning technology and visual positioning technology are the two mainstream positioning technologies in the current VR industry. Summary of the invention
[0004] The embodiments of the present application provide a positioning method, device and system, and a computer-readable storage medium, which can improve positioning accuracy and reduce computing power requirements.
[0005] An embodiment of the present application provides a positioning method, including: obtaining first positioning data of a second device according to a camera module, the first positioning data including first two-dimensional plane data, or including first two-dimensional plane data and first depth data; obtaining second positioning data of the second device according to an ultrasonic module, the second positioning data including second depth data, or including second two-dimensional plane data and second depth data; obtaining position data of the second device according to the first positioning data and the second positioning data.
[0006] In an exemplary embodiment, the method also includes: determining the number of ultrasonic modules on the first device and the second device;
[0007] When the number of ultrasonic modules on the first device and the second device is less than a preset number threshold, the first positioning data includes first two-dimensional plane data, or includes first two-dimensional plane data and first depth data, and the second positioning data includes second depth data;
[0008] The obtaining the position data of the second device based on the first positioning data and the second positioning data includes: using the first two-dimensional plane data and the second depth data as the position data of the second device, or fusing the first depth data and the second depth data to obtain the depth data of the second device, and using the first two-dimensional plane data and the depth data of the second device as the position data of the second device.
[0009] In an exemplary embodiment, when the number of ultrasonic modules on the first device and the second device is greater than or equal to a preset number threshold, the first positioning data includes first two-dimensional plane data and first depth data, and the second positioning data includes second two-dimensional plane data and second depth data;
[0010] The obtaining the position data of the second device based on the first positioning data and the second positioning data includes: detecting the size of any one or more of the first depth data and the second depth data and a preset depth threshold; determining the position data of the second device based on the first positioning data, the second positioning data and the size of any one or more of the first depth data and the second depth data and the preset depth threshold.
[0011] In an exemplary embodiment, determining the position data of the second device according to the first positioning data, the second positioning data, and any one or more of the first depth data and the second depth data and a preset depth threshold includes:
[0012] Determine a first depth confidence value according to a difference between any one or more of the first depth data and the second depth data and a preset depth threshold;
[0013] Determine depth data of the second device, wherein depth data=first depth confidence value*first depth data+(1-first depth confidence value)*second depth data.
[0014] In an exemplary embodiment, determining a first depth confidence value according to a difference between any one or more of the first depth data and the second depth data and a preset depth threshold includes:
[0015] When any one or more of the first depth data or the second depth data is greater than or equal to a preset depth threshold, the first depth confidence value is greater than or equal to 0 and less than 0.5;
[0016] When the first depth data or the second depth data is less than a preset depth threshold, the first depth confidence value is greater than or equal to 0.5 and less than 1.
[0017] In an exemplary embodiment, determining the first depth confidence value according to the difference between any one or more of the first depth data and the second depth data and a preset depth threshold includes any one of the following:
[0018] The first depth confidence value is in a linear relationship with the first depth data or the second depth data, and the larger the first depth data or the second depth data is, the smaller the first depth confidence value is;
[0019] The first depth confidence value is in an exponential relationship with the first depth data or the second depth data, and the larger the first depth data or the second depth data is, the smaller the first depth confidence value is;
[0020] The first depth confidence value is determined according to a depth interval in which the first depth data or the second depth data is located, and the higher the depth interval is, the smaller the first depth confidence value is.
[0021] In an exemplary embodiment, determining the position data of the second device according to the first positioning data, the second positioning data, and any one or more of the first depth data and the second depth data and a preset depth threshold includes:
[0022] Determine a first two-dimensional confidence value according to a difference between any one or more of the first depth data and the second depth data and a preset depth threshold;
[0023] Determine the two-dimensional plane data (X, Y) of the second device, where (X, Y) = first two-dimensional confidence value*(x, y)+(1-first two-dimensional confidence value)*(x', y'), (x, y) is the first two-dimensional plane data, and (x', y') is the second two-dimensional plane data.
[0024] In an exemplary embodiment, determining a first two-dimensional confidence value according to a difference between any one or more of the first depth data and the second depth data and a preset depth threshold includes:
[0025] When any one or more of the first depth data or the second depth data is greater than or equal to a preset depth threshold, the first two-dimensional confidence value is greater than or equal to 0.5 and less than 1;
[0026] When the first depth data or the second depth data is less than a preset depth threshold, the first two-dimensional confidence value is greater than or equal to 0 and less than 0.5.
[0027] In an exemplary embodiment, determining the first two-dimensional confidence value according to the difference between any one or more of the first depth data and the second depth data and a preset depth threshold includes any one of the following:
[0028] The first two-dimensional confidence value is in a linear relationship with the first depth data or the second depth data, and the larger the first depth data or the second depth data is, the larger the first two-dimensional confidence value is;
[0029] The first two-dimensional confidence value is in an exponential relationship with the first depth data or the second depth data, and the larger the first depth data or the second depth data is, the larger the first two-dimensional confidence value is;
[0030] The first two-dimensional confidence value is determined according to a depth interval in which the first depth data or the second depth data is located, and the higher the depth interval is, the greater the first two-dimensional confidence value is.
[0031] An embodiment of the present application further provides a positioning device, including a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement the steps of the positioning method as described above.
[0032] An embodiment of the present application also provides a positioning system, including the positioning device as described above, and also including a first device and a second device, wherein the first device is provided with an ultrasonic module and a camera module; the second device is provided with an ultrasonic module and a visual marker point.
[0033] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-described positioning methods are implemented.
[0034] The positioning method, device and system, and computer-readable storage medium provided in the embodiments of the present application obtain the position data of the second device based on the first positioning data obtained by the camera module and the second positioning data obtained by the ultrasonic module, integrate visual positioning and ultrasonic positioning technologies, improve the positioning accuracy of the second device, reduce the number of required optical marker points, reduce computing power requirements, and improve the environmental adaptability of the positioning algorithm.
[0035] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by implementing the present application. Other advantages of the present application can be realized and obtained by the schemes described in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] The accompanying drawings are used to provide an understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.
[0037] Figure 1 A schematic diagram of a positioning method according to an embodiment of the present application;
[0038] Figure 2 A schematic diagram of the structure of a first device according to an embodiment of the present application;
[0039] Figure 3 A schematic diagram of the structure of a second device according to an embodiment of the present application;
[0040] Figure 4 A schematic diagram of the structure of a positioning device according to an embodiment of the present application;
[0041] Figure 5 A structural schematic diagram of a positioning system according to an embodiment of the present application. DETAILED DESCRIPTION
[0042] The present application describes multiple embodiments, but the description is exemplary rather than restrictive, and it is obvious to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present application. Although many possible feature combinations are shown in the drawings and discussed in the specific embodiments, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with any other feature or element in any other embodiment, or may replace any other feature or element in any other embodiment.
[0043] The present application includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The embodiments, features and elements disclosed in the present application may also be combined with any conventional features or elements to form a unique invention scheme defined by the claims. Any features or elements of any embodiment may also be combined with features or elements from other invention schemes to form another unique invention scheme defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in the present application may be implemented individually or in any appropriate combination. Therefore, except for the limitations made according to the attached claims and their equivalents, the embodiments are not subject to other restrictions. In addition, various modifications and changes may be made within the scope of protection of the attached claims.
[0044] Unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprising" and similar words mean that the elements or objects before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects.
[0045] In addition, when describing representative embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps of the specific order described. As will be understood by those of ordinary skill in the art, other sequences of steps are also possible. Therefore, the specific sequence of the steps set forth in the specification should not be interpreted as a limitation to the claims. In addition, the claims for the method and / or process should not be limited to the steps of performing them in the order written, and those skilled in the art can easily understand that these sequences can be changed and still remain within the spirit and scope of the embodiments of the present application.
[0046] Ultrasonic positioning technology and visual positioning technology are the two mainstream positioning technologies in the current VR industry.
[0047] Among them, ultrasonic positioning technology uses the spatial propagation characteristics of ultrasound to determine the specific location of the target. In one implementation scheme, an ultrasonic transmitter is placed above the target to be located. The ultrasonic transmitter transmits ultrasonic pulses to the surrounding area at a certain time interval, and the pulse signals emitted by the ultrasonic transmitter are received at three fixed positions around. Since the transmission speed of ultrasound in space is relatively slow, the specific position of the ultrasonic transmitter, that is, the position of the located target, can be inverted by comparing the time when the three ultrasonic receiving devices receive the signal. When the target moves, the movement trajectory of the target can be depicted through uninterrupted measurement. The advantage of ultrasonic positioning technology is that the calculation amount of positioning solution is small. The disadvantage of ultrasonic positioning technology is that since ultrasonic positioning is based on three-sided positioning solution, the error of ultrasonic positioning will be magnified at a distance.
[0048] The principle of visual positioning technology is spatial multi-view geometry. In one implementation, a certain number of light emitting modules are installed on the handle as visual markers. For example, the light emitting module can be an active light emitting diode (LED). The handle image is captured by a camera device (including but not limited to a digital camera, an optical camera, and an infrared scanner, etc.) installed on a head-mounted display or an external device, and the handle motion information is obtained based on the captured handle image. The disadvantage of visual positioning is that the amount of calculation is large, and it has high requirements for computing power and power consumption.
[0049] The inventors have found that the accuracy of ultrasonic positioning is higher in the depth dimension (directly observable), and is less accurate in the other two dimensions (i.e., x and y dimensions) (the error will be magnified after positioning solution), while the accuracy of visual positioning is higher in the x and y dimensions (directly observable), and is less accurate in the depth dimension (the error will be magnified after positioning solution).
[0050] The embodiments of the present application provide a positioning method, apparatus and system, and a computer-readable storage medium. The position data of the second device is obtained by obtaining the first positioning data obtained by the camera module and the second positioning data obtained by the ultrasonic module, and the visual positioning and ultrasonic positioning technologies are integrated. Compared with the method of simply using ultrasonic positioning, the integration with visual positioning can greatly improve the positioning performance of ultrasonic positioning in the x and y dimensions; compared with the method of simply using visual positioning, since ultrasonic positioning can directly measure the depth dimension, the positioning performance of visual positioning in the depth dimension is improved; and by integrating the two positioning methods, it has better environmental adaptability than a single sensing method. In addition, the positioning method of the embodiment of the present application requires a smaller number of visual mark points. Since the computing power requirement of visual positioning depends to a large extent on the required number of Mark points, after the number of Mark points is reduced, the complexity of positioning solution is greatly reduced, thereby reducing the computing power requirement.
[0051] like Figure 1 As shown, an embodiment of the present application provides a positioning method, including steps 101 to 103.
[0052] Step 101: obtaining first positioning data of a second device according to a camera module, where the first positioning data includes first two-dimensional plane data, or includes the first two-dimensional plane data and first depth data;
[0053] Step 102: obtaining second positioning data of the second device according to the ultrasonic module, where the second positioning data includes second depth data, or includes second two-dimensional plane data and second depth data;
[0054] Step 103: Obtain location data of the second device according to the first positioning data and the second positioning data.
[0055] The execution order of step 101 and step 102 may be swapped, and this application does not impose any limitation on this.
[0056] In an exemplary embodiment, Figure 2 As shown, the first device may be a head-mounted display in a virtual reality system, wherein the head-mounted display may include a mobile head-mounted display, a PC head-mounted display, an integrated head-mounted display, a split head-mounted display, and the like.
[0057] In an exemplary embodiment, Figure 3 As shown, the second device may be an interactive device in a virtual reality system, wherein the interactive device may include a handle, a glove, a mouse, a keyboard, and other hand-held or wearable devices.
[0058] In an exemplary embodiment, the positioning method further includes: determining the number of ultrasonic modules on the first device and the second device.
[0059] In this embodiment, the first device and the second device are both equipped with a plurality of ultrasonic modules ( Figure 2 In one exemplary embodiment, a plurality of ultrasonic emitting devices are installed on the second device, and a plurality of ultrasonic receiving devices are installed on the first device; in another exemplary embodiment, a plurality of ultrasonic emitting devices are installed on the first device, and a plurality of ultrasonic receiving devices are installed on the second device.
[0060] In an exemplary embodiment, 2 to 6 ultrasonic transmitting devices are installed on the first device.
[0061] When two ultrasonic transmitting devices are installed on the first device, the field of view (FOV) can be doubled in extreme cases; when three or more ultrasonic transmitting devices are installed on the first device, not only can the distance between the first device and the second device be measured, but the position data of the second device can also be fused and optimized through ultrasonic three-sided positioning solution.
[0062] In an exemplary embodiment, 4 to 6 ultrasonic receiving devices are installed on the second device.
[0063] The number of ultrasonic modules on the second device can be divided into two cases, namely, one ultrasonic module of the second device can be seen at any viewing angle, and multiple ultrasonic modules of the second device can be seen at any viewing angle.
[0064] like Figure 2 As shown, the positioning system is also equipped with a camera module, which can be installed on the front surface of the first device or on a device independent of the first device and the second device. The camera module includes but is not limited to a digital camera, an optical camera, and an infrared scanner. The camera module can be a monocular camera or a binocular camera. The second device is equipped with a plurality of infrared LED lights, or other visible light emitting devices as visual marking points ( Figure 3 not shown).
[0065] Since the camera module is affected by its own structure, the area range of the camera module to collect images is limited. For example, the left and right illumination angles of the camera module are 120 degrees. When the illumination angle is greater than 120 degrees, it exceeds the area range of the camera module that can collect images. Therefore, when the visual mark point on the second device exceeds the area range of the camera module that can collect images, the second positioning data obtained by the ultrasonic module can be directly used as the position data of the second device.
[0066] In an exemplary embodiment, the positioning method further includes: detecting whether the number of ultrasonic modules on the first device and the second device is less than a preset number threshold.
[0067] In this embodiment, the number of ultrasonic modules on the first device and the second device is less than the preset number threshold, which means that the number of ultrasonic modules on the first device and the second device can satisfy the ranging function between the first device and the second device, but cannot perform three-sided positioning solution for the second device. Exemplarily, the number of ultrasonic modules on the first device and the second device is less than 3. On the contrary, the number of ultrasonic modules on the first device and the second device is greater than or equal to the preset number threshold, which means that the number of ultrasonic modules on the first device and the second device can not only satisfy the ranging function between the first device and the second device, but also perform three-sided positioning solution for the second device. Exemplarily, the number of ultrasonic modules on at least one of the first device and the second device is greater than or equal to 3, and the number of ultrasonic modules on the other device is greater than or equal to 1.
[0068] In an exemplary embodiment, the number of ultrasonic transmitting devices on the first device may include at least three, and the number of ultrasonic receiving devices on the second device is sufficient to enable the first device to see at least one ultrasonic receiving device on the second device at any field of view. At this time, the number of ultrasonic modules on the first device and the second device simultaneously meets the functions of ultrasonic ranging and ultrasonic three-sided positioning solution, that is, the number of ultrasonic modules on the first device and the second device is greater than or equal to a preset number threshold.
[0069] In another exemplary embodiment, the number of ultrasonic receiving devices on the first device may include at least one, and the number of ultrasonic transmitting devices on the second device may include at least three. At this time, the number of ultrasonic modules on the first device and the second device also simultaneously meets the functions of ultrasonic ranging and ultrasonic three-sided positioning solution, that is, the number of ultrasonic modules on the first device and the second device is greater than or equal to a preset number threshold.
[0070] In an exemplary embodiment, when the number of ultrasonic modules on the first device and the second device is less than a preset threshold value, the first positioning data may include only the first two-dimensional plane data, or may include the first two-dimensional plane data and the first depth data, and the second positioning data may include only the second depth data. At this time, the position data of the second device is obtained according to the first positioning data and the second positioning data, including: directly using the first two-dimensional plane data and the second depth data as the position data of the second device, or, fusing the first depth data and the second depth data to obtain the depth data of the second device, and using the first two-dimensional plane data and the depth data of the second device as the position data of the second device.
[0071] When the number of ultrasonic modules installed on the first device and the second device is small, the distance between the first device and the second device can be obtained through the ultrasonic modules. At this time, when determining the position data of the second device, position fusion should be based on the first positioning data obtained by the camera module, and the depth solution in the visual positioning should be simplified according to the result of ultrasonic ranging to reduce the computing power requirements of the visual positioning.
[0072] In an exemplary embodiment, fusing the first depth data and the second depth data to obtain the depth data of the second device includes:
[0073] Determine a first depth confidence value according to a difference between any one or more of the first depth data and the second depth data and a preset depth threshold;
[0074] Determine depth data of the second device, wherein the depth data of the second device=first depth confidence value*first depth data+(1-first depth confidence value)*second depth data.
[0075] In an exemplary embodiment, when the number of ultrasonic modules on the first device and the second device is greater than or equal to a preset number threshold, the first positioning data includes first two-dimensional plane data and first depth data, and the second positioning data includes second two-dimensional plane data and second depth data;
[0076] The location data of the second device is obtained according to the first positioning data and the second positioning data, including: detecting the size of any one or more of the first depth data and the second depth data and a preset depth threshold; determining the location data of the second device according to the first positioning data, the second positioning data and the size of any one or more of the first depth data and the second depth data and the preset depth threshold.
[0077] When the number of ultrasonic modules installed on the first device and the second device is large (that is, the number of ultrasonic modules installed on the first device and the second device can realize the trilateral solution function), at this time, the original data of the first positioning data of the second device can be obtained through the camera module, and the original data of the second positioning data of the second device can be obtained through the ultrasonic module.
[0078] In an exemplary embodiment, before the first positioning data and the second positioning data are positionally fused, the positioning method further includes: when the first coordinate system of the original data for obtaining the first positioning data and the second coordinate system of the original data for obtaining the second positioning data are the same coordinate system, directly performing position fusion through the original data of the first positioning data and the original data of the second positioning data; when the first coordinate system of the original data for obtaining the first positioning data and the second coordinate system of the original data for obtaining the second positioning data are not the same coordinate system, first converting the original data in two different coordinate systems into positioning data in one coordinate system, and then performing position fusion, that is, the original data of the first positioning data obtained in the first coordinate system can be converted into the first positioning data of the second coordinate system, or the original data of the second positioning data obtained in the second coordinate system can be converted into the second positioning data of the first coordinate system, or the original data of the first positioning data obtained in the first coordinate system can be converted into the first positioning data of the third coordinate system, and the original data of the second positioning data obtained in the second coordinate system can be converted into the second positioning data of the third coordinate system, and then the first positioning data and the second positioning data can be positionally fused.
[0079] Exemplarily, the third coordinate system may be a user coordinate system of the first device.
[0080] In an exemplary embodiment, when the first positioning data and the second positioning data are positionally fused, the second positioning data can be used as the main method, and the first positioning data obtained by the camera module can be used to positionally fuse the two-dimensional plane data in the second positioning data to reduce the computing power requirements of ultrasonic positioning.
[0081] In an exemplary embodiment, determining the position data of the second device according to the first positioning data, the second positioning data, and any one or more of the first depth data and the second depth data and a preset depth threshold value includes:
[0082] Determine a first depth confidence value according to a difference between any one or more of the first depth data and the second depth data and a preset depth threshold;
[0083] Depth data Z of the second device is determined, wherein depth data Z=first depth confidence value*first depth data+(1-first depth confidence value)*second depth data.
[0084] At this time, the two-dimensional plane data (X, Y) of the second device can directly use the first two-dimensional plane data (x, y), or it can be obtained by performing confidence calculation on the first two-dimensional plane data (x, y) and the second two-dimensional plane data (x', y') according to the confidence calculation method described later.
[0085] In an exemplary embodiment, determining a first depth confidence value according to a difference between any one or more of the first depth data and the second depth data and a preset depth threshold includes:
[0086] When any one or more of the first depth data z or the second depth data z' is greater than or equal to a preset depth threshold, the first depth confidence value is greater than or equal to 0 and less than 0.5;
[0087] When the first depth data z or the second depth data z′ is less than a preset depth threshold, the first depth confidence value is greater than or equal to 0.5 and less than 1.
[0088] In this embodiment, when determining the first depth confidence value, when the second device is farther away from the first device, that is, the value of the depth data (first depth data z and / or second depth data z') is larger, the first depth data z obtained by visual positioning is less accurate at this time, then in the position fusion calculation, a smaller confidence is assigned to the first depth data z, and a larger confidence is assigned to the second depth data z'. For example, the first depth confidence value may be 0.1, that is, the confidences of the first depth data z and the second depth data z' are 10% and 90% respectively; when the second device is closer to the first device, that is, the value of the depth data (first depth data z and / or second depth data z') is smaller, the first depth data z obtained by visual positioning is more accurate at this time, then in the position fusion calculation, a larger confidence is assigned to the first depth data z, and a smaller confidence is assigned to the second depth data z'. For example, the first depth confidence value may be 0.5, that is, the confidences of the first depth data z and the second depth data z' are 50% respectively.
[0089] In some other exemplary embodiments, determining the first depth confidence value according to the difference between any one or more of the first depth data and the second depth data and a preset depth threshold includes any one of the following:
[0090] The first depth confidence value is linearly related to the first depth data z or the second depth data z', and the larger the first depth data z or the second depth data z' is, the smaller the first depth confidence value is;
[0091] The first depth confidence value is in exponential relationship with the first depth data z or the second depth data z', and the larger the first depth data z or the second depth data z' is, the smaller the first depth confidence value is;
[0092] The first depth confidence value is determined according to the depth interval in which the first depth data z or the second depth data z′ is located, and the higher the depth interval is, the smaller the first depth confidence value is.
[0093] Exemplarily, the depth interval in which the first depth data z or the second depth data z' is located includes five intervals, (0, 10 cm], (10, 30 cm], (30, 50 cm], (50, 70 cm], (70, ∞), each depth interval corresponds to a first depth confidence, and the higher the depth interval, the smaller the first depth confidence value, that is, the first depth confidence values corresponding to (0, 10 cm], (10, 30 cm], (30, 50 cm], (50, 70 cm], (70, ∞) decrease successively.
[0094] In another exemplary embodiment, determining the position data of the second device according to the first positioning data, the second positioning data, and any one or more of the first depth data and the second depth data and a preset depth threshold includes:
[0095] Determine a first two-dimensional confidence value according to a difference between any one or more of the first depth data and the second depth data and a preset depth threshold;
[0096] Determine the two-dimensional plane data (X, Y) of the second device, where (X, Y) = first two-dimensional confidence value*(x, y)+(1-first two-dimensional confidence value)*(x', y'), (x, y) is the first two-dimensional plane data, and (x', y') is the second two-dimensional plane data.
[0097] At this time, the depth data Z of the second device can directly use the second depth data z', or can be obtained by performing confidence calculation on the first depth data and the second depth data according to the aforementioned confidence calculation method.
[0098] In an exemplary embodiment, determining a first two-dimensional confidence value according to a difference between any one or more of the first depth data and the second depth data and a preset depth threshold includes:
[0099] When any one or more of the first depth data z or the second depth data z' is greater than or equal to a preset depth threshold, the first two-dimensional confidence value is greater than or equal to 0.5 and less than 1;
[0100] When the first depth data z or the second depth data z′ is less than a preset depth threshold, the first two-dimensional confidence value is greater than or equal to 0 and less than 0.5.
[0101] In this embodiment, when determining the first two-dimensional confidence value, when the distance between the second device and the first device is farther, that is, the value of the depth data (first depth data z and / or second depth data z') is larger, the second two-dimensional plane data (x', y') obtained by ultrasound positioning is less accurate. In the position fusion calculation, a smaller confidence is assigned to the second two-dimensional plane data (x', y') and a larger confidence is assigned to the first two-dimensional plane data (x, y). For example, the first two-dimensional confidence value may be 0.5, that is, the first two-dimensional plane data (x, y) and the second two-dimensional plane data (x', y') are respectively 50% confidence; when the second device is closer to the first device, that is, the value of the depth data (the first depth data z and / or the second depth data z') is smaller, the second two-dimensional plane data (x', y') obtained by ultrasonic positioning is more accurate. In the position fusion calculation, the second two-dimensional plane data (x', y') is assigned a larger confidence, and the first two-dimensional plane data (x, y) is assigned a smaller confidence. For example, the first two-dimensional confidence value can be 0.1, that is, the first two-dimensional plane data (x, y) and the second two-dimensional plane data (x', y') have confidence levels of 10% and 90%, respectively.
[0102] In some other exemplary embodiments, determining the first two-dimensional confidence value according to the difference between any one or more of the first depth data and the second depth data and a preset depth threshold includes any one of the following:
[0103] The first two-dimensional confidence value is linearly related to the first depth data z or the second depth data z', and the larger the first depth data z or the second depth data z', the larger the first two-dimensional confidence value;
[0104] The first two-dimensional confidence value is exponentially related to the first depth data z or the second depth data z', and the larger the first depth data z or the second depth data z', the larger the first two-dimensional confidence value;
[0105] The first two-dimensional confidence value is determined according to the depth interval in which the first depth data z or the second depth data z′ is located, and the higher the depth interval is, the greater the first two-dimensional confidence value is.
[0106] Exemplarily, the depth interval in which the first depth data z or the second depth data z' is located includes five intervals: (0, 10 cm], (10, 30 cm], (30, 50 cm], (50, 70 cm], (70, ∞), each depth interval corresponds to a first two-dimensional confidence value, and the higher the depth interval, the larger the first two-dimensional confidence value, that is, the first two-dimensional confidence values corresponding to the five intervals (0, 10 cm], (10, 30 cm], (30, 50 cm], (50, 70 cm], (70, ∞) increase successively.
[0107] The present application also provides a positioning device, such as Figure 4 As shown, the positioning device 4 includes: a memory 41 and a processor 42. The memory 41 stores a computer program. When the computer program in the memory 41 is executed by the processor 42, the steps of the positioning method described in any of the above embodiments are performed.
[0108] The present application also provides a positioning system. Figure 5 As shown, the positioning system 5 includes the positioning device as described above, and also includes a first device 51 and a second device 52, wherein:
[0109] The first device 51 is provided with an ultrasonic module and a camera module;
[0110] The second device 52 is provided with an ultrasonic module and visual marking points.
[0111] In an exemplary embodiment, the ultrasonic module on the first device 51 may be an ultrasonic transmitting device or an ultrasonic receiving device, and the ultrasonic module on the second device 52 may be an ultrasonic receiving device or an ultrasonic transmitting device.
[0112] In an exemplary embodiment, the camera module includes but is not limited to a digital camera, an optical camera, and an infrared scanner. The camera module can be provided on the first device 51 or on a device independent of the first device 51 and the second device 52.
[0113] In an exemplary embodiment, the visual marking point may be an infrared LED light, an infrared LED pattern, or other visible light emitting device.
[0114] In an exemplary embodiment, the positioning device may be provided on the first device 51 , or may be provided on a device independent of the first device 51 and the second device 52 .
[0115] An embodiment of the present application further provides a computer-readable storage medium, on which computer-executable commands are stored, and the computer-executable commands are used to execute the positioning method provided in any of the above embodiments.
[0116] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
Claims
1. A positioning method, characterized in that: include: Obtaining first positioning data of the second device according to the camera module, wherein the first positioning data includes first two-dimensional plane data and first depth data; Obtaining second positioning data of the second device according to the ultrasonic module, the second positioning data including second two-dimensional plane data and second depth data; wherein the number of the ultrasonic modules on the first device and the second device is greater than or equal to a preset number threshold; Obtaining the position data of the second device according to the first positioning data and the second positioning data, including: detecting the size of any one or more of the first depth data and the second depth data and a preset depth threshold; determining the position data of the second device according to the first positioning data, the second positioning data, and the size of any one or more of the first depth data and the second depth data and the preset depth threshold; the position data of the second device includes two-dimensional plane data (X, Y) of the second device and depth data of the second device; Among them, determining the position data of the second device according to the first positioning data, the second positioning data, and any one or more of the first depth data and the second depth data and the size of a preset depth threshold includes: determining a first depth confidence value according to the size of any one or more of the first depth data and the second depth data and the size of a preset depth threshold; determining the depth data of the second device, wherein the depth data = first depth confidence value * first depth data + (1-first depth confidence value) * second depth data.
2. The positioning method according to claim 1, characterized in that: The determining a first depth confidence value according to a difference between any one or more of the first depth data and the second depth data and a preset depth threshold value includes: When any one or more of the first depth data or the second depth data is greater than or equal to a preset depth threshold, the first depth confidence value is greater than or equal to 0 and less than 0.5; When the first depth data or the second depth data is less than a preset depth threshold, the first depth confidence value is greater than or equal to 0.5 and less than 1.
3. The positioning method according to claim 1, characterized in that: The determining of the first depth confidence value according to the difference between any one or more of the first depth data and the second depth data and a preset depth threshold value includes any one of the following: The first depth confidence value is in a linear relationship with the first depth data or the second depth data, and the larger the first depth data or the second depth data is, the smaller the first depth confidence value is; The first depth confidence value is in an exponential relationship with the first depth data or the second depth data, and the larger the first depth data or the second depth data is, the smaller the first depth confidence value is; The first depth confidence value is determined according to a depth interval in which the first depth data or the second depth data is located, and the higher the depth interval is, the smaller the first depth confidence value is.
4. The positioning method according to claim 1, characterized in that: The determining the position data of the second device according to the first positioning data, the second positioning data, and any one or more of the first depth data and the second depth data and a preset depth threshold includes: Determine a first two-dimensional confidence value according to a difference between any one or more of the first depth data and the second depth data and a preset depth threshold; Determine the two-dimensional plane data (X, Y) of the second device, where (X, Y) = first two-dimensional confidence value * (x, y) + (1-first two-dimensional confidence value) * (x', y'), where (x, y) is the first two-dimensional plane data and (x', y') is the second two-dimensional plane data.
5. The positioning method according to claim 4, characterized in that: The determining of a first two-dimensional confidence value according to a difference between any one or more of the first depth data and the second depth data and a preset depth threshold value includes: When any one or more of the first depth data or the second depth data is greater than or equal to a preset depth threshold, the first two-dimensional confidence value is greater than or equal to 0.5 and less than 1; When the first depth data or the second depth data is less than a preset depth threshold, the first two-dimensional confidence value is greater than or equal to 0 and less than 0.
5.
6. The positioning method according to claim 4, characterized in that: Determining the first two-dimensional confidence value according to the difference between any one or more of the first depth data and the second depth data and a preset depth threshold includes any one of the following: The first two-dimensional confidence value is in a linear relationship with the first depth data or the second depth data, and the larger the first depth data or the second depth data is, the larger the first two-dimensional confidence value is; The first two-dimensional confidence value is in an exponential relationship with the first depth data or the second depth data, and the larger the first depth data or the second depth data is, the larger the first two-dimensional confidence value is; The first two-dimensional confidence value is determined according to a depth interval in which the first depth data or the second depth data is located, and the higher the depth interval is, the greater the first two-dimensional confidence value is.
7. A positioning device, characterized in that: It comprises a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement the steps of the positioning method according to any one of claims 1 to 6.
8. A positioning system, characterized in that: The positioning device according to claim 7 further comprises a first device and a second device, wherein: The first device is provided with an ultrasonic module and a camera module; The second device is provided with an ultrasonic module and a visual marking point; The number of ultrasonic modules on the first device and the second device is greater than or equal to a preset number threshold.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the positioning method according to any one of claims 1 to 6 are implemented.
