Spatial positioning method, device, medium and equipment for an interaction device

Abstract

An embodiment of the present application provides a spatial positioning method, device, medium and equipment for an interaction device. The method includes: determining the straight-line distance between the interaction device and the receiving end; obtaining a real-time image of the signal transmitting module relative to the receiving end based on the monocular camera of the receiving end, and determining the three-dimensional coordinate information of the signal transmitting module in the three-dimensional space in the real-time image according to the straight-line distance, so as to realize the real-time positioning of the interaction device in the virtual scene. By using the spatial positioning method for the interaction device provided by the embodiment of the present application, by changing the original calculation method of the three-dimensional coordinate information, the straight-line distance for assisting the calculation of the three-dimensional coordinate information is obtained in advance by using a simple algorithm, so as to reduce a large number of complex calculation processes, and only one monocular camera combined with the straight-line distance can quickly and accurately obtain the three-dimensional coordinate information of the interaction device in the three-dimensional space on the premise of reducing the product cost.

Description

Technical Field

[0001] This application relates to the field of electronic communication technologies, and in particular to the field of data transmission technologies, and particularly to a method, device, medium, and equipment for spatial positioning of an interactive device. Background Art

[0002] In order to determine the three-dimensional coordinate information of a wireless pen in three-dimensional space to achieve real-time positioning of the wireless pen in a virtual scene, usually multiple cameras are used to capture the position of the infrared lamp installed on the wireless pen in real time, and the spatial position of the infrared lamp point is calculated through the triangular relationship between the camera angles, so as to calculate the spatial position of the wireless pen. However, since this solution requires at least two sets of intersecting cameras to capture the infrared lamp in real time to calculate its position information, not only is the algorithm complex, but also it occupies a lot of system resources. Summary of the Invention

[0003] Embodiments of this application provide a method, device, medium, and equipment for spatial positioning of an interactive device. Using the method for spatial positioning of an interactive device provided by the embodiments of this application, by installing at least one set of signal transmission modules on the interactive device, the straight-line distance between the interactive device and the receiving end is calculated, and a single camera installed on the receiving end is used to obtain the real-time image of the signal transmission module relative to the receiving end, and the three-dimensional coordinate information of the signal transmission module in three-dimensional space is determined in the real-time image according to the calculated straight-line distance, so as to achieve real-time positioning of the interactive device in a virtual scene. Compared with the technical solution adopted in the prior art, this solution changes the original calculation method of three-dimensional coordinate information, and uses a simple algorithm to obtain in advance the straight-line distance for assisting in calculating the three-dimensional coordinate information, so as to reduce a large number of complex calculation processes, and only by using a single camera in combination with the straight-line distance can the three-dimensional coordinate information of the interactive device in three-dimensional space be quickly and accurately obtained on the premise of reducing the product cost.

[0004] One aspect of the embodiments of this application provides a method for spatial positioning of an interactive device for a receiving end, and the method for spatial positioning of the interactive device includes:

[0005] Receiving the transmission signals emitted by at least one set of signal transmission modules installed on the interactive device, and determining the transmission duration of the transmission signals from being emitted to being converted into electrical signals by the receiving end;

[0006] Calculating the straight-line distance between the interactive device and the receiving end according to the transmission duration and the transmission speed of the transmission signals;

[0007] Based on the monocular camera of the receiving end, obtain the real-time image of the signal transmitting module relative to the receiving end, and determine the three-dimensional coordinate information of the signal transmitting module in the three-dimensional space according to the straight-line distance, so as to realize the real-time positioning of the interaction device in the virtual scene.

[0008] In the spatial positioning method of the interaction device according to the embodiment of the present application, the signal transmitting module includes an infrared signal transmitting module and an ultrasonic signal transmitting module. The infrared signal transmitting module and the ultrasonic signal transmitting module are configured to send synchronously, and the transmission speed of the infrared signal of the infrared signal transmitting module is greater than the transmission speed of the ultrasonic signal of the ultrasonic signal transmitting module.

[0009] In the spatial positioning method of the interaction device according to the embodiment of the present application, receiving the transmission signal emitted by the signal transmitting module installed on the interaction device, and determining the transmission duration of the transmission signal from being emitted to being converted into an electrical signal by the receiving end, includes:

[0010] Respectively obtain the first life cycle information of the infrared signal from being generated to being converted into the first electrical signal by the receiving end, the second life cycle information of the ultrasonic signal from being generated to being converted into the second electrical signal by the receiving end, and the waiting duration between the receiving end detecting the first electrical signal and detecting the second electrical signal;

[0011] According to the first life cycle information, the second life cycle information and the waiting duration, calculate the second transmission duration of the second transmission signal from being emitted to being converted into the second electrical signal by the receiving end.

[0012] In the spatial positioning method of the interaction device according to the embodiment of the present application, the first life cycle information at least includes the first transmission duration of the infrared signal being received by the receiving end and the first conversion delay of the receiving end converting the infrared signal into an electrical signal, and the second life cycle information at least includes the second transmission duration of the ultrasonic signal being received by the receiving end and the second conversion delay of the receiving end converting the ultrasonic signal into an electrical signal.

[0013] In the spatial positioning method of the interaction device according to the embodiment of the present application, the calculating the second transmission duration of the second transmission signal from being emitted to being converted into the second electrical signal by the receiving end according to the first life cycle information, the second life cycle information and the waiting duration, includes:

[0014] Substitute the first transmission duration, the first conversion delay, the second conversion delay and the waiting duration into the first calculation formula for calculation, to obtain the second transmission duration of the second transmission signal from being emitted to being converted into the second electrical signal by the receiving end;

[0015] The first calculation formula is as follows:

[0016] t b = t 0 + t r1 + t r2 - t b2

[0017] Wherein, t b represents the second transmission duration, t 0 represents the first transmission duration, t r1 represents the first conversion delay, t r2 represents the second conversion delay, t b2 represents the waiting duration.

[0018] In the spatial positioning method of the interaction device according to the embodiments of the present application, the determining the three-dimensional coordinate information of the signal transmission module in the three-dimensional space according to the straight-line distance in the real-time image includes:

[0019] Determining the first distance at which the light spot generated by the infrared signal transmission module reaches the nearest end boundary of the real-time image and the second distance at which it reaches the farthest end of the real-time image in the real-time image;

[0020] Substituting the first distance, the second distance, the straight-line distance, and the shooting field angle of the receiving end into the second calculation formula for calculation to obtain the Z coordinate of the infrared signal transmission module in the three-dimensional space;

[0021] Calculating the X coordinate and the Y coordinate of the infrared signal transmission module in the three-dimensional space according to the Z coordinate and a preset third calculation formula;

[0022] The second calculation formula is as follows:

[0023]

[0024]

[0025] Wherein, Z represents the distance value in the Z-axis direction of the infrared signal transmission module in the three-dimensional space, l represents the angle formed between the connection line between the infrared signal transmission module and the receiving end and the normal line of the shooting field angle, a represents the shooting field angle, l a represents the first distance, l b represents the second distance.

[0026] In the spatial positioning method of the interaction device according to the embodiments of the present application, the calculating the X coordinate and the Y coordinate of the signal transmission module in the three-dimensional space according to the Z coordinate and a preset third calculation formula includes:

[0027] Determine the shooting resolution and shooting calibration value of the receiving end, where the shooting calibration value is used to indicate the shooting distance when the calibration object has the same spatial coordinates and true coordinates;

[0028] Substitute the shooting calibration value, the distance value in the x-axis direction and the distance value in the y-axis direction of the infrared signal emission module in the real-time image with the center point of the real-time image as the origin into the third calculation formula to calculate the X coordinate and Y coordinate of the infrared signal emission module in the three-dimensional space;

[0029] The third calculation formula is:

[0030] X = Z / n * x'

[0031] Y = Z / n * y′

[0032] Wherein, X represents the distance value in the x-axis direction of the infrared signal emission module in the three-dimensional space, Y represents the distance value in the y-axis direction of the infrared signal emission module in the three-dimensional space, x' represents the distance value in the x-axis direction of the infrared signal emission module in the real-time image with the center point of the real-time image as the origin, y' represents the distance value in the y-axis direction of the infrared signal emission module in the real-time image with the center point of the real-time image as the origin, Z represents the distance value in the z-axis direction of the infrared signal emission module in the three-dimensional space, and n represents the shooting calibration value.

[0033] Correspondingly, another aspect of the embodiments of the present application further provides a spatial positioning device for an interactive device, and the spatial positioning device for the interactive device includes:

[0034] A receiving module, configured to receive a transmission signal emitted by at least one group of signal emission modules installed on the interactive device, and determine the transmission duration of the transmission signal from being emitted to being converted into an electrical signal by the receiving end;

[0035] A calculation module, configured to calculate the straight-line distance between the interactive device and the receiving end according to the transmission duration and the transmission speed of the transmission signal;

[0036] A determination module, configured to obtain a real-time image of the signal emission module relative to the receiving end based on the monocular camera of the receiving end, and determine the three-dimensional coordinate information of the signal emission module in the three-dimensional space in the real-time image according to the straight-line distance, so as to realize real-time positioning of the interactive device in the virtual scene.

[0037] Correspondingly, another aspect of the embodiments of the present application further provides a storage medium, which stores multiple instructions, and the instructions are suitable for being loaded by a processor to execute the spatial positioning method of the interactive device as described above.

[0038] Correspondingly, another aspect of the embodiments of the present application further provides a terminal device, including a processor and a memory. The memory stores multiple instructions, and the processor loads the instructions to execute the spatial positioning method of the interactive device as described above.

[0039] The embodiments of the present application provide a spatial positioning method, device, medium and device for an interactive device. The method determines the transmission duration of the transmission signal from being emitted to being converted into an electrical signal by the receiving end by receiving at least one set of transmission signals emitted by a signal transmitting module installed on the interactive device; calculates the straight-line distance between the interactive device and the receiving end according to the transmission duration and the transmission speed of the transmission signal; obtains a real-time image of the signal transmitting module relative to the receiving end based on the monocular camera of the receiving end, and determines the three-dimensional coordinate information of the signal transmitting module in the three-dimensional space in the real-time image, so as to realize the real-time positioning of the interactive device in the virtual scene. By using the spatial positioning method of the interactive device provided by the embodiments of the present application, by installing at least one set of signal transmitting modules on the interactive device and calculating the transmission duration of the transmission signal from being emitted to being converted into an electrical signal by the receiving end, since the transmission duration and the transmission speed of the transmission signal are known, the straight-line distance between the interactive device and the receiving end can be calculated through simple mathematics, and a monocular camera installed on the receiving end is used to obtain a real-time image of the signal transmitting module relative to the receiving end, and the three-dimensional coordinate information of the signal transmitting module in the three-dimensional space is determined in the real-time image according to the calculated straight-line distance, so as to realize the real-time positioning of the interactive device in the virtual scene. Compared with the technical solutions adopted in the prior art, this solution changes the original calculation method of three-dimensional coordinate information, uses a simple algorithm to obtain the straight-line distance for assisting in calculating three-dimensional coordinate information in advance, so as to reduce a large number of complex calculation processes, and only uses a monocular camera combined with the straight-line distance to quickly and accurately obtain the three-dimensional coordinate information of the interactive device in the three-dimensional space on the premise of reducing the product cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those skilled in the art, without creative efforts, other drawings can be obtained according to these drawings.

[0041] Figure 1 It is a schematic diagram of the application scenario of the spatial positioning method of the interactive device provided by the embodiments of the present application.

[0042] Figure 2 It is a schematic flowchart of the spatial positioning method of the interactive device provided by the embodiments of the present application.

[0043] Figure 3 This is an auxiliary diagram for calculating the Z coordinate of the infrared signal transmitting module in three-dimensional space in the embodiments of the present application.

[0044] Figure 4 This is an auxiliary diagram for calculating the X and Y coordinates of the infrared signal transmitting module in three-dimensional space in the embodiments of the present application.

[0045] Figure 5 This is another auxiliary diagram for calculating the X and Y coordinates of the infrared signal transmitting module in three-dimensional space in the embodiments of the present application.

[0046] Figure 6 This is a schematic structural diagram of the spatial positioning device of the interactive device provided in the embodiments of the present application.

[0047] Figure 7 This is another schematic structural diagram of the spatial positioning device of the interactive device provided in the embodiments of the present application.

[0048] Figure 8 This is a schematic structural diagram of the terminal device provided in the embodiments of the present application. Specific embodiments

[0049] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the protection scope of the present application.

[0050] The embodiments of the present application provide a spatial positioning method for an interactive device. By installing at least one set of signal transmitting modules on the interactive device and calculating the transmission duration from the transmission of the transmission signal to its conversion into an electrical signal by the receiving end, since the transmission duration and transmission speed of the transmission signal are known, the straight-line distance between the interactive device and the receiving end can be calculated through simple mathematics. And a monocular camera installed on the receiving end is used to obtain the real-time image of the signal transmitting module relative to the receiving end, and the three-dimensional coordinate information of the signal transmitting module in three-dimensional space is determined in the real-time image according to the calculated straight-line distance to achieve the real-time positioning of the interactive device in the virtual scene. Compared with the technical solutions adopted in the prior art, this solution changes the original three-dimensional coordinate information calculation method, uses a simple algorithm to obtain in advance the straight-line distance for assisting in calculating the three-dimensional coordinate information, so as to reduce a large number of complex calculation processes, and only uses a monocular camera combined with the straight-line distance to quickly and accurately obtain the three-dimensional coordinate information of the interactive device in three-dimensional space on the premise of reducing the product cost.

[0051] The following will be described in detail respectively.

[0052] The term "and / or" appearing in this application may be an association relationship describing associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Additionally, the character " / " in this application generally indicates that the associated objects before and after are in an "or" relationship.

[0053] The terms "first", "second", etc. in the description, claims, and the above-mentioned drawings of this application are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules does not have to be limited to those steps or modules clearly listed, but may include other steps or modules not clearly listed or inherent to these processes, methods, products, or devices. The naming or numbering of steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The named or numbered process steps can be changed in the order of execution according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved. The division of modules in this application is a logical division. In actual implementation, there can be other division methods. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Additionally, the shown or discussed coupling, direct coupling, or communication connection to each other can be through some interfaces. The indirect coupling or communication connection between modules can be in an electrical or other similar form, which is not limited in this application. And the modules or sub-modules described as separate components can be physically separated or not, can be physical modules or not, or can be distributed to multiple circuit modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this application.

[0054] The application scenarios of the embodiments of this application are as Figure 1 shown, including: an interaction device 101 and a receiving end 102. Communication can be achieved between the receiving end 102 and the interaction device 101 by means of infrared signals or network connections.

[0055] Exemplarily, the interaction device 101 can be a wireless pen, a TV, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a smart voice interaction device, a smart home appliance, a vehicle-mounted terminal, etc., which is not limited here.

[0056] The receiving end 102 can be a computer, a server, etc., which is not limited herein. Optionally, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, which is not limited herein.

[0057] It should be noted that the spatial positioning method of the interactive device provided in this solution is mainly applicable to application scenarios where a combination of a VR headset, an electronic device with a camera (such as a computer), and an interactive device (such as a wireless pen) is used to enable a user to interact with a virtual interaction space. The virtual interaction space can include a virtual scene and / or virtual objects according to actual applications. When the user wears the VR headset, a virtual interaction space including a virtual scene and / or virtual objects can be displayed in the display area of the electronic device. At this time, the user can use an interactive device such as a wireless pen to perform virtual interaction with the virtual scene and / or virtual objects in the virtual interaction space. For example, the user can use the wireless pen to move to the position of the virtual object and click the control switch on the wireless pen to indicate that the virtual object is selected.

[0058] Taking the wireless pen as an example, currently, in order to determine the three-dimensional coordinate information of the wireless pen in the three-dimensional space to achieve real-time positioning of the wireless pen in the virtual scene, usually multiple cameras are used to capture the position of the infrared lamp installed on the wireless pen in real time, and the spatial position of the infrared lamp point is calculated through the triangular relationship between the camera angles, so as to calculate the spatial position of the wireless pen. However, since this solution requires at least two sets of intersecting cameras to capture the infrared lamp in real time to calculate its position information, not only is the algorithm complex, but also it occupies a lot of system resources.

[0059] To solve the above problems, at least one set of signal transmission modules is installed on the interaction device, and the transmission duration of the transmission signal from being emitted to being converted into an electrical signal by the receiving end is calculated. Since the transmission duration and transmission speed of the transmission signal are known, the straight-line distance between the interaction device and the receiving end can be calculated through simple mathematics. A monocular camera installed on the receiving end is used to obtain the real-time image of the signal transmission module relative to the receiving end, and the three-dimensional coordinate information of the signal transmission module in the three-dimensional space is determined in the real-time image according to the calculated straight-line distance, so as to realize the real-time positioning of the interaction device in the virtual scene. Compared with the technical solution adopted in the prior art, this solution changes the original calculation method of three-dimensional coordinate information, uses a simple algorithm to obtain in advance the straight-line distance for assisting in calculating the three-dimensional coordinate information, so as to reduce a large number of complex calculation processes, and only uses a monocular camera combined with the straight-line distance to quickly and accurately obtain the three-dimensional coordinate information of the interaction device in the three-dimensional space on the premise of reducing the product cost.

[0060] For ease of understanding, the specific process in the embodiment of the present application will be described below. Please refer to Figure 2 .

[0061] Figure 2 FIG. is a schematic flowchart of an embodiment of the method for spatial positioning of an interaction device provided by an embodiment of the present application.

[0062] In Figure 2 the embodiment shown, the method may include the following steps:

[0063] Step 201: Receive the transmission signal emitted by at least one set of signal transmission modules installed on the interaction device, and determine the transmission duration of the transmission signal from being emitted to being converted into an electrical signal by the receiving end.

[0064] In this embodiment, the signal transmission module specifically refers to a combination of an infrared signal transmission module and an ultrasonic signal transmission module, and the infrared signal transmission module and the ultrasonic signal transmission module are configured to send synchronously, and the infrared signal transmission speed of the infrared signal transmission module is greater than the ultrasonic signal transmission speed of the ultrasonic signal transmission module.

[0065] It should be noted that, since this solution needs to calculate the straight-line distance between the interaction device and the receiving end before calculating the three-dimensional coordinate information of the interaction device in the three-dimensional space, and calculating the straight-line distance requires determining the transmission duration of the transmitted signal from being sent to being converted into an electrical signal by the receiving end and the signal transmission speed (when the selected signal module is determined, the corresponding signal transmission speed is also known). It can be understood that although an infrared signal emission module is installed on a traditional wireless pen, and ranging can be achieved based on the mature infrared technology, that is, the straight-line distance described in this solution can be obtained. In this way, the infrared signal emission module is connected to the infrared signal receiving module installed at the receiving end through a cable, and then an electrical signal is sent through the cable so that the receiving end can determine the start time of the infrared signal emission, and the transmission duration of the infrared signal is calculated by recording the receiving time when the receiving end receives the infrared signal. Since this method cannot achieve wireless connection between the wireless pen and the receiving end due to the need to rely on a cable, it is inconvenient in practical applications.

[0066] Therefore, in this solution, the signal emission module is designed to be composed of an infrared signal emission module and an ultrasonic signal emission module, and the infrared signal emission module and the ultrasonic signal emission module are configured to send signals synchronously, and the infrared signal transmission speed of the infrared signal emission module is greater than the ultrasonic signal transmission speed of the ultrasonic signal emission module, so that the infrared signal will reach the receiving end earlier than the ultrasonic signal. The transmission of the infrared signal is used to replace the cable in the traditional solution, and the ultrasonic signal acts as the infrared signal in the traditional solution. Based on the infrared signal and the ultrasonic signal, the straight-line distance between the interaction device and the receiving end can be measured without cable connection.

[0067] Specifically, by respectively obtaining the first life cycle information of the infrared signal from being generated to being converted into the first electrical signal by the receiving end, the second life cycle information of the ultrasonic signal from being generated to being converted into the second electrical signal by the receiving end, and the waiting duration between the receiving end detecting the first electrical signal and detecting the second electrical signal; according to the first life cycle information, the second life cycle information and the waiting duration, the second transmission duration of the second transmission signal from being sent to being converted into the second electrical signal by the receiving end is calculated.

[0068] It should be explained that the first life cycle information at least includes the first transmission duration of the infrared signal received by the receiving end and the first conversion delay of the receiving end converting the infrared signal into an electrical signal, and the second life cycle information at least includes the second transmission duration of the ultrasonic signal received by the receiving end and the second conversion delay of the receiving end converting the ultrasonic signal into an electrical signal.

[0069] By substituting the first transmission duration, the first conversion delay, the second conversion delay and the waiting duration into the first calculation formula for calculation, the second transmission duration of the second transmission signal from being sent to being converted into the second electrical signal by the receiving end is obtained;

[0070] The first calculation formula is as follows:

[0071] t b = t 0 + t r1 + t r2 - t b2

[0072] Wherein, t b represents the second transmission duration, t 0 represents the first transmission duration (since the speed of light is much greater than the speed of ultrasonic waves, it can be ignored and calculated as 0 here), t r1 represents the first conversion delay, t r2 represents the second conversion delay, t b2 represents the waiting duration.

[0073] Step 202: Calculate the straight-line distance between the interaction device and the receiving end according to the transmission duration and the transmission speed of the transmission signal.

[0074] Since the second transmission duration of ultrasonic waves has been calculated, and the transmission speed of ultrasonic waves can be directly obtained, the straight-line distance between the interaction device and the receiving end can be obtained through simple mathematical calculations.

[0075] Step 203: Obtain the real-time image of the signal transmission module relative to the receiving end based on the monocular camera of the receiving end, and determine the three-dimensional coordinate information of the signal transmission module in the three-dimensional space in the real-time image according to the straight-line distance, so as to realize the real-time positioning of the interaction device in the virtual scene.

[0076] In this embodiment, as Figure 3 shown, determine the first distance that the light spot generated by the infrared signal transmission module reaches the nearest end boundary of the real-time image and the second distance that reaches the farthest end of the real-time image in the real-time image; substitute the first distance, the second distance, the straight-line distance, and the shooting field angle of the receiving end into the second calculation formula for calculation to obtain the Z coordinate of the infrared signal transmission module in the three-dimensional space; calculate the X coordinate and Y coordinate of the infrared signal transmission module in the three-dimensional space according to the Z coordinate and the preset third calculation formula;

[0077] The second calculation formula is as follows:

[0078]

[0079]

[0080]

[0081]

[0082]

[0083]

[0084] Among them, Z represents the distance value in the Z-axis direction of the infrared signal emission module in three-dimensional space, β represents the included angle formed between the line connecting the infrared signal emission module and the receiving end and the normal line of the shooting field of view angle, a represents the shooting field of view angle (which can be directly obtained), l a represents the first distance (which can be measured), l b represents the second distance (which can be measured), l c represents the vertical line length between the infrared signal emission module and the normal line.

[0085] As Figures 4 - 5 shown, by determining the shooting resolution and shooting calibration value of the receiving end, the shooting calibration value is used to indicate the shooting distance when the calibration object is at the same spatial coordinates and true coordinates; substituting the shooting calibration value, the X-axis direction distance value and Y-axis direction distance value of the infrared signal emission module in the real-time image with the center point of the real-time image as the origin into the third calculation formula to calculate the X coordinate and Y coordinate of the infrared signal emission module in three-dimensional space;

[0086] The third calculation formula is:

[0087] X = Z / n * x'

[0088] Y = Z / n * y′

[0089] Among them, X represents the X-axis direction distance value of the infrared signal emission module in three-dimensional space, Y represents the Y-axis direction distance value of the infrared signal emission module in three-dimensional space, x' represents the X-axis direction distance value of the infrared signal emission module in the real-time image with the center point of the real-time image as the origin, y' represents the Y-axis direction distance value of the infrared signal emission module in the real-time image with the center point of the real-time image as the origin, Z represents the Z-axis direction distance value of the infrared signal emission module in three-dimensional space, and n represents the shooting calibration value.

[0090] In some embodiments, the signal emission module includes two groups and is respectively installed at the head and tail ends of the interaction device. After calculating the straight-line distance between the interaction device and the receiving end, the method further includes:

[0091] Compare the straight-line distances of two groups of signal transmission modules from the receiving end, and distinguish the head and tail ends of the interaction device according to the working parameters preset by the interaction device. The working parameters are used to indicate the azimuth information corresponding to the end facing the receiving end in the working state, and the azimuth information includes any one of east, south, west, north, and middle. When the interaction device is a wireless pen, use this solution to determine the tip and the pen head of the wireless pen, and then obtain the spatial position of the positioning pen. Combining the IMU sensor data, the 6dof attitude of the wireless pen can be obtained. The 6dof attitude refers to six motion attitudes of translating along the X / Y / Z axes respectively and rotating around the X / Y / Z axes respectively in a three-dimensional space.

[0092] Compared with the method of carrying azimuth information through carrier technology, this solution is easier to implement and has stronger stability. And the working parameters only need to be received and stored in the receiving end when the interaction device is paired with the receiving end, and there is no need to repeatedly obtain them subsequently.

[0093] All the above optional technical solutions can be combined arbitrarily to form the optional embodiments of this application, which will not be elaborated one by one here.

[0094] In specific implementation, this application is not limited by the execution order of the described steps. Without conflict, some steps can also be carried out in other orders or simultaneously.

[0095] As described above, the spatial positioning method of the interaction device provided by the embodiments of the present application determines the transmission duration of the transmission signal from being emitted to being converted into an electrical signal by the receiving end by receiving at least one set of transmission signals emitted by the signal transmission modules installed on the interaction device; calculates the straight-line distance between the interaction device and the receiving end according to the transmission duration and the transmission speed of the transmission signal; obtains the real-time image of the signal transmission module relative to the receiving end based on the monocular camera of the receiving end, and determines the three-dimensional coordinate information of the signal transmission module in the three-dimensional space in the real-time image, so as to realize the real-time positioning of the interaction device in the virtual scene. By using the spatial positioning method of the interaction device provided by the embodiments of the present application, by installing at least one set of signal transmission modules on the interaction device and calculating the transmission duration of the transmission signal from being emitted to being converted into an electrical signal by the receiving end, since the transmission duration and the transmission speed of the transmission signal are known, the straight-line distance between the interaction device and the receiving end can be calculated through simple mathematics, and a monocular camera installed on the receiving end is used to obtain the real-time image of the signal transmission module relative to the receiving end, and the three-dimensional coordinate information of the signal transmission module in the three-dimensional space is determined in the real-time image according to the calculated straight-line distance, so as to realize the real-time positioning of the interaction device in the virtual scene. Compared with the technical solution adopted in the prior art, this solution changes the original calculation method of the three-dimensional coordinate information, and uses a simple algorithm to obtain in advance the straight-line distance for assisting in calculating the three-dimensional coordinate information, so as to reduce a large number of complex calculation processes, and only one monocular camera combined with the straight-line distance can quickly and accurately obtain the three-dimensional coordinate information of the interaction device in the three-dimensional space on the premise of reducing the product cost.

[0096] The embodiments of the present application further provide a spatial positioning device for an interaction device, and the spatial positioning device for the interaction device can be integrated in a terminal device.

[0097] Please refer to Figure 6 , Figure 6 which is a schematic structural diagram of the spatial positioning device for the interaction device provided by the embodiments of the present application. The spatial positioning device 30 for the interaction device may include:

[0098] A receiving module 31, configured to receive at least one set of transmission signals emitted by the signal transmission modules installed on the interaction device, and determine the transmission duration of the transmission signal from being emitted to being converted into an electrical signal by the receiving end;

[0099] A calculation module 32, configured to calculate the straight-line distance between the interaction device and the receiving end according to the transmission duration and the transmission speed of the transmission signal;

[0100] A determination module 33, configured to obtain, based on a monocular camera of the receiving end, a real-time image of the signal transmitting module relative to the receiving end, and determine three-dimensional coordinate information of the signal transmitting module in a three-dimensional space in the real-time image according to the linear distance, so as to implement real-time positioning of the interaction device in a virtual scene.

[0101] In some embodiments, the signal transmitting module includes an infrared signal transmitting module and an ultrasonic signal transmitting module. The infrared signal transmitting module and the ultrasonic signal transmitting module are configured to send signals synchronously, and the transmission speed of the infrared signal of the infrared signal transmitting module is greater than the transmission speed of the ultrasonic signal of the ultrasonic signal transmitting module.

[0102] In some embodiments, the receiving module 31 is configured to respectively obtain first life cycle information of the infrared signal from generation to conversion into a first electrical signal by the receiving end, second life cycle information of the ultrasonic signal from generation to conversion into a second electrical signal by the receiving end, and a waiting duration between the receiving end detecting the first electrical signal and detecting the second electrical signal; calculate a second transmission duration of the second transmission signal from being sent to being converted into the second electrical signal by the receiving end according to the first life cycle information, the second life cycle information, and the waiting duration.

[0103] In some embodiments, the first life cycle information at least includes a first transmission duration of the infrared signal received by the receiving end and a first conversion delay of the receiving end converting the infrared signal into an electrical signal, and the second life cycle information at least includes a second transmission duration of the ultrasonic signal received by the receiving end and a second conversion delay of the receiving end converting the ultrasonic signal into an electrical signal.

[0104] In some embodiments, the calculation module 32 is configured to substitute the first transmission duration, the first conversion delay, the second conversion delay, and the waiting duration into a first calculation formula for calculation, to obtain a second transmission duration of the second transmission signal from being sent to being converted into the second electrical signal by the receiving end;

[0105] The first calculation formula is:

[0106] t b =t 0 +t r1 +t r2 -t b2

[0107] Wherein, t b represents the second transmission duration, t 0 represents the first transmission duration, t r1 represents the first conversion delay, t r2 represents the second conversion delay, tb2 Indicates the waiting duration.

[0108] In some embodiments, the determining module 33 is configured to determine, in the real-time image, a first distance at which the light spot generated by the infrared signal transmitting module reaches the nearest end boundary of the real-time image and a second distance at which the light spot reaches the farthest end of the real-time image;

[0109] Substitute the first distance, the second distance, the straight-line distance, and the shooting field angle of the receiving end into a second calculation formula for calculation to obtain the Z coordinate of the infrared signal transmitting module in three-dimensional space;

[0110] Calculate the X coordinate and Y coordinate of the infrared signal transmitting module in three-dimensional space according to the Z coordinate and a preset third calculation formula;

[0111] The second calculation formula is:

[0112]

[0113]

[0114] where Z represents the distance value in the Z-axis direction of the infrared signal transmitting module in three-dimensional space, l represents the angle formed between the line connecting the infrared signal transmitting module and the receiving end and the normal line of the shooting field angle, a represents the shooting field angle, l a represents the first distance, l b represents the second distance.

[0115] In some embodiments, the determining module 33 is configured to determine the shooting resolution (e.g., 480*640) and shooting calibration value (e.g., 100) of the receiving end, and the shooting calibration value is used to indicate the shooting distance when the spatial coordinates and real coordinates of the calibration object are consistent;

[0116] Substitute the shooting calibration value, the x-axis direction distance value and y-axis direction distance value of the infrared signal transmitting module in the real-time image with the center point of the real-time image as the origin into a third calculation formula for calculation to obtain the X coordinate and Y coordinate of the infrared signal transmitting module in three-dimensional space;

[0117] The third calculation formula is:

[0118] X = Z / n * x'

[0119] Y = Z / n * y′

[0120] Wherein, X represents the distance value in the X-axis direction of the infrared signal transmitting module in the three-dimensional space, Y represents the distance value in the Y-axis direction of the infrared signal transmitting module in the three-dimensional space, x' represents the distance value in the X-axis direction of the infrared signal transmitting module in the real-time image with the center point of the real-time image as the origin, y' represents the distance value in the Y-axis direction of the infrared signal transmitting module in the real-time image with the center point of the real-time image as the origin, Z represents the distance value in the Z-axis direction of the infrared signal transmitting module in the three-dimensional space, and n represents the shooting calibration value.

[0121] In specific implementation, each of the above modules can be implemented as an independent entity, or can be arbitrarily combined and implemented as the same or several entities.

[0122] As can be seen from the above, the spatial positioning device 30 of the interaction device provided in the embodiment of the present application, wherein the receiving module 31 is used to receive the transmission signal emitted by at least one set of signal transmitting modules installed on the interaction device, and determine the transmission duration from the emission of the transmission signal to its conversion into an electrical signal by the receiving end; the calculation module 32 is used to calculate the straight-line distance between the interaction device and the receiving end according to the transmission duration and the transmission speed of the transmission signal; the determination module 33 is used to obtain the real-time image of the signal transmitting module relative to the receiving end based on the monocular camera of the receiving end, and determine the three-dimensional coordinate information of the signal transmitting module in the three-dimensional space in the real-time image according to the straight-line distance, so as to realize the real-time positioning of the interaction device in the virtual scene.

[0123] Please refer to Figure 7 , Figure 7 FIG. is another structural schematic diagram of the spatial positioning device of the interaction device provided in the embodiment of the present application. The spatial positioning device 30 of the interaction device includes a memory 120, one or more processors 180, and one or more application programs, wherein the one or more application programs are stored in the memory 120 and are configured to be executed by the processor 180; the processor 180 may include a receiving module 31, a calculation module 32, and a determination module 33. For example, the structures and connection relationships of the above components can be as follows:

[0124] The memory 120 can be used to store application programs and data. The application programs stored in the memory 120 contain executable codes. The application programs can form various functional modules. The processor 180 executes various functional applications and data processing by running the application programs stored in the memory 120. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices. Correspondingly, the memory 120 may further include a memory controller to provide the processor 180 with access to the memory 120.

[0125] The processor 180 is the control center of the device, connecting various parts of the entire terminal using various interfaces and circuits. By running or executing application programs stored in the memory 120 and calling data stored in the memory 120, it performs various functions of the device and processes data, thereby monitoring the device as a whole. Optionally, the processor 180 may include one or more processing cores; preferably, the processor 180 may integrate an application processor and a modem processor, where the application processor mainly processes the operating system, user interface, application programs, etc.

[0126] Specifically in this embodiment, the processor 180 will, according to the following instructions, load the executable code corresponding to the processes of one or more application programs into the memory 120, and the processor 180 will run the application programs stored in the memory 120 to implement various functions:

[0127] The receiving instruction is used to receive the transmission signals emitted by at least one set of signal emission modules installed on the interaction device, and determine the transmission duration of the transmission signals from being emitted to being converted into electrical signals by the receiving end;

[0128] The calculating instruction is used to calculate the straight-line distance between the interaction device and the receiving end according to the transmission duration and the transmission speed of the transmission signals;

[0129] The determining instruction is used to obtain the real-time image of the signal emission module relative to the receiving end based on the monocular camera of the receiving end, and determine the three-dimensional coordinate information of the signal emission module in the three-dimensional space in the real-time image according to the straight-line distance, so as to realize the real-time positioning of the interaction device in the virtual scene.

[0130] In some embodiments, the signal emission module includes an infrared signal emission module and an ultrasonic signal emission module. The infrared signal emission module and the ultrasonic signal emission module are configured to send synchronously, and the infrared signal transmission speed of the infrared signal emission module is greater than the ultrasonic signal transmission speed of the ultrasonic signal emission module.

[0131] In some embodiments, the receiving instruction is used to respectively obtain the first life cycle information of the infrared signal from being generated to being converted into the first electrical signal by the receiving end, the second life cycle information of the ultrasonic signal from being generated to being converted into the second electrical signal by the receiving end, and the waiting duration between the receiving end detecting the first electrical signal and detecting the second electrical signal; according to the first life cycle information, the second life cycle information and the waiting duration, calculate the second transmission duration of the second transmission signal from being emitted to being converted into the second electrical signal by the receiving end.

[0132] In some embodiments, the first lifecycle information at least includes a first transmission duration during which the infrared signal is received by the receiving end and a first conversion delay during which the receiving end converts the infrared signal into an electrical signal, and the second lifecycle information at least includes a second transmission duration during which the ultrasonic signal is received by the receiving end and a second conversion delay during which the receiving end converts the ultrasonic signal into an electrical signal.

[0133] In some embodiments, the calculation instruction is used to substitute the first transmission duration, the first conversion delay, the second conversion delay, and the waiting duration into a first calculation formula for calculation to obtain a second transmission duration from when the second transmission signal is sent until it is converted into a second electrical signal by the receiving end;

[0134] The first calculation formula is:

[0135] t b =t 0 +t r1 +t r2 -t b2

[0136] Wherein, t b represents the second transmission duration, t 0 represents the first transmission duration, t r1 represents the first conversion delay, t r2 represents the second conversion delay, t b2 represents the waiting duration.

[0137] In some embodiments, the determination instruction is used to determine, in the real-time image, a first distance at which the light spot generated by the infrared signal emission module reaches the nearest end boundary of the real-time image and a second distance at which it reaches the farthest end of the real-time image;

[0138] Substitute the first distance, the second distance, the straight-line distance, and the shooting field angle of the receiving end into a second calculation formula for calculation to obtain the Z coordinate of the infrared signal emission module in three-dimensional space;

[0139] Calculate the X coordinate and Y coordinate of the infrared signal emission module in three-dimensional space according to the Z coordinate and a preset third calculation formula;

[0140] The second calculation formula is:

[0141]

[0142]

[0143] Wherein, Z represents the distance value of the infrared signal transmitting module in the Z-axis direction in three-dimensional space, l represents the angle formed between the line connecting the infrared signal transmitting module and the receiving end and the normal line of the shooting field of view, a represents the shooting field of view, l a represents the first distance, l b represents the second distance.

[0144] In some embodiments, the determination instruction is used to determine the shooting resolution and shooting calibration value of the receiving end, and the shooting calibration value is used to indicate the shooting distance when the spatial coordinates and real coordinates of the calibration object are consistent;

[0145] Substitute the shooting calibration value, the distance value in the x-axis direction and the distance value in the y-axis direction of the infrared signal transmitting module in the real-time image with the center point of the real-time image as the origin into the third calculation formula to calculate the X coordinate and Y coordinate of the infrared signal transmitting module in three-dimensional space;

[0146] The third calculation formula is:

[0147] X = Z / n * x'

[0148] Y = Z / n * y′

[0149] Wherein, X represents the distance value of the infrared signal transmitting module in the X-axis direction in three-dimensional space, Y represents the distance value of the infrared signal transmitting module in the Y-axis direction in three-dimensional space, x' represents the distance value of the infrared signal transmitting module in the X-axis direction in the real-time image with the center point of the real-time image as the origin, y' represents the distance value of the infrared signal transmitting module in the Y-axis direction in the real-time image with the center point of the real-time image as the origin, Z represents the distance value of the infrared signal transmitting module in the Z-axis direction in three-dimensional space, and n represents the shooting calibration value.

[0150] The embodiment of the present application also provides a terminal device. The terminal device can be a device such as a server, a smart phone, a computer, a tablet computer, etc.

[0151] Please refer to Figure 8 , Figure 8 shows a schematic structural diagram of the terminal device provided by the embodiment of the present application. The terminal device can be used to implement the spatial positioning method of the interaction device provided in the above embodiment. The terminal device 1200 can be a television, a smart phone, or a tablet computer.

[0152] Such as Figure 8As shown in the figure, the terminal device 1200 may include an RF (Radio Frequency) circuit 110, a memory 120 including one or more (only one is shown in the figure) computer-readable storage media, an input unit 130, a display unit 140, a sensor 150, an audio circuit 160, a transmission module 170, a processor 180 including one or more (only one is shown in the figure) processing cores, a power supply 190, and other components. Those skilled in the art can understand that Figure 8 the structure of the terminal device 1200 shown in the figure does not limit the terminal device 1200, and it may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements. Among them:

[0153] The RF circuit 110 is used to receive and send electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, and thus communicate with a communication network or other devices. The RF circuit 110 may include various existing circuit elements for performing these functions. For example, antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity module (SIM) cards, memories, and so on. The RF circuit 110 can communicate with various networks such as the Internet, enterprise intranets, wireless networks, or communicate with other devices through a wireless network.

[0154] The memory 120 can be used to store software programs and modules, such as the program instructions / modules corresponding to the spatial positioning method of the interactive device in the above embodiments. The processor 180 executes various functional applications and data processing by running the software programs and modules stored in the memory 120, and can automatically select a vibration reminder mode for data transmission according to the current scene where the terminal device is located, which can not only ensure that scenes such as meetings are not disturbed, but also ensure that users can perceive incoming calls, improving the intelligence of the terminal device. The memory 120 may include a high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 120 may further include a memory remotely set relative to the processor 180, and these remote memories can be connected to the terminal device 1200 through a network. Examples of the above networks include but are not limited to the Internet, enterprise intranets, local area networks, mobile communication networks, and their combinations.

[0155] The input unit 130 can be used to receive input numerical or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control. Specifically, the input unit 130 can include a touch-sensitive surface 131 and other input devices 132. The touch-sensitive surface 131, also known as a touch display screen or a touchpad, can collect touch operations of a user thereon or nearby (such as operations of the user using any suitable object or accessory such as a finger or a stylus on or near the touch-sensitive surface 131), and drive corresponding connection devices according to a preset program. Optionally, the touch-sensitive surface 131 can include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the touch position of the user and detects signals brought by the touch operation, and transmits the signals to the touch controller; the touch controller receives touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 180, and can receive commands sent by the processor 180 and execute them. In addition, the touch-sensitive surface 131 can be implemented in multiple types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface 131, the input unit 130 can also include other input devices 132. Specifically, the other input devices 132 can include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0156] The display unit 140 can be used to display information input by the user or information provided to the user and various graphical user interfaces of the terminal device 1200, and these graphical user interfaces can be composed of graphics, text, icons, videos, and any combination thereof. The display unit 140 can include a display panel 141. Optionally, the display panel 141 can be configured in forms such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode). Further, the touch-sensitive surface 131 can cover the display panel 141. After the touch-sensitive surface 131 detects a touch operation thereon or nearby, it transmits it to the processor 180 to determine the type of touch event, and then the processor 180 provides corresponding visual output on the display panel 141 according to the type of touch event. Although in Figure 8 the touch-sensitive surface 131 and the display panel 141 are implemented as two independent components to realize input and output functions, in some embodiments, the touch-sensitive surface 131 and the display panel 141 can be integrated to realize input and output functions.

[0157] The terminal device 1200 may also include at least one sensor 150, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. Among them, the ambient light sensor can adjust the brightness of the display panel 141 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 141 and / or the backlight when the terminal device 1200 is moved to the ear. As a kind of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity, and can be used in applications for identifying the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. As for other sensors that the terminal device 1200 may also be configured with, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be elaborated here.

[0158] The audio circuit 160, the speaker 161, and the microphone 162 can provide an audio interface between the user and the terminal device 1200. The audio circuit 160 can transmit the electrical signal converted from the received audio data to the speaker 161, and the speaker 161 converts it into a sound signal for output. On the other hand, the microphone 162 converts the collected sound signal into an electrical signal, which is received by the audio circuit 160 and then converted into audio data. After the audio data is output to the processor 180 for processing, it is sent through the RF circuit 110 to, for example, another terminal, or the audio data is output to the memory 120 for further processing. The audio circuit 160 may also include an earphone jack to provide communication between the peripheral earphone and the terminal device 1200.

[0159] The terminal device 1200 can help the user send and receive emails, browse the web, and access streaming media, etc. through the transmission module 170 (such as a Wi-Fi module), which provides the user with wireless broadband Internet access. Although Figure 8 the transmission module 170 is shown, it can be understood that it does not belong to the essential components of the terminal device 1200 and can be completely omitted within the scope of not changing the essence of the invention according to needs.

[0160] The processor 180 is the control center of the terminal device 1200, connecting various parts of the entire mobile phone through various interfaces and lines. By running or executing software programs and / or modules stored in the memory 120, and by invoking data stored in the memory 120, it executes various functions of the terminal device 1200 and processes data, thereby monitoring the mobile phone as a whole. Optionally, the processor 180 may include one or more processing cores; in some embodiments, the processor 180 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor may not be integrated into the processor 180 either.

[0161] The terminal device 1200 further includes a power supply 190 for powering each component. In some embodiments, the power supply may be logically connected to the processor 180 through a power management system, so as to realize functions such as management of discharging and power consumption management through the power management system. The power supply 190 may further include any components such as one or more DC or AC power supplies, a recharge system, a power failure detection circuit, a power converter or inverter, a power status indicator, etc.

[0162] Although not shown, the terminal device 1200 may further include a camera (such as a front camera and a rear camera), a Bluetooth module, etc., which will not be elaborated here. Specifically, in this embodiment, the display unit 140 of the terminal device 1200 is a touch screen display. The terminal device 1200 further includes a memory 120, and one or more programs, where one or more programs are stored in the memory 120 and are configured to be executed by one or more processors 180. The one or more programs include instructions for performing the following operations:

[0163] A receiving instruction, for receiving a transmission signal emitted by at least one set of signal transmission modules installed on the interaction device, and determining the transmission duration of the transmission signal from being emitted to being converted into an electrical signal by the receiving end;

[0164] A calculating instruction, for calculating the straight-line distance between the interaction device and the receiving end according to the transmission duration and the transmission speed of the transmission signal;

[0165] A determining instruction, for acquiring a real-time image of the signal transmission module relative to the receiving end based on the monocular camera of the receiving end, and determining the three-dimensional coordinate information of the signal transmission module in the three-dimensional space in the real-time image according to the straight-line distance, so as to realize the real-time positioning of the interaction device in the virtual scene.

[0166] In some embodiments, the signal transmitting module includes an infrared signal transmitting module and an ultrasonic signal transmitting module. The infrared signal transmitting module and the ultrasonic signal transmitting module are configured to transmit synchronously, and the transmission speed of the infrared signal of the infrared signal transmitting module is greater than the transmission speed of the ultrasonic signal of the ultrasonic signal transmitting module.

[0167] In some embodiments, the receiving instruction is used to respectively obtain the first life cycle information of the infrared signal from generation to conversion into a first electrical signal by the receiving end, the second life cycle information of the ultrasonic signal from generation to conversion into a second electrical signal by the receiving end, and the waiting duration between the receiving end detecting the first electrical signal and detecting the second electrical signal; according to the first life cycle information, the second life cycle information and the waiting duration, calculate the second transmission duration of the second transmission signal from being sent to being converted into the second electrical signal by the receiving end.

[0168] In some embodiments, the first life cycle information at least includes the first transmission duration of the infrared signal received by the receiving end and the first conversion delay of the receiving end converting the infrared signal into an electrical signal, and the second life cycle information at least includes the second transmission duration of the ultrasonic signal received by the receiving end and the second conversion delay of the receiving end converting the ultrasonic signal into an electrical signal.

[0169] In some embodiments, the calculation instruction is used to substitute the first transmission duration, the first conversion delay, the second conversion delay and the waiting duration into a first calculation formula for calculation, to obtain the second transmission duration of the second transmission signal from being sent to being converted into the second electrical signal by the receiving end;

[0170] The first calculation formula is:

[0171] t b =t 0 +t r1 +t r2 -t b2

[0172] Wherein, t b represents the second transmission duration, t 0 represents the first transmission duration, t r1 represents the first conversion delay, t r2 represents the second conversion delay, t b2 represents the waiting duration.

[0173] In some embodiments, the determination instruction is used to determine, in the real-time image, the first distance that the light spot generated by the infrared signal transmitting module reaches the nearest end boundary of the real-time image and the second distance that reaches the farthest end of the real-time image;

[0174] Substitute the first distance, the second distance, the straight-line distance, and the shooting field of view angle of the receiving end into the second calculation formula for calculation to obtain the Z coordinate of the infrared signal transmitting module in the three-dimensional space;

[0175] Calculate the X coordinate and Y coordinate of the infrared signal transmitting module in the three-dimensional space according to the Z coordinate and the preset third calculation formula;

[0176] The second calculation formula is:

[0177]

[0178]

[0179] Where Z represents the distance value in the Z-axis direction of the infrared signal transmitting module in the three-dimensional space, l represents the angle formed between the line connecting the infrared signal transmitting module and the receiving end and the normal line of the shooting field of view angle, a represents the shooting field of view angle, l a represents the first distance, l b represents the second distance.

[0180] In some embodiments, the determination instruction is used to determine the shooting resolution and shooting calibration value of the receiving end, and the shooting calibration value is used to indicate the shooting distance when the calibration object has the same spatial coordinates and real coordinates;

[0181] Substitute the shooting calibration value, the x-axis direction distance value and the y-axis direction distance value of the infrared signal transmitting module in the real-time image with the center point of the real-time image as the origin into the third calculation formula to calculate the X coordinate and Y coordinate of the infrared signal transmitting module in the three-dimensional space;

[0182] The third calculation formula is:

[0183] X = Z / n * x'

[0184] Y = Z / n * y′

[0185] Where X represents the distance value in the X-axis direction of the infrared signal transmitting module in the three-dimensional space, Y represents the distance value in the Y-axis direction of the infrared signal transmitting module in the three-dimensional space, x' represents the x-axis direction distance value of the infrared signal transmitting module in the real-time image with the center point of the real-time image as the origin, y' represents the y-axis direction distance value of the infrared signal transmitting module in the real-time image with the center point of the real-time image as the origin, Z represents the distance value in the Z-axis direction of the infrared signal transmitting module in the three-dimensional space, and n represents the shooting calibration value.

[0186] The embodiments of the present application further provide a terminal device. The terminal device may be a device such as a smart phone or a computer.

[0187] As can be seen from the above, an embodiment of the present application provides a terminal device 1200, and the terminal device 1200 performs the following steps:

[0188] Receive a transmission signal emitted by at least one set of signal transmission modules installed on the interaction device, and determine the transmission duration of the transmission signal from being emitted to being converted into an electrical signal by the receiving end;

[0189] Calculate the straight-line distance between the interaction device and the receiving end according to the transmission duration and the transmission speed of the transmission signal;

[0190] Based on the monocular camera of the receiving end, obtain a real-time image of the signal transmission module relative to the receiving end, and determine the three-dimensional coordinate information of the signal transmission module in the three-dimensional space in the real-time image, so as to realize the real-time positioning of the interaction device in the virtual scene.

[0191] An embodiment of the present application further provides a storage medium, in which a computer program is stored. When the computer program runs on a computer, the computer executes the spatial positioning method of the interaction device described in any one of the above embodiments.

[0192] It should be noted that for the spatial positioning method of the interaction device described in the present application, those of ordinary skill in the art can understand that all or part of the processes of implementing the spatial positioning method of the interaction device described in the embodiments of the present application can be completed by controlling related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, such as stored in the memory of the terminal device and executed by at least one processor in the terminal device. During the execution process, it can include the processes of the embodiments of the spatial positioning method of the interaction device. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM, Read Only Memory), a random access memory (RAM, Random Access Memory), etc.

[0193] For the spatial positioning device of the interaction device described in the embodiments of the present application, its various functional modules can be integrated in a processing chip, or each module can exist physically alone, or two or more modules can be integrated in one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium, such as a read-only memory, a magnetic disk or an optical disk, etc.

[0194] The above has introduced in detail the spatial positioning method, device, medium and equipment of the interaction device provided by the embodiments of the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A spatial positioning method for an interaction device, for a receiving end, characterized in that, comprising: Receiving transmission signals emitted by at least one group of signal transmission modules installed on the interaction device, and determining the transmission duration of the transmission signals from being emitted to being converted into electrical signals by the receiving end; Calculating the straight-line distance between the interaction device and the receiving end according to the transmission duration and the transmission speed of the transmission signals; Based on the monocular camera of the receiving end, obtaining a real-time image of the signal transmission module relative to the receiving end, and determining the three-dimensional coordinate information of the signal transmission module in three-dimensional space in the real-time image, so as to realize the real-time positioning of the interaction device in a virtual scene; The signal transmission module includes an infrared signal transmission module and an ultrasonic signal transmission module, the infrared signal transmission module and the ultrasonic signal transmission module are configured to send synchronously, and the infrared signal transmission speed of the infrared signal transmission module is greater than the ultrasonic signal transmission speed of the ultrasonic signal transmission module; The receiving transmission signals emitted by at least one group of signal transmission modules installed on the interaction device, and determining the transmission duration of the transmission signals from being emitted to being converted into electrical signals by the receiving end, includes: Respectively obtaining first life cycle information of the infrared signal from generation to being converted into a first electrical signal by the receiving end, second life cycle information of the ultrasonic signal from generation to being converted into a second electrical signal by the receiving end, and the waiting duration between the receiving end detecting the first electrical signal and detecting the second electrical signal; Calculating the second transmission duration of the second transmission signal from being emitted to being converted into a second electrical signal by the receiving end according to the first life cycle information, the second life cycle information and the waiting duration.

2. The spatial positioning method according to claim 1, characterized in that, The first life cycle information at least includes the first transmission duration of the infrared signal received by the receiving end and the first conversion delay of the receiving end converting the infrared signal into an electrical signal, and the second life cycle information at least includes the second transmission duration of the ultrasonic signal received by the receiving end and the second conversion delay of the receiving end converting the ultrasonic signal into an electrical signal.

3. The spatial positioning method according to claim 2, characterized in that, The calculating the second transmission duration of the second transmission signal from being emitted to being converted into a second electrical signal by the receiving end according to the first life cycle information, the second life cycle information and the waiting duration includes: Substituting the first transmission duration, the first conversion delay, the second conversion delay and the waiting duration into a first calculation formula for calculation, to obtain the second transmission duration of the second transmission signal from being emitted to being converted into a second electrical signal by the receiving end; The first calculation formula is: t b = t 0 + t r1 + t r2 - t b2 Among them, t b represents the second transmission duration, t 0 represents the first transmission duration, t r1 represents the first conversion delay, t r2 represents the second conversion delay, t b2 represents the waiting duration.

4. The spatial positioning method according to claim 1, characterized in that, The determining the three-dimensional coordinate information of the signal transmission module in three-dimensional space in the real-time image according to the straight-line distance includes: Determine the first distance at which the light spot generated by the infrared signal emission module reaches the nearest end boundary of the real-time image and the second distance at which it reaches the farthest end of the real-time image in the real-time image; Substitute the first distance, the second distance, the straight-line distance, and the shooting field angle of the receiving end into the second calculation formula for calculation to obtain the Z coordinate of the infrared signal emission module in three-dimensional space; Calculate the X coordinate and Y coordinate of the infrared signal emission module in three-dimensional space according to the Z coordinate and a preset third calculation formula; The second calculation formula is: Wherein, Z represents the distance value in the Z-axis direction of the infrared signal transmitting module in three-dimensional space, l represents the connecting line between the infrared signal transmitting module and the receiving end, β represents the angle formed between the connecting line between the infrared signal transmitting module and the receiving end and the normal line of the shooting field of view, a represents the shooting field of view, l a represents the first distance, l b represents the second distance.

5. The spatial positioning method according to claim 4, characterized in that The calculating the X coordinate and Y coordinate of the signal emission module in three-dimensional space according to the Z coordinate and a preset third calculation formula includes: Determine the shooting resolution and shooting calibration value of the receiving end, where the shooting calibration value is used to indicate the shooting distance when the calibration object has the same spatial coordinates and real coordinates; Substitute the shooting calibration value, the X-axis direction distance value and the Y-axis direction distance value of the infrared signal emission module in the real-time image with the center point of the real-time image as the origin into the third calculation formula for calculation to obtain the X coordinate and Y coordinate of the infrared signal emission module in three-dimensional space; The third calculation formula is: X = Z / n * x' Y = Z / n * y′ where X represents the X-axis direction distance value of the infrared signal emission module in three-dimensional space, Y represents the Y-axis direction distance value of the infrared signal emission module in three-dimensional space, x' represents the X-axis direction distance value of the infrared signal emission module in the real-time image with the center point of the real-time image as the origin, y' represents the Y-axis direction distance value of the infrared signal emission module in the real-time image with the center point of the real-time image as the origin, Z represents the Z-axis direction distance value of the infrared signal emission module in three-dimensional space, and n represents the shooting calibration value.

6. A spatial positioning device for an interactive device, for a receiving end, characterized in that The spatial positioning device of the interactive device includes: A receiving module, configured to receive the transmission signals emitted by at least one group of signal emission modules installed on the interactive device, and determine the transmission duration from when the transmission signals are emitted to when they are converted into electrical signals by the receiving end. The signal emission modules include an infrared signal emission module and an ultrasonic signal emission module. The infrared signal emission module and the ultrasonic signal emission module are configured to send signals synchronously, and the infrared signal transmission speed of the infrared signal emission module is greater than the ultrasonic signal transmission speed of the ultrasonic signal emission module; A calculation module, configured to calculate the straight-line distance between the interactive device and the receiving end according to the transmission duration and the transmission speed of the transmission signals; A determination module, configured to obtain the real-time image of the signal emission module relative to the receiving end based on the monocular camera of the receiving end, and determine the three-dimensional coordinate information of the signal emission module in three-dimensional space in the real-time image according to the straight-line distance, so as to realize the real-time positioning of the interactive device in the virtual scene; The receiving module is configured to respectively obtain first life cycle information of the infrared signal from generation to conversion into a first electrical signal by the receiving end, second life cycle information of the ultrasonic signal from generation to conversion into a second electrical signal by the receiving end, and the waiting duration between the receiving end detecting the first electrical signal and detecting the second electrical signal; and calculate a second transmission duration of the second transmission signal from being sent to being converted into the second electrical signal by the receiving end according to the first life cycle information, the second life cycle information, and the waiting duration.

7. A computer-readable storage medium, characterized in that the computer-readable storage medium stores multiple instructions, and the instructions are adapted to be loaded by a processor to execute the spatial positioning method of the interaction device according to any one of claims 1-5.

8. A terminal device, characterized in that it includes a processor and a memory, the memory stores multiple instructions, and the processor loads the instructions to execute the spatial positioning method of the interaction device according to any one of claims 1-5.

Images