Spatial locating method and apparatus for interactive device, medium, and device

Abstract

Embodiments of the present application provide a spatial locating method and apparatus for an interactive device, a medium, and a device. The method comprises: determining a linear distance; and acquiring a real-time image, and determining three-dimensional coordinate information of a signal emitting module in a three-dimensional space in the real-time image on the basis of the linear distance, so as to implement real-time locating of an interactive device in a virtual scenario. According to the present application, the process involving massive and complex calculation can be reduced, and three-dimensional coordinate information of an interactive device in a three-dimensional space can be quickly and accurately obtained.

Description

Spatial positioning method, device, medium and equipment for interactive device

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on August 10, 2023, with application number 2023110078124 and invention name “Spatial positioning method, device, medium and equipment for interactive device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to a method, apparatus, medium and equipment for spatial positioning of an interactive device. Background Art

[0003] To determine the 3D coordinates of a wireless pen in 3D space and achieve real-time positioning within a virtual scene, multiple cameras are typically used to capture the position of infrared lamps mounted on the pen. The spatial position of the infrared lamps is calculated using the trigonometric relationship between the camera angles, thereby determining the spatial position of the pen. However, this approach requires at least two sets of intersecting cameras to capture the infrared lamps in real time to calculate their position, resulting in a complex algorithm and high system resource consumption.

[0004] Application Contents

[0005] The embodiments of the present application provide a method, apparatus, medium and device for spatial positioning of an interactive device. Utilizing the spatial positioning method of an interactive device provided by the embodiments of the present application, by installing at least one set of signal transmitting modules on the interactive device, the straight-line distance between the interactive device and the receiving end is calculated, and a monocular camera installed on the receiving end is used to obtain a real-time image of the signal transmitting module relative to the receiving end, and the three-dimensional coordinate information of the signal transmitting module in the three-dimensional space is determined in the real-time image based on the calculated straight-line distance, so as to realize real-time positioning of the interactive device in the virtual scene. Compared with the technical solutions adopted in the prior art, this solution changes the original three-dimensional coordinate information calculation method, and uses a simple algorithm to obtain the straight-line distance used to assist in the calculation of the three-dimensional coordinate information in advance, so as to reduce a large number of complex calculation processes, and only uses a monocular camera combined with the straight-line distance to quickly and accurately obtain the three-dimensional coordinate information of the interactive device in the three-dimensional space while reducing product costs.

[0006] On one hand, an embodiment of the present application provides a spatial positioning method for an interactive device, which is used at a receiving end. The spatial positioning method for an interactive device includes:

[0007] receiving a transmission signal transmitted by at least one group of signal transmission modules installed on the interactive device, and determining a transmission time from when the transmission signal is transmitted to when the transmission signal is converted into an electrical signal by the receiving end;

[0008] Calculating a straight-line distance between the interactive device and the receiving end according to the transmission duration and the transmission speed of the transmission signal;

[0009] Based on the monocular camera of the receiving end, a real-time image of the signal transmitting module relative to the receiving end is obtained, and the three-dimensional coordinate information of the signal transmitting module in the three-dimensional space is determined in the real-time image according to the straight-line distance, so as to realize real-time positioning of the interactive device in the virtual scene.

[0010] In the spatial positioning method of an interactive device described in an embodiment of the present application, the signal transmitting module includes an infrared signal transmitting module and an ultrasonic signal transmitting module. The infrared signal transmitting module and the ultrasonic signal transmitting module are configured to transmit synchronously, and the infrared signal transmission speed of the infrared signal transmitting module is greater than the ultrasonic signal transmission speed of the ultrasonic signal transmitting module.

[0011] In the spatial positioning method of an interactive device described in an embodiment of the present application, the receiving of a transmission signal transmitted by a signal transmitting module installed on the interactive device and determining the transmission time from the transmission of the transmission signal to the conversion of the transmission signal by the receiving end into an electrical signal include:

[0012] Respectively obtain first life cycle information of the infrared signal from generation to conversion by the receiving end into a first electrical signal, second declaration period information of the ultrasonic signal from generation to conversion by the receiving end into a second electrical signal, and a waiting time between the receiving end detecting the first electrical signal and detecting the second electrical signal;

[0013] A second transmission time from when the second transmission signal is sent to when it is converted into a second electrical signal by the receiving end is calculated based on the first life cycle information, the second life cycle information and the waiting time.

[0014] In the spatial positioning method of an interactive device described in an embodiment of the present application, the first life cycle information includes at least the first transmission duration of the infrared signal received by the receiving end and the first conversion delay of the receiving end converting the infrared signal into an electrical signal, and the second declaration period information includes at least the second transmission duration of the ultrasonic signal received by the receiving end and the second conversion delay of the receiving end converting the ultrasonic signal into an electrical signal.

[0015] In the spatial positioning method of an interactive device described in an embodiment of the present application, calculating, based on the first lifecycle information, the second lifecycle information, and the waiting time, the second transmission time from when the second transmission signal is sent to when it is converted by the receiving end into a second electrical signal includes:

[0016] Substituting the first transmission duration, the first conversion delay, the second conversion delay, and the waiting duration into the first calculation formula to calculate, a second transmission duration from when the second transmission signal is sent to when it is converted into a second electrical signal by the receiving end is obtained;

[0017] The first calculation formula is: b =t0+t r1 +t r2 -t b2

[0018] Among them, t b represents the second transmission duration, t0 represents the first transmission duration, t r1 represents the first switching delay, t r2 represents the second conversion delay, t b2 Indicates the waiting time.

[0019] In the spatial positioning method of the interactive device described in the embodiment of the present application, determining the three-dimensional coordinate information of the signal transmitting module in the three-dimensional space in the real-time image based on the straight-line distance includes:

[0020] Determining in the real-time image a first distance that the light spot generated by the infrared signal transmitting module reaches the nearest boundary of the real-time image and a second distance that the light spot reaches the farthest end of the real-time image;

[0021] Substituting the first distance, the second distance, the straight-line distance, and the shooting field of view angle of the receiving end into a second calculation formula to calculate, to obtain the Z coordinate of the infrared signal transmitting module in three-dimensional space;

[0022] Calculating the X coordinate and Y coordinate of the infrared signal transmitting module in three-dimensional space according to the Z coordinate and a preset third calculation formula;

[0023] The second calculation formula is:

[0024] Wherein, Z represents the distance value of the infrared signal transmitting module in the Z-axis direction in three-dimensional space, l represents the angle formed by the line connecting the infrared signal transmitting module to the receiving end and the normal line of the shooting field of view, a represents the shooting field of view angle, l a Indicates the first distance, l b Indicates the second distance.

[0025] In the spatial positioning method of the interactive device described in the embodiment of the present application, the step of calculating the X coordinate and the Y coordinate of the signal transmitting module in the three-dimensional space based on the Z coordinate and a preset third calculation formula includes:

[0026] Determining a shooting resolution and a shooting calibration value of the receiving end, wherein the shooting calibration value is used to indicate a shooting distance of the calibration object when the spatial coordinates are consistent with the real coordinates;

[0027] Substituting the shooting calibration value, the x-axis distance value and the y-axis distance value of the infrared signal transmitting module in the real-time image with the center point of the real-time image as the origin into a third calculation formula to calculate the X coordinate and Y coordinate of the infrared signal transmitting module in three-dimensional space;

[0028] The third calculation formula is: X = Z / n*x' Y = Z / n*y'

[0029] Among them, X represents the X-axis distance value of the infrared signal transmission module in the three-dimensional space, Y represents the Y-axis distance value of the infrared signal transmission module in the three-dimensional space, x' represents the X-axis distance value of the infrared signal transmission module in the real-time image with the center point of the real-time image as the origin, y' represents the Y-axis distance value of the infrared signal transmission module in the real-time image with the center point of the real-time image as the origin, Z represents the Z-axis distance value of the infrared signal transmission module in the three-dimensional space, and n represents the shooting calibration value.

[0030] Accordingly, another aspect of the present application further provides a spatial positioning device for an interactive device, the spatial positioning device for the interactive device comprising:

[0031] a receiving module, configured to receive transmission signals transmitted by at least one group of signal transmitting modules installed on the interactive device, and determine a transmission time from when the transmission signals are transmitted to when they are converted into electrical signals by the receiving end;

[0032] a calculation module, configured to calculate a straight-line distance between the interactive device and the receiving end based on the transmission duration and the transmission speed of the transmission signal;

[0033] A determination module is used to obtain a real-time image of the signal transmitting module relative to the receiving end based on the monocular camera of the receiving end, and determine the three-dimensional coordinate information of the signal transmitting module in the three-dimensional space in the real-time image according to the straight-line distance, so as to realize real-time positioning of the interactive device in the virtual scene.

[0034] Correspondingly, another aspect of the embodiment of the present application further provides a storage medium, which stores a plurality of instructions suitable for loading by a processor to execute the spatial positioning method of the interactive device as described above.

[0035] Correspondingly, on the other hand, an embodiment of the present application further provides a terminal device, including a processor and a memory, wherein the memory stores multiple instructions, and the processor loads the instructions to execute the spatial positioning method of the interactive device as described above.

[0036] An embodiment of the present application provides a method, apparatus, medium, and device for spatial positioning of an interactive device. The method receives a transmission signal transmitted by at least one group of signal transmission modules installed on the interactive device, determines the transmission time from the transmission of the transmission signal to the conversion of the transmission signal into an electrical signal by the receiving end; calculates the straight-line distance between the interactive device and the receiving end based on the transmission time and the transmission speed of the transmission signal; obtains a real-time image of the signal transmission module relative to the receiving end based on the monocular camera of the receiving end, and determines the three-dimensional coordinate information of the signal transmission module in three-dimensional space in the real-time image based on the straight-line distance, so as to realize real-time positioning of the interactive device in a virtual scene. The spatial positioning method of an interactive device provided in an embodiment of the present application is implemented by installing at least one set of signal transmission modules on the interactive device and calculating the transmission time from the time the transmission signal is emitted to the time it is converted into an electrical signal by the receiving end. Since the transmission time and transmission speed of the transmission signal are known, the straight-line distance between the interactive device and the receiving end can be calculated through simple mathematical calculation. A monocular camera installed on the receiving end is used to obtain a real-time image of the signal transmission module relative to the receiving end. Based on the calculated straight-line distance, the three-dimensional coordinate information of the signal transmission module in three-dimensional space is determined in the real-time image to achieve real-time positioning of the interactive device in a virtual scene. Compared with the technical solutions adopted in the prior art, this solution changes the original three-dimensional coordinate information calculation method and uses a simple algorithm to obtain the straight-line distance used to assist in the calculation of the three-dimensional coordinate information in advance, thereby reducing a large number of complex calculation processes. In addition, only a monocular camera combined with the straight-line distance can quickly and accurately obtain the three-dimensional coordinate information of the interactive device in three-dimensional space while reducing product costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] To more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0038] FIG1 is a schematic diagram of an application scenario of a spatial positioning method for an interactive device provided in an embodiment of the present application.

[0039] FIG2 is a flow chart of a spatial positioning method for an interactive device provided in an embodiment of the present application.

[0040] FIG3 is an auxiliary diagram for calculating the Z coordinate of the infrared signal transmitting module in three-dimensional space according to an embodiment of the present application.

[0041] FIG4 is an auxiliary diagram for calculating the X coordinate and Y coordinate of the infrared signal transmitting module in three-dimensional space according to an embodiment of the present application.

[0042] FIG5 is another auxiliary diagram for calculating the X coordinate and the Y coordinate of the infrared signal transmitting module in three-dimensional space according to an embodiment of the present application.

[0043] FIG6 is a schematic structural diagram of a spatial positioning device of an interactive device provided in an embodiment of the present application.

[0044] FIG7 is another structural diagram of the spatial positioning device of the interactive device provided in an embodiment of the present application.

[0045] FIG8 is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0047] The embodiment of the present application provides a method for spatial positioning of an interactive device. By installing at least one set of signal transmission modules on the interactive device and calculating the transmission time from the time the transmission signal is emitted to the time it is converted into an electrical signal by the receiving end, since the transmission time and transmission speed of the transmission signal are known, the straight-line distance between the interactive device and the receiving end can be calculated through simple mathematical calculation. A monocular camera installed on the receiving end is used to obtain a real-time image of the signal transmission module relative to the receiving end. Based on the calculated straight-line distance, the three-dimensional coordinate information of the signal transmission module in three-dimensional space is determined in the real-time image to achieve real-time positioning of the interactive device in a virtual scene. Compared with the technical solutions adopted in the prior art, this solution changes the original three-dimensional coordinate information calculation method and uses a simple algorithm to obtain the straight-line distance used to assist in the calculation of the three-dimensional coordinate information in advance, thereby reducing a large number of complex calculation processes. Moreover, only a monocular camera combined with the straight-line distance can quickly and accurately obtain the three-dimensional coordinate information of the interactive device in three-dimensional space while reducing product costs.

[0048] The following are detailed descriptions of each.

[0049] The term "and / or" as used in this application can be used to describe an association relationship between related objects, indicating that three possible relationships exist. For example, A and / or B can represent the following three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this application generally indicates that the related objects are in an "or" relationship.

[0050] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to the steps or modules clearly listed, but may include other steps or modules that are not clearly listed or that are inherent to these processes, methods, products or devices. The naming or numbering of steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The named or numbered process steps can be changed in the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved. The division of modules in this application is a logical division. In actual application, there may be other division methods. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection between modules can be electrical or other similar forms, which are not limited in this application. Moreover, the modules or submodules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed into multiple circuit modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this application.

[0051] The application scenario of the embodiment of the present application is shown in Figure 1, which includes: an interactive device 101 and a receiving terminal 102. The receiving terminal 102 and the interactive device 101 can communicate with each other through infrared signals or network connection.

[0052] Exemplarily, the interactive device 101 can be a wireless pen, a television, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, an intelligent voice interactive device, a smart home appliance, and a car terminal, etc., which are not limited here.

[0053] The receiving end 102 can be a computer, a server, etc., which is not limited here. Optionally, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, which is not limited here.

[0054] It should be noted that the spatial positioning method of the interactive device provided by this solution is mainly applicable to application scenarios in which a user interacts with a virtual interactive space by means of a combination of VR glasses, an electronic device with a camera (such as a computer) and an interactive device (such as a wireless pen). The virtual interactive space can be selected to include virtual scenes and / or virtual objects according to the actual application. When the user wears VR glasses, a virtual interactive space containing virtual scenes and / or virtual objects can be displayed in the display area of ​​the electronic device. At this time, the user can use an interactive device such as a wireless pen to conduct virtual interaction with the virtual scene and / or virtual object in the virtual interactive space. For example, the user can use the wireless pen to move to the virtual object to make a position and click the control switch on the wireless pen to indicate that the virtual object is selected.

[0055] Taking a wireless pen as an example, to determine the pen's 3D coordinates in 3D space and achieve real-time positioning within a virtual scene, multiple cameras are typically used to capture the position of infrared lamps mounted on the pen. The spatial position of the infrared lamps is calculated using the trigonometric relationship between the camera angles, thereby determining the pen's spatial position. However, this approach requires at least two sets of intersecting cameras to capture the infrared lamps in real time to calculate their position, resulting in a complex algorithm and high system resource consumption.

[0056] In order to solve the above problem, at least one set of signal transmission modules is installed on the interactive device, and the transmission time from the transmission of the transmission signal to the conversion of the transmission signal into an electrical signal by the receiving end is calculated. Since the transmission time and transmission speed of the transmission signal are known, the straight-line distance between the interactive device and the receiving end can be calculated through simple mathematics. A monocular camera installed on the receiving end is used to obtain a real-time image of the signal transmission module relative to the receiving end. Based on the calculated straight-line distance, the three-dimensional coordinate information of the signal transmission module in three-dimensional space is determined in the real-time image to achieve real-time positioning of the interactive device in the virtual scene. Compared with the technical solutions adopted in the prior art, this solution changes the original three-dimensional coordinate information calculation method and uses a simple algorithm to obtain the straight-line distance used to assist in the calculation of the three-dimensional coordinate information in advance, thereby reducing a large number of complex calculation processes. In addition, only a monocular camera combined with the straight-line distance can quickly and accurately obtain the three-dimensional coordinate information of the interactive device in three-dimensional space while reducing product costs.

[0057] For ease of understanding, the specific process in the embodiment of the present application is described below, please refer to Figure 2.

[0058] FIG2 is a flow chart of an embodiment of a method for spatial positioning of an interactive device provided in an embodiment of the present application.

[0059] In the embodiment shown in FIG2 , the method may include the following steps:

[0060] Step 201: receiving a transmission signal transmitted by at least one group of signal transmission modules installed on the interactive device, and determining a transmission time from when the transmission signal is transmitted to when it is converted into an electrical signal by the receiving end.

[0061] In this embodiment, the signal transmitting module specifically refers to a combination of an infrared signal transmitting module and an ultrasonic signal transmitting module, and the infrared signal transmitting module and the ultrasonic signal transmitting module are configured to transmit synchronously, and the infrared signal transmission speed of the infrared signal transmitting module is greater than the ultrasonic signal transmission speed of the ultrasonic signal transmitting module.

[0062] It should be noted that, since this solution needs to calculate the straight-line distance between the interactive device and the receiving end before calculating the three-dimensional coordinate information of the interactive device in the three-dimensional space, and calculating the straight-line distance requires first determining the transmission time and signal transmission speed from the time the transmission signal is sent to the time it is converted into an electrical signal by the receiving end (when the selected signal module is determined, the corresponding signal transmission speed is also known). It is understandable that although the traditional wireless pen is equipped with an infrared signal transmitting module and can achieve distance measurement based on mature infrared technology, that is, obtain the straight-line distance described in this solution. In this way, the infrared signal transmitting module is connected to the infrared signal receiving module installed at the receiving end through a cable, and then sends an electrical signal through the cable so that the receiving end can determine the start time of the infrared signal transmission, and calculate the transmission time of the infrared signal by recording the receiving time of the infrared signal received by the receiving end. Since this method requires the use of a cable, it cannot achieve wireless connection between the wireless pen and the receiving end, which is inconvenient in actual application.

[0063] Therefore, this solution designs the signal transmitting module to be composed of an infrared signal transmitting module and an ultrasonic signal transmitting module, and the infrared signal transmitting module and the ultrasonic signal transmitting module are configured to transmit synchronously, and the infrared signal transmission speed of the infrared signal transmitting module is greater than the ultrasonic signal transmission speed of the ultrasonic signal transmitting module, so that the infrared signal will reach the receiving end earlier than the ultrasonic signal. The transmission of infrared signals replaces the cable work in the traditional solution, and the ultrasonic signal plays the role of the infrared signal in the traditional solution. Based on the infrared signal and the ultrasonic signal, the straight-line distance between the interactive device and the receiving end can be measured without a cable connection.

[0064] Specifically, by respectively obtaining the first life cycle information of the infrared signal from its generation to its conversion into the first electrical signal by the receiving end, the second declaration cycle information of the ultrasonic signal from its generation to its conversion into the second electrical signal by the receiving end, and the waiting time between the receiving end detecting the first electrical signal and detecting the second electrical signal; based on the first life cycle information, the second life cycle information and the waiting time, the second transmission time of the second transmission signal from its issuance to its conversion into the second electrical signal by the receiving end is calculated.

[0065] It should be explained that the first life cycle information includes at least the first transmission duration of the infrared signal received by the receiving end and the first conversion delay of the receiving end converting the infrared signal into an electrical signal, and the second life cycle information includes at least the second transmission duration of the ultrasonic signal received by the receiving end and the second conversion delay of the receiving end converting the ultrasonic signal into an electrical signal.

[0066] The second transmission time from when the second transmission signal is sent to when it is converted into the second electrical signal by the receiving end is obtained by substituting the first transmission time, the first conversion delay, the second conversion delay, and the waiting time into the first calculation formula for calculation;

[0067] The first calculation formula is: b =t0+t r1 +t r2 -t b2

[0068] Among them, t b represents the second transmission time, t0 represents the first transmission time (because the speed of light is much greater than the speed of ultrasound, it can be ignored and calculated as 0), t r1 represents the first conversion delay, t r2 represents the second conversion delay, t b2 Indicates the waiting time.

[0069] Step 202: Calculate the straight-line distance between the interactive device and the receiving end according to the transmission duration and the transmission speed of the transmission signal.

[0070] Since the second transmission duration of the ultrasonic wave has been calculated and the transmission speed of the ultrasonic wave can be directly obtained, the straight-line distance between the interactive device and the receiving end can be obtained through simple mathematical calculations.

[0071] Step 203: Acquire a real-time image of the signal transmitting module relative to the receiving end based on the monocular camera of the receiving end, and determine the three-dimensional coordinate information of the signal transmitting module in the three-dimensional space in the real-time image according to the straight-line distance to achieve real-time positioning of the interactive device in the virtual scene.

[0072] In this embodiment, as shown in FIG3 , a first distance from a light spot generated by an infrared signal transmitting module to the nearest boundary of the real-time image and a second distance from a light spot to the farthest edge of the real-time image are determined in the real-time image; the first distance, the second distance, the straight-line distance, and the field of view of the receiving end are substituted into a second calculation formula to calculate the Z coordinate of the infrared signal transmitting module in three-dimensional space; and the X coordinate and Y coordinate of the infrared signal transmitting module in three-dimensional space are calculated based on the Z coordinate and a preset third calculation formula.

[0073] The second calculation formula is:

[0074] Where Z represents the distance value of the infrared signal transmitting module in the Z-axis direction in three-dimensional space, β represents the angle formed by the line connecting the infrared signal transmitting module to the receiving end and the normal line of the shooting field of view, a represents the shooting field of view angle (which can be directly obtained), l aIndicates the first distance (measurable), l b Indicates the second distance (measurable), l c Indicates the length of the vertical line between the infrared signal transmitting module and the normal line.

[0075] As shown in Figure 4-5, by determining the receiving end's shooting resolution and shooting calibration value, the shooting calibration value is used to indicate the shooting distance when the spatial coordinates of the calibration object are consistent with the real-world coordinates. The shooting calibration value and the X-axis distance value and Y-axis distance value of the infrared signal transmitting module in the real-time image with the real-time image center point as the origin are substituted into the third calculation formula to calculate the X and Y coordinates of the infrared signal transmitting module in three-dimensional space.

[0076] The third calculation formula is: X=Z / n*x' Y=Z / n*y'

[0077] Where X represents the distance value of the infrared signal transmitting module in the X-axis direction in three-dimensional space, Y represents the distance value of the infrared signal transmitting module in the Y-axis direction in three-dimensional space, x' represents the distance value of the infrared signal transmitting module in the X-axis direction with the center point of the real-time image as the origin, y' represents the distance value of the infrared signal transmitting module in the Y-axis direction with the center point of the real-time image as the origin, Z represents the distance value of the infrared signal transmitting module in the Z-axis direction in three-dimensional space, and n represents the shooting calibration value.

[0078] In some embodiments, the signal transmission module includes two groups and is installed at the head and tail ends of the interactive device respectively. After calculating the straight-line distance between the interactive device and the receiving end, the method further includes:

[0079] The straight-line distances between the two signal transmission modules and the receiving end are compared, and the two ends of the interactive device are distinguished based on the device's preset operating parameters. The operating parameters indicate the orientation of the receiving end when facing the receiving end. This orientation information can be any of the following: east, south, west, and north. If the interactive device is a wireless pen, this solution is used to determine the pen tip and pen head, thereby determining the pen's spatial position. Combined with IMU sensor data, the pen's 6Dof pose is obtained. The 6Dof pose refers to six motion positions in three-dimensional space: translation along the X / Y / Z axes and rotation around the X / Y / Z axes.

[0080] Compared with the method of carrying position information through carrier technology, this solution is easier to implement and more stable, and the working parameters only need to be received and stored at the receiving end when the interactive device is paired with the receiving end, and do not need to be repeatedly obtained later.

[0081] All of the above optional technical solutions can be combined in any way to form optional embodiments of the present application, and will not be described in detail here.

[0082] During specific implementation, the present application is not limited by the execution order of the various steps described. If no conflict occurs, some steps can be performed in other orders or simultaneously.

[0083] As can be seen from the above, the spatial positioning method of the interactive device provided in the embodiment of the present application determines the transmission time from the transmission of the transmission signal to the conversion of the transmission signal into an electrical signal by the receiving end by receiving the transmission signal transmitted by at least one group of signal transmission modules installed on the interactive device; calculates the straight-line distance between the interactive device and the receiving end based on the transmission time and the transmission speed of the transmission signal; obtains a real-time image of the signal transmission module relative to the receiving end based on the monocular camera of the receiving end, and determines the three-dimensional coordinate information of the signal transmission module in the three-dimensional space in the real-time image based on the straight-line distance, so as to realize real-time positioning of the interactive device in the virtual scene. The spatial positioning method of an interactive device provided in an embodiment of the present application is implemented by installing at least one set of signal transmission modules on the interactive device and calculating the transmission time from the time the transmission signal is emitted to the time it is converted into an electrical signal by the receiving end. Since the transmission time and transmission speed of the transmission signal are known, the straight-line distance between the interactive device and the receiving end can be calculated through simple mathematical calculation. A monocular camera installed on the receiving end is used to obtain a real-time image of the signal transmission module relative to the receiving end. Based on the calculated straight-line distance, the three-dimensional coordinate information of the signal transmission module in three-dimensional space is determined in the real-time image to achieve real-time positioning of the interactive device in a virtual scene. Compared with the technical solutions adopted in the prior art, this solution changes the original three-dimensional coordinate information calculation method and uses a simple algorithm to obtain the straight-line distance used to assist in the calculation of the three-dimensional coordinate information in advance, thereby reducing a large number of complex calculation processes. In addition, only a monocular camera combined with the straight-line distance can quickly and accurately obtain the three-dimensional coordinate information of the interactive device in three-dimensional space while reducing product costs.

[0084] An embodiment of the present application further provides a spatial positioning device for an interactive device, which can be integrated into a terminal device.

[0085] Please refer to Figure 6, which is a schematic diagram of the structure of the spatial positioning device of the interactive device provided in an embodiment of the present application. The spatial positioning device 30 of the interactive device may include:

[0086] a receiving module 31 configured to receive transmission signals transmitted by at least one group of signal transmitting modules installed on the interactive device, and determine a transmission time from when the transmission signals are transmitted to when they are converted into electrical signals by the receiving end;

[0087] a calculation module 32, configured to calculate a straight-line distance between the interactive device and the receiving end based on the transmission duration and the transmission speed of the transmission signal;

[0088] The determination module 33 is used to obtain a real-time image of the signal transmitting module relative to the receiving end based on the monocular camera of the receiving end, and determine the three-dimensional coordinate information of the signal transmitting module in the three-dimensional space in the real-time image according to the straight-line distance, so as to realize real-time positioning of the interactive device in the virtual scene.

[0089] In some embodiments, the signal transmitting module includes an infrared signal transmitting module and an ultrasonic signal transmitting module, the infrared signal transmitting module and the ultrasonic signal transmitting module are configured to transmit synchronously, and the infrared signal transmission speed of the infrared signal transmitting module is greater than the ultrasonic signal transmission speed of the ultrasonic signal transmitting module.

[0090] In some embodiments, the receiving module 31 is used to respectively obtain the first life cycle information of the infrared signal from its generation to its conversion into a first electrical signal by the receiving end, the second declaration period information of the ultrasonic signal from its generation to its conversion into a second electrical signal by the receiving end, and the waiting time between the receiving end detecting the first electrical signal and detecting the second electrical signal; based on the first life cycle information, the second life cycle information and the waiting time, the second transmission time of the second transmission signal from its issuance to its conversion into a second electrical signal by the receiving end is calculated.

[0091] In some embodiments, the first life cycle information includes at least the first transmission duration of the infrared signal received by the receiving end and the first conversion delay of the receiving end converting the infrared signal into an electrical signal, and the second declaration period information includes at least the second transmission duration of the ultrasonic signal received by the receiving end and the second conversion delay of the receiving end converting the ultrasonic signal into an electrical signal.

[0092] In some embodiments, the calculation module 32 is configured to substitute the first transmission duration, the first conversion delay, the second conversion delay, and the waiting duration into a first calculation formula to calculate a second transmission duration from when the second transmission signal is sent to when it is converted into a second electrical signal by the receiving end;

[0093] The first calculation formula is: b =t0+t r1 +t r2 -t b2

[0094] Among them, t b represents the second transmission duration, t0 represents the first transmission duration, t r1represents the first conversion delay, t r2 represents the second conversion delay, t b2 Indicates the waiting time.

[0095] In some embodiments, the determining module 33 is configured to determine, in the real-time image, a first distance at which the light spot generated by the infrared signal transmitting module reaches the nearest boundary of the real-time image and a second distance at which the light spot reaches the farthest end of the real-time image;

[0096] Substituting the first distance, the second distance, the straight-line distance, and the shooting field of view angle of the receiving end into a second calculation formula to calculate, to obtain the Z coordinate of the infrared signal transmitting module in three-dimensional space;

[0097] Calculating the X coordinate and Y coordinate of the infrared signal transmitting module in three-dimensional space according to the Z coordinate and a preset third calculation formula;

[0098] The second calculation formula is:

[0099] Wherein, Z represents the distance value of the infrared signal transmitting module in the Z-axis direction in three-dimensional space, l represents the angle formed by the line connecting the infrared signal transmitting module to the receiving end and the normal line of the shooting field of view, a represents the shooting field of view angle, l a Indicates the first distance, l b Indicates the second distance.

[0100] In some embodiments, the determination module 33 is used to determine the shooting resolution (e.g., 480*640) and the shooting calibration value (e.g., 100) of the receiving end, where the shooting calibration value is used to indicate the shooting distance of the calibration object when the spatial coordinates are consistent with the real coordinates;

[0101] Substituting the shooting calibration value, the x-axis distance value and the y-axis distance value of the infrared signal transmitting module in the real-time image with the center point of the real-time image as the origin into a third calculation formula to calculate the X coordinate and Y coordinate of the infrared signal transmitting module in three-dimensional space;

[0102] The third calculation formula is: X = Z / n*x' Y = Z / n*y'

[0103] Among them, X represents the X-axis distance value of the infrared signal transmission module in the three-dimensional space, Y represents the Y-axis distance value of the infrared signal transmission module in the three-dimensional space, x' represents the X-axis distance value of the infrared signal transmission module in the real-time image with the center point of the real-time image as the origin, y' represents the Y-axis distance value of the infrared signal transmission module in the real-time image with the center point of the real-time image as the origin, Z represents the Z-axis distance value of the infrared signal transmission module in the three-dimensional space, and n represents the shooting calibration value.

[0104] During specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same entity or several entities.

[0105] As can be seen from the above, the spatial positioning device 30 of the interactive device provided in the embodiment of the present application, wherein the receiving module 31 is used to receive the transmission signal transmitted by at least one group of signal transmitting modules installed on the interactive device, and determine the transmission time from the transmission of the transmission signal to the conversion of the transmission signal into an electrical signal by the receiving end; the calculation module 32 is used to calculate the straight-line distance between the interactive device and the receiving end based on the transmission time and the transmission speed of the transmission signal; the determination module 33 is used to obtain a real-time image of the signal transmitting module relative to the receiving end based on the monocular camera of the receiving end, and determine the three-dimensional coordinate information of the signal transmitting module in the three-dimensional space in the real-time image according to the straight-line distance, so as to realize real-time positioning of the interactive device in the virtual scene.

[0106] Please refer to Figure 7, which is another schematic diagram of the structure of the spatial positioning device of an interactive device provided in an embodiment of the present application. The spatial positioning device 30 of the interactive device includes a memory 120, one or more processors 180, and one or more application programs, wherein the one or more application programs are stored in the memory 120 and configured to be executed by the processor 180; the processor 180 may include a receiving module 31, a calculating module 32, and a determining module 33. For example, the structure and connection relationship of the above components can be as follows:

[0107] The memory 120 can be used to store applications and data. The applications stored in the memory 120 include executable code. The applications can be composed of various functional modules. The processor 180 executes various functional applications and data processing by running the applications stored in the memory 120. In addition, the memory 120 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 120 may also include a memory controller to provide the processor 180 with access to the memory 120.

[0108] Processor 180 is the control center of the device, connecting all components of the terminal using various interfaces and circuits. It executes or runs applications stored in memory 120 and accesses data stored in memory 120 to perform various device functions and process data, thereby providing overall monitoring of the device. Optionally, processor 180 may include one or more processing cores; preferably, processor 180 may integrate an application processor and a modem processor, with the application processor primarily processing the operating system, user interface, and application programs.

[0109] Specifically, in this embodiment, the processor 180 loads the executable code corresponding to one or more application processes into the memory 120 according to the following instructions, and the processor 180 runs the application stored in the memory 120 to implement various functions:

[0110] a receiving instruction for receiving a transmission signal transmitted by at least one group of signal transmitting modules installed on the interactive device, and determining a transmission time from when the transmission signal is transmitted to when it is converted into an electrical signal by the receiving end;

[0111] a calculation instruction, configured to calculate a straight-line distance between the interactive device and the receiving end based on the transmission duration and the transmission speed of the transmission signal;

[0112] A determination instruction is used to obtain a real-time image of the signal transmitting module relative to the receiving end based on the monocular camera of the receiving end, and determine the three-dimensional coordinate information of the signal transmitting module in the three-dimensional space in the real-time image according to the straight-line distance, so as to realize real-time positioning of the interactive device in the virtual scene.

[0113] In some embodiments, the signal transmitting module includes an infrared signal transmitting module and an ultrasonic signal transmitting module, the infrared signal transmitting module and the ultrasonic signal transmitting module are configured to transmit synchronously, and the infrared signal transmission speed of the infrared signal transmitting module is greater than the ultrasonic signal transmission speed of the ultrasonic signal transmitting module.

[0114] In some embodiments, the receiving instruction is used to respectively obtain the first life cycle information of the infrared signal from its generation to its conversion into a first electrical signal by the receiving end, the second declaration period information of the ultrasonic signal from its generation to its conversion into a second electrical signal by the receiving end, and the waiting time between the receiving end detecting the first electrical signal and detecting the second electrical signal; based on the first life cycle information, the second life cycle information and the waiting time, the second transmission time of the second transmission signal from its issuance to its conversion into a second electrical signal by the receiving end is calculated.

[0115] In some embodiments, the first life cycle information includes at least the first transmission duration of the infrared signal received by the receiving end and the first conversion delay of the receiving end converting the infrared signal into an electrical signal, and the second declaration period information includes at least the second transmission duration of the ultrasonic signal received by the receiving end and the second conversion delay of the receiving end converting the ultrasonic signal into an electrical signal.

[0116] In some embodiments, the calculation instruction is used to substitute the first transmission duration, the first conversion delay, the second conversion delay, and the waiting time into a first calculation formula to calculate and obtain a second transmission duration from the time the second transmission signal is sent to the time it is converted into a second electrical signal by the receiving end;

[0117] The first calculation formula is: b =t0+t r1 +t r2 -t b2

[0118] Among them, t b represents the second transmission duration, t0 represents the first transmission duration, t r1 represents the first conversion delay, t r2 represents the second conversion delay, t b2 Indicates the waiting time.

[0119] In some embodiments, the determining instruction is used to determine, in the real-time image, a first distance that the light spot generated by the infrared signal transmitting module reaches the nearest boundary of the real-time image and a second distance that the light spot reaches the farthest end of the real-time image;

[0120] Substituting the first distance, the second distance, the straight-line distance, and the shooting field of view angle of the receiving end into a second calculation formula to calculate, to obtain the Z coordinate of the infrared signal transmitting module in three-dimensional space;

[0121] Calculating the X coordinate and Y coordinate of the infrared signal transmitting module in three-dimensional space according to the Z coordinate and a preset third calculation formula;

[0122] The second calculation formula is:

[0123] Wherein, Z represents the distance value of the infrared signal transmitting module in the Z-axis direction in three-dimensional space, l represents the angle formed by the line connecting the infrared signal transmitting module to the receiving end and the normal line of the shooting field of view, a represents the shooting field of view angle, l a Indicates the first distance, l b Indicates the second distance.

[0124] In some embodiments, the determining instruction is used to determine the shooting resolution and shooting calibration value of the receiving end, and the shooting calibration value is used to indicate the shooting distance of the calibration object when the spatial coordinates are consistent with the real coordinates;

[0125] Substituting the shooting calibration value, the x-axis distance value and the y-axis distance value of the infrared signal transmitting module in the real-time image with the center point of the real-time image as the origin into a third calculation formula to calculate the X coordinate and Y coordinate of the infrared signal transmitting module in three-dimensional space;

[0126] The third calculation formula is: X = Z / n*x' Y = Z / n*y'

[0127] Among them, X represents the X-axis distance value of the infrared signal transmission module in the three-dimensional space, Y represents the Y-axis distance value of the infrared signal transmission module in the three-dimensional space, x' represents the X-axis distance value of the infrared signal transmission module in the real-time image with the center point of the real-time image as the origin, y' represents the Y-axis distance value of the infrared signal transmission module in the real-time image with the center point of the real-time image as the origin, Z represents the Z-axis distance value of the infrared signal transmission module in the three-dimensional space, and n represents the shooting calibration value.

[0128] The present application also provides a terminal device, which may be a server, a smart phone, a computer, a tablet computer, or the like.

[0129] Please refer to Figure 8, which shows a schematic diagram of the structure of a terminal device provided in an embodiment of the present application, which can be used to implement the spatial positioning method of the interactive device provided in the above embodiment. The terminal device 1200 can be a television, a smart phone, or a tablet computer.

[0130] As shown in FIG8 , the terminal device 1200 may include components such as an RF (Radio Frequency) circuit 110, a memory 120 including one or more (only one is shown in the figure) computer-readable storage media, an input unit 130, a display unit 140, a sensor 150, an audio circuit 160, a transmission module 170, a processor 180 including one or more (only one is shown in the figure) processing cores, and a power supply 190. It will be understood by those skilled in the art that the structure of the terminal device 1200 shown in FIG8 does not limit the terminal device 1200 and may include more or fewer components than shown, or combine certain components, or arrange the components differently. Among them:

[0131] RF circuit 110 is used to receive and transmit electromagnetic waves, converting them into electrical signals, thereby enabling communication with a communications network or other devices. RF circuit 110 may include various existing circuit components for performing these functions, such as an antenna, a radio frequency transceiver, a digital signal processor, an encryption / decryption chip, a subscriber identity module (SIM) card, and memory. RF circuit 110 can communicate with various networks, such as the Internet, an intranet, or a wireless network, or with other devices via a wireless network.

[0132] The memory 120 can be used to store software programs and modules, such as the program instructions / modules corresponding to the spatial positioning method of the interactive device in the above-mentioned embodiment. The processor 180 executes various functional applications and data processing by running the software programs and modules stored in the memory 120. It can automatically select a vibration reminder mode for data transmission based on the current scene of the terminal device. This can not only ensure that scenes such as meetings are not interrupted, but also ensure that users can be aware of incoming calls, thereby enhancing the intelligence of the terminal device. The memory 120 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 120 may further include memory located remotely from the processor 180, and these remote memories can be connected to the terminal device 120 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0133] The input unit 130 can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical, or trackball signal input related to user settings and function control. Specifically, the input unit 130 may include a touch-sensitive surface 131 and other input devices 132. The touch-sensitive surface 131, also known as a touch display or touchpad, can collect user touch operations on or near it (for example, operations performed by a user using a finger, stylus, or any other suitable object or accessory on or near the touch-sensitive surface 131) and drive corresponding connected devices according to a pre-set program. Optionally, the touch-sensitive surface 131 may include a touch detection device and a touch controller. The touch detection device detects the user's touch position and detects signals generated by the touch operation, transmitting the signals to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and then sends it to the processor 180. It can also receive and execute commands from the processor 180. In addition, the touch-sensitive surface 131 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface 131, the input unit 130 can also include other input devices 132. Specifically, the other input devices 132 can include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power keys, etc.), a trackball, a mouse, a joystick, and the like.

[0134] The display unit 140 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the terminal device 1200. These graphical user interfaces can be composed of graphics, text, icons, videos, or any combination thereof. The display unit 140 may include a display panel 141. Optionally, the display panel 141 can be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), or the like. Furthermore, the touch-sensitive surface 131 can cover the display panel 141. When the touch-sensitive surface 131 detects a touch operation on or near it, it transmits the information to the processor 180 to determine the type of touch event. The processor 180 then provides a corresponding visual output on the display panel 141 based on the type of touch event. Although in Figure 8, the touch-sensitive surface 131 and the display panel 141 are implemented as two independent components to implement input and output functions, in some embodiments, the touch-sensitive surface 131 and the display panel 141 can be integrated to implement input and output functions.

[0135] The terminal device 1200 may also include at least one sensor 150, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 141 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 141 and / or the backlight when the terminal device 1200 is moved to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the terminal device 1200, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be described here.

[0136] The audio circuit 160, speaker 161, and microphone 162 provide an audio interface between the user and the terminal device 1200. The audio circuit 160 can convert received audio data into electrical signals and transmit them to the speaker 161, which then converts them into sound signals for output. Meanwhile, the microphone 162 converts collected sound signals into electrical signals, which are then received by the audio circuit 160 and converted into audio data. The audio data is then processed by the processor 180 and transmitted via the RF circuit 110 to, for example, another terminal. Alternatively, the audio data can be output to the memory 120 for further processing. The audio circuit 160 may also include an earphone jack to allow communication between an external headset and the terminal device 1200.

[0137] The terminal device 1200 can help users send and receive emails, browse web pages, and access streaming media through the transmission module 170 (e.g., a Wi-Fi module), thereby providing users with wireless broadband Internet access. Although FIG8 shows the transmission module 170, it is understood that it is not a necessary component of the terminal device 1200 and can be omitted as needed without changing the essence of the invention.

[0138] Processor 180 is the control center of terminal device 1200. It connects all components of the phone using various interfaces and circuits. By running or executing software programs and / or modules stored in memory 120 and accessing data stored in memory 120, it executes various functions of terminal device 1200 and processes data, thereby providing overall monitoring of the phone. Optionally, processor 180 may include one or more processing cores. In some embodiments, processor 180 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 180.

[0139] The terminal device 1200 also includes a power supply 190 for supplying power to various components. In some embodiments, the power supply can be logically connected to the processor 180 via a power management system, thereby enabling the power management system to manage discharge and power consumption. The power supply 190 can also include any components such as one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, and a power status indicator.

[0140] Although not shown, the terminal device 1200 may further include a camera (such as a front camera, a rear camera), a Bluetooth module, etc., which will not be described in detail here. Specifically, in this embodiment, the display unit 140 of the terminal device 1200 is a touch screen display, and the terminal device 1200 also includes a memory 120 and one or more programs, wherein one or more programs are stored in the memory 120 and are configured to be executed by one or more processors 180. The one or more programs include instructions for performing the following operations:

[0141] a receiving instruction for receiving a transmission signal transmitted by at least one group of signal transmitting modules installed on the interactive device, and determining a transmission time from when the transmission signal is transmitted to when it is converted into an electrical signal by the receiving end;

[0142] a calculation instruction, configured to calculate a straight-line distance between the interactive device and the receiving end based on the transmission duration and the transmission speed of the transmission signal;

[0143] A determination instruction is used to obtain a real-time image of the signal transmitting module relative to the receiving end based on the monocular camera of the receiving end, and determine the three-dimensional coordinate information of the signal transmitting module in the three-dimensional space in the real-time image according to the straight-line distance, so as to realize real-time positioning of the interactive device in the virtual scene.

[0144] In some embodiments, the signal transmitting module includes an infrared signal transmitting module and an ultrasonic signal transmitting module, the infrared signal transmitting module and the ultrasonic signal transmitting module are configured to transmit synchronously, and the infrared signal transmission speed of the infrared signal transmitting module is greater than the ultrasonic signal transmission speed of the ultrasonic signal transmitting module.

[0145] In some embodiments, the receiving instruction is used to respectively obtain the first life cycle information of the infrared signal from its generation to its conversion into a first electrical signal by the receiving end, the second declaration period information of the ultrasonic signal from its generation to its conversion into a second electrical signal by the receiving end, and the waiting time between the receiving end detecting the first electrical signal and detecting the second electrical signal; based on the first life cycle information, the second life cycle information and the waiting time, the second transmission time of the second transmission signal from its issuance to its conversion into a second electrical signal by the receiving end is calculated.

[0146] In some embodiments, the first life cycle information includes at least the first transmission duration of the infrared signal received by the receiving end and the first conversion delay of the receiving end converting the infrared signal into an electrical signal, and the second declaration period information includes at least the second transmission duration of the ultrasonic signal received by the receiving end and the second conversion delay of the receiving end converting the ultrasonic signal into an electrical signal.

[0147] In some embodiments, the calculation instruction is used to substitute the first transmission duration, the first conversion delay, the second conversion delay, and the waiting time into a first calculation formula to calculate and obtain a second transmission duration from the time the second transmission signal is sent to the time it is converted into a second electrical signal by the receiving end;

[0148] The first calculation formula is: b =t0+t r1 +t r2 -t b2

[0149] Among them, t b represents the second transmission duration, t0 represents the first transmission duration, t r1 represents the first conversion delay, t r2 represents the second conversion delay, t b2 Indicates the waiting time.

[0150] In some embodiments, the determining instruction is used to determine, in the real-time image, a first distance that the light spot generated by the infrared signal transmitting module reaches the nearest boundary of the real-time image and a second distance that the light spot reaches the farthest end of the real-time image;

[0151] Substituting the first distance, the second distance, the straight-line distance, and the shooting field of view angle of the receiving end into a second calculation formula to calculate, to obtain the Z coordinate of the infrared signal transmitting module in three-dimensional space;

[0152] Calculating the X coordinate and Y coordinate of the infrared signal transmitting module in three-dimensional space according to the Z coordinate and a preset third calculation formula;

[0153] The second calculation formula is:

[0154] Wherein, Z represents the distance value of the infrared signal transmitting module in the Z-axis direction in three-dimensional space, l represents the angle formed by the line connecting the infrared signal transmitting module to the receiving end and the normal line of the shooting field of view, a represents the shooting field of view angle, l a Indicates the first distance, l b Indicates the second distance.

[0155] In some embodiments, the determining instruction is used to determine the shooting resolution and shooting calibration value of the receiving end, and the shooting calibration value is used to indicate the shooting distance of the calibration object when the spatial coordinates are consistent with the real coordinates;

[0156] Substituting the shooting calibration value, the x-axis distance value and the y-axis distance value of the infrared signal transmitting module in the real-time image with the center point of the real-time image as the origin into a third calculation formula to calculate the X coordinate and Y coordinate of the infrared signal transmitting module in three-dimensional space;

[0157] The third calculation formula is: X = Z / n*x' Y = Z / n*y'

[0158] Among them, X represents the X-axis distance value of the infrared signal transmission module in the three-dimensional space, Y represents the Y-axis distance value of the infrared signal transmission module in the three-dimensional space, x' represents the X-axis distance value of the infrared signal transmission module in the real-time image with the center point of the real-time image as the origin, y' represents the Y-axis distance value of the infrared signal transmission module in the real-time image with the center point of the real-time image as the origin, Z represents the Z-axis distance value of the infrared signal transmission module in the three-dimensional space, and n represents the shooting calibration value.

[0159] The present application also provides a terminal device, which may be a smartphone, computer, or other device.

[0160] As can be seen from the above, the embodiment of the present application provides a terminal device 1200, and the terminal device 1200 performs the following steps:

[0161] receiving a transmission signal transmitted by at least one group of signal transmission modules installed on the interactive device, and determining a transmission time from when the transmission signal is transmitted to when the transmission signal is converted into an electrical signal by the receiving end;

[0162] Calculating a straight-line distance between the interactive device and the receiving end according to the transmission duration and the transmission speed of the transmission signal;

[0163] Based on the monocular camera of the receiving end, a real-time image of the signal transmitting module relative to the receiving end is obtained, and the three-dimensional coordinate information of the signal transmitting module in the three-dimensional space is determined in the real-time image according to the straight-line distance, so as to realize real-time positioning of the interactive device in the virtual scene.

[0164] An embodiment of the present application further provides a storage medium, wherein a computer program is stored in the storage medium. When the computer program is run on a computer, the computer executes the spatial positioning method of the interactive device described in any of the above embodiments.

[0165] It should be noted that, with respect to the spatial positioning method for an interactive device described in this application, ordinary testers in this field can understand that all or part of the process of implementing the spatial positioning method for an interactive device described in the embodiments of this application can be accomplished by controlling related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, such as a memory of a terminal device, and executed by at least one processor in the terminal device. During execution, the process may include the process of the embodiment of the spatial positioning method for an interactive device. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), etc.

[0166] For the spatial positioning device of the interactive device of the embodiment of the present application, its various functional modules can be integrated into a processing chip, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium, such as a read-only memory, a magnetic disk or an optical disk.

[0167] The above is a detailed introduction to the spatial positioning method, device, medium, and equipment of the interactive device provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only intended to help understand the method and core ideas of the present application. At the same time, for those skilled in the art, based on the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the contents of this specification should not be understood as limiting the present application.

Claims

1. A spatial positioning method for an interactive device, used at a receiving end, wherein: include: Receiving a transmission signal transmitted by at least one group of signal transmission modules installed on the interactive device, and determining the transmission time length from the transmission of the transmission signal to the conversion of the transmission signal into an electrical signal by the receiving end; Calculating a straight-line distance between the interactive device and the receiving end according to the transmission duration and the transmission speed of the transmission signal; Based on the monocular camera of the receiving end, a real-time image of the signal transmitting module relative to the receiving end is obtained, and the three-dimensional coordinate information of the signal transmitting module in the three-dimensional space is determined in the real-time image according to the straight-line distance, so as to realize real-time positioning of the interactive device in the virtual scene.

2. The spatial positioning method according to claim 1, wherein: The signal transmitting module includes an infrared signal transmitting module and an ultrasonic signal transmitting module. The infrared signal transmitting module and the ultrasonic signal transmitting module are configured to transmit synchronously, and the infrared signal transmission speed of the infrared signal transmitting module is greater than the ultrasonic signal transmission speed of the ultrasonic signal transmitting module.

3. The spatial positioning method according to claim 2, wherein: The receiving a transmission signal transmitted by a signal transmitting module installed on the interactive device, and determining the transmission time length from when the transmission signal is transmitted to when the transmission signal is converted into an electrical signal by the receiving end, comprises: Respectively obtain first life cycle information of the infrared signal from being generated to being converted into a first electrical signal by the receiving end, second declaration cycle information of the ultrasonic signal from being generated to being converted into a second electrical signal by the receiving end, and the waiting time between the receiving end detecting the first electrical signal and detecting the second electrical signal; The second transmission time duration from when the second transmission signal is sent to when it is converted into a second electrical signal by the receiving end is calculated according to the first life cycle information, the second life cycle information and the waiting time duration.

4. The spatial positioning method according to claim 3, wherein: The first life cycle information includes at least a first transmission duration of the infrared signal received by the receiving end and a first conversion delay of the receiving end converting the infrared signal into an electrical signal, and the second life cycle information includes at least a second transmission duration of the ultrasonic signal received by the receiving end and a second conversion delay of the receiving end converting the ultrasonic signal into an electrical signal.

5. The spatial positioning method according to claim 4, wherein: The calculating, according to the first life cycle information, the second life cycle information and the waiting time, a second transmission time from when the second transmission signal is sent to when it is converted by the receiving end into a second electrical signal comprises: The first transmission duration, the first conversion delay, the second conversion delay and the waiting duration are substituted into the first calculation formula for calculation to obtain the second transmission duration of the second transmission signal from being sent to being converted into a second electrical signal by the receiving end.

6. The spatial positioning method according to claim 5, wherein: The first calculation formula is: b =t0+t r1 +t r2 -t b2 Among them, t b represents the second transmission duration, t0 represents the first transmission duration, t r1 represents the first switching delay, t r2 represents the second switching delay, t b2 Indicates the waiting time.

7. The spatial positioning method according to claim 2, wherein: The determining the three-dimensional coordinate information of the signal transmitting module in the three-dimensional space in the real-time image according to the straight-line distance includes: Determine in the real-time image the first distance at which the light spot generated by the infrared signal transmitting module reaches the nearest boundary of the real-time image and the second distance at which the light spot reaches the farthest end of the real-time image; Substituting the first distance, the second distance, the straight-line distance and the shooting field of view of the receiving end into a second calculation formula for calculation, to obtain the Z coordinate of the infrared signal transmitting module in the three-dimensional space; The X coordinate and the Y coordinate of the infrared signal transmitting module in the three-dimensional space are calculated according to the Z coordinate and a preset third calculation formula.

8. The spatial positioning method according to claim 7, wherein: The second calculation formula is: Wherein, Z represents the distance value of the infrared signal transmitting module in the Z-axis direction in the three-dimensional space, β represents the angle formed by the line connecting the infrared signal transmitting module to the receiving end and the normal line of the shooting field of view, a represents the shooting field of view, l a represents the first distance, l b Indicates the second distance.

9. The spatial positioning method according to claim 7, wherein: The step of calculating the X coordinate and the Y coordinate of the signal transmitting module in the three-dimensional space according to the Z coordinate and a preset third calculation formula includes: Determining a shooting resolution and a shooting calibration value of the receiving end, wherein the shooting calibration value is used to indicate a shooting distance when the spatial coordinates of the calibration object are consistent with the real coordinates; Substitute the shooting calibration value, the distance value in the X-axis direction and the distance value in the Y-axis direction of the infrared signal transmitting module in the real-time image with the center point of the real-time image as the origin into the third calculation formula to calculate the The X coordinate and Y coordinate of the infrared signal transmitting module in three-dimensional space.

10. The spatial positioning method according to claim 9, wherein: The third calculation formula is: X = Z / n*x' Y = Z / n*y' Among them, X represents the distance value of the infrared signal transmitting module in the X-axis direction in the three-dimensional space, Y represents the distance value of the infrared signal transmitting module in the Y-axis direction in the three-dimensional space, x' represents the distance value of the infrared signal transmitting module in the X-axis direction in the real-time image with the center point of the real-time image as the origin, y' represents the distance value of the infrared signal transmitting module in the Y-axis direction in the real-time image with the center point of the real-time image as the origin, Z represents the distance value of the infrared signal transmitting module in the Z-axis direction in the three-dimensional space, and n represents the shooting calibration value.

11. The spatial positioning method according to claim 1, wherein: The signal transmission module includes two groups and is respectively installed at the head and tail ends of the interactive device. After the straight-line distance between the interactive device and the receiving end is calculated, the method further includes: The straight-line distances between the two groups of signal transmission modules and the receiving end are compared, and the two ends of the interactive device are distinguished according to the preset working parameters of the interactive device.

12. The spatial positioning method according to claim 11, wherein: The working parameter is used to indicate the azimuth information corresponding to an end of the receiving end facing the receiving end in the working state, and the azimuth information includes any one of east, south, west and north.

13. A spatial positioning device for an interactive device, wherein: The spatial positioning device of the interactive device comprises: A receiving module, used to receive a transmission signal transmitted by at least one group of signal transmitting modules installed on the interactive device, and determine the transmission time length from the transmission of the transmission signal to the conversion of the transmission signal into an electrical signal by the receiving end; A calculation module, configured to calculate a straight-line distance between the interactive device and the receiving end according to the transmission duration and the transmission speed of the transmission signal; A determination module is used to obtain a real-time image of the signal transmitting module relative to the receiving end based on the monocular camera of the receiving end, and determine the three-dimensional coordinate information of the signal transmitting module in the three-dimensional space in the real-time image according to the straight-line distance, so as to realize real-time positioning of the interactive device in the virtual scene.

14. The spatial positioning device according to claim 13, wherein: The signal transmitting module includes an infrared signal transmitting module and an ultrasonic signal transmitting module. The infrared signal transmitting module and the ultrasonic signal transmitting module are configured to transmit synchronously, and the infrared signal transmission speed of the infrared signal transmitting module is greater than the ultrasonic signal transmission speed of the ultrasonic signal transmitting module.

15. The spatial positioning device according to claim 14, wherein: The receiving module is used to respectively obtain the first life cycle information of the infrared signal from its generation to its conversion into the first electrical signal by the receiving end, the second declaration cycle information of the ultrasonic signal from its generation to its conversion into the second electrical signal by the receiving end, and the waiting time between the receiving end detecting the first electrical signal and detecting the second electrical signal; based on the first life cycle information, the second life cycle information and the waiting time, the second transmission time of the second transmission signal from its issuance to its conversion into the second electrical signal by the receiving end is calculated.

16. The spatial positioning device according to claim 15, wherein: The first life cycle information includes at least a first transmission duration of the infrared signal received by the receiving end and a first conversion delay of the receiving end converting the infrared signal into an electrical signal, and the second life cycle information includes at least a second transmission duration of the ultrasonic signal received by the receiving end and a second conversion delay of the receiving end converting the ultrasonic signal into an electrical signal.

17. The spatial positioning device according to claim 16, wherein: The receiving module is used to substitute the first transmission duration, the first conversion delay, the second conversion delay and the waiting duration into the first calculation formula for calculation, so as to obtain the second transmission duration of the second transmission signal from being sent to being converted into a second electrical signal by the receiving end.

18. The spatial positioning device according to claim 14, wherein: The determination module is used to determine the shooting resolution and shooting calibration value of the receiving end, and the shooting calibration value is used to indicate the shooting distance of the calibration object when the spatial coordinates are consistent with the real coordinates; the shooting calibration value and the X-axis direction distance value and Y-axis direction distance value of the infrared signal transmitting module in the real-time image with the center point of the real-time image as the origin are substituted into the third calculation formula to calculate the X coordinate and Y coordinate of the infrared signal transmitting module in the three-dimensional space.

19. A computer-readable storage medium, wherein: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the spatial positioning method of an interactive device according to any one of claims 1-12.

20. A terminal device, wherein: It comprises a processor and a memory, wherein the memory stores a plurality of instructions, and the processor loads the instructions to execute the spatial positioning method of the interactive device according to any one of claims 1 to 12.

Images