Pose data processing method and system, electronic device and computer readable medium

Abstract

Embodiments of the present disclosure disclose a pose data processing method, system, electronic device and computer readable medium. A specific embodiment of the method comprises: obtaining head posture data of a user collected by a head-mounted display device; in response to detecting that the head-mounted display device does not meet a head position data acquisition condition, obtaining master pose data collected by a master device, wherein the master device is in communication connection with the head-mounted display device, the master pose data comprises master position data, and the head-mounted display device and the master device both correspond to the user; performing merging processing on the head posture data and the master position data to obtain merged pose data; and controlling the head-mounted display device to operate according to the obtained merged pose data. This embodiment can improve the sensitivity of the user in interacting with the head-mounted display device through head pose without increasing the weight and power consumption of the head-mounted display device.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, and more specifically to pose data processing methods, systems, electronic devices, and computer-readable media. Background Technology

[0002] Head-mounted displays allow users to view content in virtual spaces, providing an immersive interactive experience. Currently, to reduce the weight and power consumption of head-mounted displays, they typically only provide head posture data acquisition capabilities, using estimated position data for posture localization.

[0003] However, the inventors discovered that when using the above method for posture localization, the following technical problems often exist: the estimated position data has poor stability, which causes the user's interaction with the head-mounted display device through head posture to be relatively laggy and has poor sensitivity.

[0004] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0006] Some embodiments of this disclosure provide pose data processing methods, systems, electronic devices, and computer-readable media to address one or more of the technical problems mentioned in the background section above.

[0007] In a first aspect, some embodiments of this disclosure provide a pose data processing method, the method comprising: acquiring head pose data of a user collected by a head-mounted display device; in response to detecting that the head-mounted display device does not meet the conditions for acquiring head position data, acquiring master control pose data collected by a master control device, wherein the master control device is communicatively connected to the head-mounted display device, the master control pose data includes master control position data, and both the head-mounted display device and the master control device correspond to the user; merging the head pose data and the master control position data to obtain merged pose data; and controlling the operation of the head-mounted display device based on the obtained merged pose data.

[0008] Optionally, the aforementioned head-mounted display device is an AR device or a VR device.

[0009] Optionally, the aforementioned master pose data also includes master posture data; and the aforementioned merging process of the aforementioned head posture data and the aforementioned master position data to obtain merged pose data includes: deleting the aforementioned master posture data from the aforementioned master pose data to obtain master position data; and merging the aforementioned head posture data and the obtained master position data to obtain merged pose data.

[0010] Optionally, controlling the operation of the head-mounted display device based on the obtained merged pose data includes updating the display content of the head-mounted display device based on the obtained merged pose data.

[0011] Optionally, updating the display content of the head-mounted display device based on the obtained merged pose data includes: determining application update information corresponding to the current application based on the obtained merged pose data; and displaying the application update information on the display screen of the head-mounted display device to update the display content of the head-mounted display device.

[0012] Optionally, the aforementioned master control pose data also includes master control posture data; and before controlling the operation of the head-mounted display device based on the obtained merged pose data, the method further includes: in response to determining that the head-mounted display device has not collected real-time head posture data, determining the target historical head posture data collected by the head-mounted display device; generating smoothed posture data based on the target historical head posture data and the master control posture data; and merging the smoothed posture data and the master control position data to obtain merged pose data.

[0013] Optionally, before controlling the operation of the head-mounted display device based on the obtained merged pose data, the method further includes: in response to determining that the master control position data is empty, determining the target historical master control position data collected by the master control device; generating estimated position data based on the head posture data; generating smoothed position data based on the target historical master control position data and the estimated position data; and merging the head posture data and the smoothed position data to obtain merged pose data.

[0014] Secondly, some embodiments of this disclosure provide a pose data processing system, including: a head-mounted display device configured to collect head pose data of a user; and a main control device configured to execute the method described in any implementation of the first aspect above, wherein the main control device is communicatively connected to the head-mounted display device, and both the head-mounted display device and the main control device correspond to the user.

[0015] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0016] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0017] The above-described embodiments of this disclosure have the following beneficial effects: The pose data processing method of some embodiments of this disclosure can improve the sensitivity of user interaction with the head-mounted display device through head pose without increasing the weight and power consumption of the head-mounted display device. Specifically, the reason for the relatively laggy and insensitive interaction between the user and the head-mounted display device through head pose is that the estimated position data has poor stability, resulting in laggy and insensitive interaction. Based on this, the pose data processing method of some embodiments of this disclosure first acquires the user's head pose data collected by the head-mounted display device. Thus, the head pose data can characterize the user's head pose when wearing the head-mounted display device. Then, in response to detecting that the head-mounted display device does not meet the conditions for acquiring head position data, the main control pose data collected by the main control device is acquired. The main control device is communicatively connected to the head-mounted display device. The main control pose data includes main control position data. Both the head-mounted display device and the main control device correspond to the user. Therefore, even when the head-mounted display device lacks the ability to acquire head position data, it can automatically acquire master control pose data collected by the main control device, which includes master control position data. Then, the head pose data and the master control position data are merged to obtain merged pose data. This allows the posture data acquired by the head-mounted display device and the position data acquired by the main control device to be combined. Finally, the operation of the head-mounted display device is controlled based on the obtained merged pose data. Thus, the operation of the head-mounted display device can be controlled based on the merged pose data. Because estimated head position data is not used for positioning, but rather the position data acquired by the main control device is merged with the posture data acquired by the head-mounted display device to determine the final posture data, the stability of the position data is improved. This improves the sensitivity of user interaction with the head-mounted display device through head pose without increasing its weight or power consumption. Attached Figure Description

[0018] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0019] Figure 1 This is a schematic diagram of an application scenario of a pose data processing method according to some embodiments of the present disclosure; Figure 2 This is a flowchart of some embodiments of the pose data processing method according to the present disclosure; Figure 3 This is a flowchart of some other embodiments of the pose data processing method according to the present disclosure; Figure 4 These are schematic diagrams illustrating the structure of some embodiments of the pose data processing system according to this disclosure; Figure 5 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0020] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0021] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0022] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0023] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0024] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0025] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0026] Figure 1 This is a schematic diagram of an application scenario of a pose data processing method according to some embodiments of the present disclosure.

[0027] exist Figure 1 In the application scenario, firstly, the main control device 101 can acquire the user's head posture data 103 collected by the head-mounted display device 102. Then, in response to detecting that the head-mounted display device 102 does not meet the conditions for acquiring head position data, the main control device 101 can acquire the main control pose data 104 collected by the main control device 101. The main control device 101 is communicatively connected to the head-mounted display device 102. For example, the connection between the main control device 101 and the head-mounted display device 102 can be wireless. The main control pose data 104 includes main control position data 105. Both the head-mounted display device 102 and the main control device 101 correspond to the user. That is, the user wears the head-mounted display device 102 and holds the main control device 101. Afterwards, the main control device 101 can merge the head posture data 103 and the main control position data 105 to obtain merged pose data 106. Finally, the main control device 101 can control the operation of the head-mounted display device 102 based on the obtained merged pose data 106.

[0028] It should be noted that the aforementioned main control device 101 can be either hardware or software. When the main control device is hardware, it can be implemented as a distributed cluster composed of multiple servers or terminal devices, or as a single server or a single terminal device. When the main control device is software, it can be installed in the hardware devices listed above. It can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are made here.

[0029] It should be understood that Figure 1 The number of main control devices shown is merely illustrative. Depending on implementation needs, any number of main control devices can be used.

[0030] Continue to refer to Figure 2 The diagram illustrates a flow 200 of some embodiments of a pose data processing method according to the present disclosure. This pose data processing method includes the following steps: Step 201: Obtain the user's head posture data collected by the head-mounted display device.

[0031] In some embodiments, the execution subject of the pose data processing method (e.g. Figure 1The main control device 101 shown can acquire the user's head posture data collected by the head-mounted display device via a wired or wireless connection. The head-mounted display device can be a device used for users to view virtual spaces. For example, the head-mounted display device can be MR glasses. The user can be a user wearing the head-mounted display device and holding the main control device. The head posture data can be three-dimensional spatial coordinates, which can have three degrees of freedom. It should be noted that the wireless connection method can include, but is not limited to, 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (Ultra Wide Band) connection, and other currently known or future wireless connection methods.

[0032] Optionally, the aforementioned head-mounted display device can be an AR device or a VR device.

[0033] Step 202: In response to the detection that the head-mounted display device does not meet the head position data acquisition conditions, acquire the master control pose data collected by the master control device.

[0034] In some embodiments, the execution entity may, in response to detecting that the head-mounted display device does not meet the head position data acquisition conditions, acquire master control pose data collected by the master control device. The head position data acquisition conditions may include position data as a type of pose data that the head-mounted display device can acquire. That is, the head-mounted display device does not have the ability to acquire position data. Position data may be three-dimensional spatial coordinates, which may have three degrees of freedom. The master control device may be a computing device used for communicative connection with the head-mounted display device. For example, the master control device may be, but is not limited to, a mobile phone, a smart terminal, or a tablet computer. The master control device may have at least one of the following built-in: a camera or radar. The master control pose data may be pose data acquired by the master control device. The master control pose data may include master control position data. The master control position data may be position data determined by SLAM (Simultaneous Localization and Mapping) technology. Both the aforementioned head-mounted display device and the aforementioned main control device correspond to the aforementioned user. This can be understood as the user wearing the aforementioned head-mounted display device and holding the aforementioned main control device. In practice, the aforementioned execution entity can collect the main control pose data using SLAM technology.

[0035] Step 203: Merge the head pose data and the master position data to obtain merged pose data.

[0036] In some embodiments, the executing entity can merge the head pose data and the master control position data to obtain merged pose data. In practice, firstly, the executing entity can determine the head pose data as three-degree-of-freedom (DOF) pose data. Then, it can determine the master control position data as three-degree-of-freedom (DOF) position data. Finally, it can combine the three-DOF pose data and the three-DOF position data into merged pose data. The merged pose data is six-degree-of-freedom (DOF) pose data.

[0037] Optionally, the aforementioned master control pose data may further include master control attitude data. The master control attitude data can be attitude data collected by the aforementioned master control device. Here, the master control attitude data can be directional coordinates in three-dimensional space, and can have three degrees of freedom.

[0038] In some optional implementations of certain embodiments, the execution entity may further merge the head pose data and the master control position data through the following steps to obtain merged pose data: The first step is to delete the master control attitude data from the master control pose data to obtain the master control position data.

[0039] The second step involves merging the aforementioned head pose data and the obtained master control position data to obtain merged pose data. The merging process can be referenced from the specific implementation of step 203, and will not be elaborated further here. Therefore, the master control pose data can be directly deleted from the master control pose data before merging, saving storage space.

[0040] Step 204: Control the operation of the head-mounted display device based on the obtained merged pose data.

[0041] In some embodiments, the execution entity can control the operation of the head-mounted display device based on the obtained merged pose data. In practice, the execution entity can determine the selection item corresponding to the merged pose data based on the merged pose data. The selection item can be a selection made by the user from various selection items through head control operation. Here, the specific content of the selection item is not limited. For example, the selection item can be "Confirm Playback". Then, the execution entity can control the head-mounted display device to perform the operation corresponding to the selection item. For example, the execution entity can control the head-mounted display device to play the audio corresponding to the selection item.

[0042] In some optional implementations of certain embodiments, the aforementioned execution entity may update the display content of the aforementioned head-mounted display device based on the obtained merged pose data.

[0043] In some optional implementations of certain embodiments, the aforementioned execution entity may update the display content of the aforementioned head-mounted display device based on the obtained merged pose data through the following steps: The first step is to determine the application update information corresponding to the current application based on the obtained merged pose data. In practice, the aforementioned execution entity can determine the application execution node corresponding to the application operation represented by the merged pose data among the various application execution nodes of the current application. Here, the current application can be the application currently running in the foreground. The various application execution nodes can be the execution nodes corresponding to the running logic of the current application. For example, if a user clicks control 'a' on application page A and is redirected to application page B, then the application execution node "application page B" can be considered the execution node following application execution node "application page A" after triggering control 'a'. Here, the application operation represented by the merged pose data can be the operation that triggers control 'a'. Then, the application information corresponding to the aforementioned application execution nodes can be determined as the application update information. This application information can be the information that needs to be displayed when executing the application execution node. For example, the application information can be the page display information of the application page. The page display information can be the information used to display the application page.

[0044] The second step is to display the application update information on the screen of the aforementioned head-mounted display device to update the display content of the head-mounted display device. In practice, the executing entity can display the application page corresponding to the application update information on the screen of the aforementioned head-mounted display device to update the display content of the aforementioned head-mounted display device.

[0045] The above-described embodiments of this disclosure have the following beneficial effects: The pose data processing method of some embodiments of this disclosure can improve the sensitivity of user interaction with the head-mounted display device through head pose without increasing the weight and power consumption of the head-mounted display device. Specifically, the reason for the relatively laggy and insensitive interaction between the user and the head-mounted display device through head pose is that the estimated position data has poor stability, resulting in laggy and insensitive interaction. Based on this, the pose data processing method of some embodiments of this disclosure first acquires the user's head pose data collected by the head-mounted display device. Thus, the head pose data can characterize the user's head pose when wearing the head-mounted display device. Then, in response to detecting that the head-mounted display device does not meet the conditions for acquiring head position data, the main control pose data collected by the main control device is acquired. The main control device is communicatively connected to the head-mounted display device. The main control pose data includes main control position data. Both the head-mounted display device and the main control device correspond to the user. Therefore, even when the head-mounted display device lacks the ability to acquire head position data, it can automatically acquire master control pose data collected by the main control device, which includes master control position data. Then, the head pose data and the master control position data are merged to obtain merged pose data. This allows the posture data acquired by the head-mounted display device and the position data acquired by the main control device to be combined. Finally, the operation of the head-mounted display device is controlled based on the obtained merged pose data. Thus, the operation of the head-mounted display device can be controlled based on the merged pose data. Because estimated head position data is not used for positioning, but rather the position data acquired by the main control device is merged with the posture data acquired by the head-mounted display device to determine the final posture data, the stability of the position data is improved. This improves the sensitivity of user interaction with the head-mounted display device through head pose without increasing its weight or power consumption.

[0046] Further reference Figure 3 This illustrates a flow 300 of another embodiment of the pose data processing method. Flow 300 of this pose data processing method includes the following steps: Step 301: Obtain the user's head posture data collected by the head-mounted display device.

[0047] Step 302: In response to the detection that the head-mounted display device does not meet the head position data acquisition conditions, acquire the master control pose data collected by the master control device.

[0048] Step 303: Merge the head pose data and the master position data to obtain merged pose data.

[0049] In some embodiments, the specific implementation of steps 301-303 and the resulting technical effects can be found in [reference needed]. Figure 2 Steps 201-203 in the corresponding embodiments will not be repeated here.

[0050] Step 304: In response to determining that the head-mounted display device has not collected real-time head posture data, determine the target historical head posture data collected by the head-mounted display device.

[0051] In some embodiments, the execution subject of the pose data processing method (e.g. Figure 1 The main control device 101 shown can, in response to determining that the head-mounted display device has not collected real-time head posture data, determine the target historical head posture data collected by the head-mounted display device. The target historical head posture data can be head posture data collected by the head-mounted display device up to the current time. The target historical head posture data can include at least one most recent historical head posture data collected by the head-mounted display device up to the current time. For example, the current time could be time 5, and the target historical head posture data could include historical head posture data collected by the head-mounted display device from time 1 to time 4.

[0052] Step 305: Generate smoothed posture data based on the target's historical head posture data and master control posture data.

[0053] In some embodiments, the executing entity can generate smoothed posture data based on the target's historical head posture data and the master control posture data. In practice, the executing entity can smooth the master control posture data based on the target's historical head posture data to obtain smoothed posture data. Specifically, the executing entity can use a smoothing algorithm to smooth the master control posture data to obtain smoothed posture data.

[0054] Step 306: Merge the smoothed pose data and the master position data to obtain merged pose data.

[0055] In some embodiments, the executing entity can merge the smoothed attitude data and the master control position data to obtain merged pose data. In practice, firstly, the executing entity can determine the smoothed attitude data as three-degree-of-freedom (DOF) attitude data. Then, it can determine the master control position data as three-degree-of-freedom (DOF) position data. Finally, it can combine the three-DOF attitude data and the three-DOF position data into merged pose data. The merged pose data is six-degree-of-freedom (DOF) attitude data.

[0056] Optionally, before step 307, the aforementioned executing entity may also perform the following steps: The first step, in response to determining that the aforementioned master control location data is empty, is to determine the target historical master control location data collected by the aforementioned master control device. Where the aforementioned master control location data is empty, it indicates that the aforementioned master control device has not collected master control location data. The aforementioned target historical master control location data can be the master control location data collected by the aforementioned master control device before the current time. The aforementioned target historical master control location data can include at least one most recent historical master control location data collected by the aforementioned master control device before the current time. For example, the current time can be time 5, and the aforementioned target historical master control location data can include the historical master control location data collected by the aforementioned master control device from time 1 to time 4.

[0057] The second step is to generate estimated position data based on the aforementioned head pose data. In practice, the executing entity can generate estimated pose data using the aforementioned head pose data via a SLAM algorithm. This estimated pose data can include both position data and pose data. Then, the position data included in the estimated pose data can be determined as the estimated position data.

[0058] The third step is to generate smoothed position data based on the historical master control position data and the estimated position data. In practice, the executing entity can smooth the estimated position data using the historical master control position data to obtain smoothed position data. Specifically, the executing entity can use a smoothing algorithm to smooth the estimated position data to obtain smoothed position data.

[0059] The fourth step involves merging the head pose data and the smoothed position data to obtain merged pose data. In practice, firstly, the executing entity can determine the head pose data as three-degree-of-freedom (DOF) pose data. Then, the smoothed position data can be determined as three-degree-of-freedom (DOF) position data. Finally, the three-degree-of-freedom pose data and the three-degree-of-freedom position data can be combined to form merged pose data. The merged pose data is six-degree-of-freedom (DOF) pose data. Therefore, when the main control device fails to acquire real-time main control position data (i.e., when the main control device's main control position data acquisition function malfunctions), the smoothed position data can be used as the three-degree-of-freedom position data. This allows the user to continue head interaction even when the main control device's main control position data acquisition function fails. Furthermore, because the three-degree-of-freedom position data is obtained by smoothing historical main control position data previously acquired by the main control device, it avoids directly using estimated position data as the three-degree-of-freedom position data, improving the stability of the three-degree-of-freedom position data. This, in turn, reduces the lag in head interaction operations.

[0060] Step 307: Control the operation of the head-mounted display device based on the obtained merged pose data.

[0061] In some embodiments, the specific implementation of step 307 and its resulting technical effects can be found in [reference needed]. Figure 2 Step 204 in the corresponding embodiments will not be repeated here.

[0062] from Figure 3 It can be seen from this that, with Figure 2 Compared to the description of some corresponding embodiments, Figure 3 The flow 300 of the pose data processing method in some corresponding embodiments embodies the step of extending the smoothed pose data. Therefore, the scheme described in these embodiments can use smoothed pose data obtained by smoothing the target historical head pose data collected by the head-mounted display device and the master control pose data collected by the master control device as the pose three-degree-of-freedom data when the head-mounted display device fails to collect real-time head pose data, i.e., when the head pose data acquisition function of the head-mounted display device malfunctions. This allows the user to continue head interaction operations even when the head pose data acquisition function of the head-mounted display device fails, reducing the lag in head interaction operations.

[0063] Further reference Figure 4 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a pose data processing system.

[0064] like Figure 4 As shown, a pose data processing system 400 in some embodiments includes: a head-mounted display device 401 configured to acquire user head pose data; and a main control device 402 configured to perform actions such as... Figure 2 or Figure 3 In any of the corresponding embodiments of the pose data processing method, the main control device is communicatively connected to the head-mounted display device, and both the head-mounted display device and the main control device correspond to the user.

[0065] The above-described embodiments of this disclosure have the following beneficial effects: Through the pose data processing system of some embodiments of this disclosure, the sensitivity of users interacting with the head-mounted display device through head pose can be improved without increasing the weight and power consumption of the head-mounted display device.

[0066] The following is for reference. Figure 5 It illustrates an electronic device 500 suitable for implementing some embodiments of the present disclosure (e.g., Figure 1 A schematic diagram of the structure of the main control device 101. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0067] like Figure 5As shown, the electronic device 500 may include a processing unit 501 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0068] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 5 Each box shown can represent a device or multiple devices as needed.

[0069] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 509, or installed from storage device 508, or installed from ROM 502. When the computer program is executed by processing device 501, it performs the functions defined in the methods of some embodiments of this disclosure.

[0070] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0071] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0072] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire head posture data of a user collected by a head-mounted display device; in response to detecting that the head-mounted display device does not meet the conditions for acquiring head position data, acquire master control posture data collected by a master control device, wherein the master control device is communicatively connected to the head-mounted display device, the master control posture data includes master control position data, and both the head-mounted display device and the master control device correspond to the aforementioned user; merge the head posture data and the master control position data to obtain merged posture data; and control the operation of the head-mounted display device based on the obtained merged posture data.

[0073] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0074] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0075] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0076] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A pose data processing method, comprising: Acquire head posture data of the user collected by a head-mounted display device, wherein the head posture data is directional coordinates in three-dimensional space and has three degrees of freedom of posture; In response to the detection that the head-mounted display device does not meet the head position data acquisition conditions, the main control device acquires the main control pose data collected by the main control device. The main control device is communicatively connected to the head-mounted display device. The main control pose data includes main control position data, which is position data determined by SLAM technology. The position data is position coordinates in three-dimensional space with three degrees of freedom. Both the head-mounted display device and the main control device correspond to the user. The head pose data and the master control position data are merged to obtain merged pose data. The merging process includes: The head posture data is defined as three-degree-of-freedom posture data. The master control position data is determined as position three-degree-of-freedom data; The attitude three-degree-of-freedom data and the position three-degree-of-freedom data are combined into merged pose data, wherein the merged pose data is six-degree-of-freedom attitude data; Based on the obtained merged pose data, the head-mounted display device is controlled to operate, wherein controlling the operation of the head-mounted display device based on the obtained merged pose data includes: Determine the application execution node corresponding to the application operation represented by the merged pose data among the various application execution nodes of the current application, wherein the application operation represented by the merged pose data is the operation of triggering the control; The application information corresponding to the application execution node is determined as application update information; An application page corresponding to the application update information is displayed on the display screen of the head-mounted display device to update the display content of the head-mounted display device.

2. The method according to claim 1, wherein, The head-mounted display device is an AR device or a VR device.

3. The method according to claim 1, wherein, The master control pose data also includes master control attitude data; and The process of merging the head pose data and the master control position data to obtain merged pose data includes: Delete the master control posture data from the master control pose data to obtain the master control position data; The head pose data and the obtained master position data are merged to obtain merged pose data.

4. The method according to claim 1, wherein, The master control pose data also includes master control attitude data; as well as Before controlling the operation of the head-mounted display device based on the obtained merged pose data, the method further includes: In response to determining that the head-mounted display device has not collected real-time head posture data, the target historical head posture data collected by the head-mounted display device is determined; Based on the target's historical head posture data and the master control posture data, smooth posture data is generated; The smoothed pose data and the master control position data are merged to obtain merged pose data.

5. The method according to any one of claims 1-4, wherein, Before controlling the operation of the head-mounted display device based on the obtained merged pose data, the method further includes: In response to determining that the master control location data is empty, the target historical master control location data collected by the master control device is determined; Based on the head pose data, estimated position data is generated; Based on the target's historical master control location data and the estimated location data, smoothed location data is generated; The head pose data and the smoothed position data are merged to obtain merged pose data.

6. A pose data processing system, comprising: Head-mounted display devices are configured to collect user head posture data; A master control device is configured to perform the method as described in any one of claims 1-5, wherein the master control device is communicatively connected to the head-mounted display device, and both the head-mounted display device and the master control device correspond to the user.

7. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

8. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-5.

Images