Data receiving method and device of split head-mounted device, equipment and medium

Abstract

The application discloses a data receiving method and device of a split head-mounted device, equipment and a medium, and relates to the technical field of head-mounted devices. The split head-mounted device comprises a control device and a head-mounted display device. The method is applied to the control device and comprises the following steps: establishing a first communication connection with the head-mounted display device and a second communication connection with a terminal device; sending first priority configuration information to the head-mounted display device based on the first communication connection; and sending second priority configuration information to the terminal device based on the second communication connection. The first priority configuration information is used to indicate that the control device uses a first receiving queue when receiving pose data sent by the head-mounted display device, and the second priority configuration information is used to indicate that the control device uses a second receiving queue when receiving display data sent by the terminal device. The priority of the first receiving queue is higher than that of the second receiving queue. The method can realize the processing of the control device on the pose data prior to the display data.

Description

Technical Field

[0001] This application relates to the field of head-mounted display technology, and more specifically, to a data receiving method for a split head-mounted display, a data receiving device for a split head-mounted display, a control device, a split head-mounted display, and a computer-readable storage medium. Background Technology

[0002] Smart head-mounted devices generally come in two forms: integrated and modular. Compared to integrated smart head-mounted devices, modular smart head-mounted devices offer advantages in power consumption, battery life, weight, and heat dissipation. Therefore, modular smart head-mounted devices have become a market and technological trend. Modular smart head-mounted devices include a head-mounted display and a computing unit.

[0003] A modular smart head-mounted device can display image data from terminal devices. That is, at least one terminal device sends image data running in the foreground and / or background to the computing unit for display by the head-mounted display device.

[0004] Currently, when split-type smart head-mounted devices display image data from terminal devices, there are issues such as dizziness caused by the image displayed by the head-mounted display not matching the user's head posture, or lag in the head-mounted display. Summary of the Invention

[0005] One objective of this application is to provide a new technical solution for data reception in a split-type head-mounted device.

[0006] According to a first aspect of this application, a data receiving method for a split-type head-mounted device is provided, the split-type head-mounted device including a control device and a head-mounted display device, the method being applied to the control device, comprising:

[0007] Establish a first communication connection with the head-mounted display device and a second communication connection with the terminal device;

[0008] Based on the first communication connection, send first priority configuration information to the head-mounted display device;

[0009] Based on the second communication connection, send second priority configuration information to the terminal device;

[0010] The first priority configuration information is used to instruct the control device to use a first receiving queue when receiving pose data sent by the head-mounted display device, and the second priority configuration information is used to instruct the control device to use a second receiving queue when receiving display data sent by the terminal device, wherein the priority of the first receiving queue is higher than the priority of the second receiving queue.

[0011] Optionally, the terminal devices are at least two, and the method further includes:

[0012] Receive pose data sent by the head-mounted display device based on the first priority configuration information;

[0013] Based on the pose data, the priority among the terminal devices is determined;

[0014] Based on the priority among the terminal devices, a second receiving queue adjustment notification is sent to the terminal devices so that the priority of the second receiving queue used by the control device when receiving display data sent by a terminal device with a higher priority is higher than the priority of the second receiving queue used when receiving display data sent by a terminal device with a lower priority.

[0015] Optionally, the method further includes:

[0016] Receive pose data sent by the head-mounted display device based on the first priority configuration information;

[0017] Based on the pose data, a target area is determined, wherein the target area is a display area that does not belong to the gaze trend of the wearer of the head-mounted device;

[0018] Send a notification to the terminal device corresponding to the target area to stop displaying data transmission;

[0019] The head-mounted display device displays the display area corresponding to the terminal device.

[0020] Optionally, the method further includes:

[0021] Based on the pose data, determine whether the target display area belongs to the display area on the wearer's gaze trend;

[0022] In this case, a notification to start displaying data transmission is sent to the terminal device corresponding to the target display area.

[0023] Optionally, the method further includes:

[0024] A first receiving queue matching the first priority configuration information and a second receiving queue matching the second priority configuration information are established.

[0025] Optionally, the first priority configuration information is a first DSCP configuration information, and the second priority configuration information is a second DSCP configuration information, wherein the receiving queue corresponding to the DSCP value indicated by the first DSCP configuration information is different from the receiving queue corresponding to the second DSCP value, and the DSCP value indicated by the first DSCP configuration information is greater than the DSCP value indicated by the second DSCP configuration information.

[0026] According to a second aspect of this application, a data receiving device for a split-type head-mounted device is provided, the split-type head-mounted device including a control device and a head-mounted display device, the device being applied to the control device, comprising:

[0027] A module is established to establish a first communication connection with the head-mounted display device and a second communication connection with the terminal device.

[0028] The first sending module is configured to send first priority configuration information to the head-mounted display device based on the first communication connection;

[0029] The second sending module is used to send second priority configuration information to the terminal device based on the second communication connection;

[0030] The first priority configuration information is used to instruct the control device to use a first receiving queue when receiving pose data sent by the head-mounted display device, and the second priority configuration information is used to instruct the control device to use a second receiving queue when receiving display data sent by the terminal device, wherein the priority of the first receiving queue is higher than the priority of the second receiving queue.

[0031] According to a third aspect of this application, a control device for a split-type head-mounted device is provided, the control device comprising a data receiving device for the split-type head-mounted device as described in the second aspect; or,

[0032] The control device includes a memory and a processor, the memory for storing computer instructions, and the processor for retrieving the computer instructions from the memory to execute the data receiving method of the split head-mounted device as described in any one of the first aspects.

[0033] According to a fourth aspect of this application, a split-type head-mounted device is provided, comprising:

[0034] The control device as described in the third aspect;

[0035] And a head-mounted display device, configured to receive first priority configuration information sent by the control device, encapsulate pose data to be sent to the control device according to the first priority configuration information, and send the encapsulated pose data to the control device.

[0036] According to a fifth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of the first aspects.

[0037] This application provides a data receiving method for a split-type head-mounted device. The split-type head-mounted device includes a control device and a head-mounted display device. The method is applied to the control device and includes: establishing a first communication connection with the head-mounted display device and a second communication connection with a terminal device; sending first priority configuration information to the head-mounted display device based on the first communication connection; and sending second priority configuration information to the terminal device based on the second communication connection. The first priority configuration information instructs the control device to use a first receiving queue when receiving pose data sent by the head-mounted display device, and the second priority configuration information instructs the control device to use a second receiving queue when receiving display data sent by the terminal device. The first receiving queue has a higher priority than the second receiving queue. This method enables the control device to prioritize pose data over display data. Furthermore, it avoids dizziness caused by mismatched images displayed on the head-mounted display device due to untimely processing of pose data, or head-mounted display device lag.

[0038] Other features and advantages of this application will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0039] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the present application and, together with their description, serve to explain the principles of the present application.

[0040] Figure 1 This is a flowchart illustrating a data receiving method for a split-type head-mounted device provided in an embodiment of this application;

[0041] Figure 2 This is a scenario illustration of a data receiving method for a split-type head-mounted device provided in an embodiment of this application. Figure 1 ;

[0042] Figure 3 This is a scenario illustration of a data receiving method for a split-type head-mounted device provided in an embodiment of this application. Figure 2 ;

[0043] Figure 4 This is a scenario illustration of a data receiving method for a split-type head-mounted device provided in an embodiment of this application. Figure 3 ;

[0044] Figure 5This is a schematic diagram of the structure of a data receiving device for a split-type head-mounted device provided in an embodiment of this application;

[0045] Figure 6 This is a schematic diagram of the structure of a control device for a split-type head-mounted device provided in an embodiment of this application. Detailed Implementation

[0046] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0047] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the scope of this application and its application or use.

[0048] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0049] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0050] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0051] Currently, the applicant has discovered that when a split-type smart head-mounted device displays image data from a terminal device, the pose data sent by the head-mounted display device and the image data sent by the terminal device enter the same receiving queue in the computing unit. When image data enters the same receiving queue before pose data, the image data is processed by the computing unit before the pose data. This can lead to pose data not being processed in a timely manner, causing dizziness due to the image displayed by the head-mounted display device not matching the user's head posture, or causing the head-mounted display device to lag. To solve this problem, this application provides a data receiving method for a split-type head-mounted device.

[0052] This application provides a data receiving method for a split-type head-mounted device. The split-type head-mounted device includes a control device and a head-mounted display device. The head-mounted display device can be, for example, AR, MR, VR, or XR glasses or helmets. The control device is specifically a data processing unit of the head-mounted display device, also referred to as a computing unit. The control device is a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application does not specifically limit this. The data receiving method for the split-type head-mounted device provided in this application is applied to the control device.

[0053] like Figure 1 As shown, the data receiving method for the split-type head-mounted device provided in this application embodiment includes the following steps S1100-S1300.

[0054] S1100, Establish a first communication connection with the head-mounted display device and a second communication connection with the terminal device.

[0055] The application scenarios of this application include a split-type head-mounted device and at least one terminal device. A first communication connection is established between the control device of the split-type head-mounted device and the head-mounted display device. A second communication connection is established between the control device and any of the terminal devices. The first and second communication connections can be, for example, any one of USB connection, WIFI connection, Bluetooth connection, etc. The specific methods of establishing the first and second communication connections are not limited in the embodiments of this application.

[0056] Furthermore, the second communication connection for different terminal devices may be the same or not entirely the same. For example, the second communication connection between terminal device A and the control device may be a Wi-Fi connection, while the second communication connection between terminal device B and the control device may be a USB connection.

[0057] In this embodiment of the application, the head-mounted display device sends its pose data to the control device via a first communication connection. The pose data can be, for example, 3DoF or 6DoF data.

[0058] Any terminal device, based on a second communication connection, sends display data to a control device. The display data consists of foreground and / or background image data to be sent by the terminal device to the control device for display. Alternatively, the terminal device can send foreground and / or background image data to the control device via streaming communication. In this method, the terminal device can be referred to as a streaming device, and the image data sent by the terminal device can be referred to as streaming data.

[0059] The control device divides the display screen of the head-mounted display device into different display areas to display processed display data sent by different terminal devices. In other words, the display screen of the head-mounted display device displays display areas corresponding one-to-one with each terminal device. The control device processes the received display data based on pose data and sends the processed display data to the head-mounted display device via a first communication connection, so that the head-mounted display device can display the data in the corresponding display area.

[0060] Based on the above, in one example, such as Figure 2 As shown, taking terminal devices 1, 2, and 3 as examples, terminal device 1 sends display data 1 to the control device via a second communication connection, terminal device 2 sends display data 2 to the control device via the second communication connection, and terminal device 3 sends display data 3 to the control device via the second communication connection. The head-mounted display device sends pose data to the control device. Based on the pose data, the control device processes display data 1, display data 2, and display data 3 (e.g., splicing, compositing, cropping, etc.) to obtain processed display data 1 to be displayed in display area 1, processed display data 2 to be displayed in display area 2, and processed display data 3 to be displayed in display area 3. Further, the control device displays processed display data 1 in display area 1, processed display data 2 in display area 2, and processed display data 3 in display area 3.

[0061] S1200: Based on the first communication connection, send first priority configuration information to the head-mounted display device.

[0062] S1300: Based on the second communication connection, send second priority configuration information to the terminal device.

[0063] The first priority configuration information is used to instruct the control device to use the first receiving queue when receiving pose data sent by the head-mounted display device, and the second priority configuration information is used to instruct the control device to use the second receiving queue when receiving display data sent by the terminal device. The priority of the first receiving queue is higher than that of the second receiving queue.

[0064] In this embodiment, the priority configuration information sent to the terminal device is collectively referred to as the second priority configuration information. This means that the second receiving queues specifically indicated by different second priority configuration information may be exactly the same or not. For example, if the terminal device specifically includes terminal device A and terminal device B, and the receiving queues on the control device side include a Video queue and a Best Effort queue, the second receiving queue indicated by the second priority configuration information sent by the control device to terminal device A is the Video queue, and the second receiving queue indicated by the second priority configuration information sent to terminal device B is the Best Effort queue.

[0065] In this embodiment, the control device sends first priority configuration information to the head-mounted display device. Upon receiving the first priority configuration information, the head-mounted display device encapsulates the pose data it has collected according to the first priority configuration information and sends the encapsulated pose data to the control device via a first communication connection. After receiving the pose data, the control device parses the received pose data and stores it in a first receiving queue.

[0066] Furthermore, for any terminal device, the control device sends second priority configuration information to the terminal device. Upon receiving the second priority configuration information, the terminal device encapsulates the display data to be sent to the control device according to the second priority configuration information, and sends the encapsulated display data to the control device via the second communication connection. After receiving the display data, the control device parses the received display data and stores the display data in a second receiving queue.

[0067] Because the first receiving queue has a higher priority than the second receiving queue, the control device prioritizes scheduling the pose data from the first receiving queue to the corresponding functional module for processing. After scheduling the pose data from the first receiving queue, the control device then schedules the display data from the second receiving queue to the corresponding functional module for processing. This ensures that the control device prioritizes pose data over display data. Furthermore, it avoids dizziness caused by mismatched images displayed on the head-mounted display due to unprocessed pose data, or head-mounted display lag.

[0068] In conjunction with the above embodiments, such as Figure 3As shown, taking the first and second communication connections as Wi-Fi connections, the terminal devices include terminal device 1, terminal device 2, and terminal device 3, and the head-mounted display device as AR glasses as an example, the control device side includes a Voice queue, a Video queue, a Best Effort queue, and a Background queue. The control device sends first priority configuration information to the AR glasses, configuring the first receiving queue as the Voice queue, and sends second priority configuration information to terminal devices 1, 2, and 3, configuring the second receiving queue as the Best Effort queue. Upon receiving the first priority configuration information, the AR glasses encapsulate pose data according to the Voice queue configuration and send the encapsulated pose data to the control device. Terminal devices 1, 2, and 3 all encapsulate display data according to the Best Effort queue configuration and send the encapsulated display data to the control device. After receiving the pose data, the control device stores the pose data in the Voice queue. After receiving the display data, the control device stores all display data in the Best Effort queue. The scheduling module in the control device first schedules the pose data in the Voice queue to the functional module 1 that processes pose data. After the pose data scheduling is completed, the Best Effort queue then schedules the display data in the Best Effort queue to the functional module 2 that processes display data.

[0069] For the example above, it is also possible to send second priority configuration information configuring the second receive queue as the Best Effort queue to terminal device 1 and terminal device 3. And to send second priority configuration information configuring the second receive queue as the Background queue to terminal device 3.

[0070] This application provides a data receiving method for a split-type head-mounted device. The split-type head-mounted device includes a control device and a head-mounted display device. The method is applied to the control device and includes: establishing a first communication connection with the head-mounted display device and a second communication connection with a terminal device; sending first priority configuration information to the head-mounted display device based on the first communication connection; and sending second priority configuration information to the terminal device based on the second communication connection. The first priority configuration information instructs the control device to use a first receiving queue when receiving pose data sent by the head-mounted display device, and the second priority configuration information instructs the control device to use a second receiving queue when receiving display data sent by the terminal device. The first receiving queue has a higher priority than the second receiving queue. This method enables the control device to prioritize pose data over display data. Furthermore, it avoids dizziness caused by mismatched images displayed on the head-mounted display device due to untimely processing of pose data, or head-mounted display device lag.

[0071] In one embodiment of this application, the aforementioned first priority configuration information and second priority configuration information may specifically be instruction information for configuring DSCP. Based on this, the first priority configuration information is first DSCP configuration information, and the second priority configuration information is second DSCP configuration information, wherein the receiving queue corresponding to the DSCP value indicated by the first DSCP configuration information is different from the receiving queue corresponding to the second DSCP value, and the DSCP value indicated by the first DSCP configuration information is greater than the DSCP value indicated by the second DSCP configuration information.

[0072] Understandably, the DSCP value is a field in the IP layer when encapsulating data, and it is used to indicate the receive queue used when the data is received.

[0073] The mapping relationship between DSCP value ranges and receiving queues is shown in Table 1 below. It can be understood that different DSCP value ranges correspond to different receiving queues, and these queues have different priorities. Specifically: when the DSCP value is [0-7] and [24-31], the corresponding receiving queue is the Best Effort queue; when the DSCP value is [8-15] and [16-23], the corresponding receiving queue is the Background queue; when the DSCP value is [32-39] and [40-47], the corresponding receiving queue is the Video queue; and when the DSCP value is [48-55] and [56-63], the corresponding receiving queue is the Voice queue. The Voice queue has a higher priority than the Video queue, which in turn has a higher priority than the Best Effort queue, which in turn has a higher priority than the Background queue.

[0074] Table 1

[0075] DSCP value range Receive queue 56-63 Voice 48-55 Voice 40-47 Video 32-39 Video 24-31 Best Effort 0-7 Best Effort 16-23 Background 8-15 Background

[0076] Based on Table 1 above, by setting the first DSCP configuration information indicator DSCP value to any value between [0-7] and [24-31], the first receiving queue indicated by the first DSCP configuration information can be set as the Best Effort queue. By setting the first DSCP configuration information indicator DSCP value to any value between [8-15] and [16-23], the first receiving queue indicated by the first DSCP configuration information can be set as the Background queue. By setting the first DSCP configuration information indicator DSCP value to any value between [32-39] and [40-47], the first receiving queue indicated by the first DSCP configuration information can be set as the Video queue. By setting the first DSCP configuration information indicator DSCP value to any value between [48-55] and [56-63], the first receiving queue indicated by the first DSCP configuration information can be set as the Voice queue. Similarly, the second DSCP configuration information is set in the same way.

[0077] Based on the above, it can be seen that the receiving queue corresponding to the DSCP value indicated by the first DSCP configuration information and the receiving queue corresponding to the second DSCP value can be set to be different, and the DSCP value indicated by the first DSCP configuration information is greater than the DSCP value indicated by the second DSCP configuration information. This can achieve the priority of the first receiving queue indicated by the first priority configuration information being higher than the priority of the second receiving queue indicated by the second priority configuration information.

[0078] It should be noted that when the first priority configuration information is the first DSCP configuration information, the head-mounted display device encapsulates the pose data by filling the pose data into the application layer of the data packet and filling the DSCP value indicated by the first DSCP configuration information into the DSCP field of the IP layer of the data packet to obtain the encapsulated pose data. The same applies when the second priority configuration information is the second DSCP configuration information.

[0079] In one embodiment of this application, when there are at least two terminal devices, the data receiving method for the split-type head-mounted device provided in this application embodiment further includes the following steps S1400-S1600.

[0080] S1400: Receive pose data sent by the head-mounted display device based on the first priority configuration information.

[0081] In this embodiment of the application, the control device can read pose data from the first receiving queue.

[0082] S1500: Determine the priority between terminal devices based on the pose data.

[0083] In this embodiment, the wearer's gaze trend can be determined based on pose data. For example, taking display area 1, display area 2, and display area 3 displayed sequentially from left to right on the display area, when the wearer is currently looking at display area 1, if the pose data indicates that the wearer's head is moving from left to right, then the wearer's gaze trend is determined to be looking at display area 1, i.e., looking at display area 2. When the wearer is currently looking at display area 2, if the pose data indicates that the wearer's head is moving from left to right, then the wearer's gaze trend is determined to be looking at display area 2, i.e., looking at display area 3. When the wearer is currently looking at display area 2, if the pose data indicates that the wearer's head is moving from right to left, then the wearer's gaze trend is determined to be looking at display area 2, i.e., looking at display area 1.

[0084] Furthermore, the priority of a terminal device is directly proportional to the order in which the wearer views the display area on the head-mounted display device corresponding to that terminal device. That is, the terminal device corresponding to the display area viewed first has a higher priority. For example, the terminal device corresponding to the display area first viewed by the wearer of the head-mounted display device has the highest priority.

[0085] It should be noted that in actual use cases, the wearer only focuses on one display area at a time.

[0086] In the embodiments of this application, in such Figure 2Based on the example shown, if it is determined from the pose data that the wearer is looking at display area 1 and will next look at display area 2, then the priority of terminal device 1 corresponding to display area 1 is determined to be higher than the priority of terminal device 2 corresponding to display area 2.

[0087] S1600. Based on the priority among the terminal devices, a second receiving queue adjustment notification is sent to the terminal devices so that the priority of the second receiving queue used by the control device when receiving display data sent by a terminal device with a higher priority is higher than the priority of the second receiving queue used when receiving display data sent by a terminal device with a lower priority.

[0088] In this embodiment, the adjustment notification is used to instruct the terminal device to adjust the second receiving queue indicated when encapsulating display data. The adjustment notification can be, for example, a notification indicating a new second receiving queue, or it can be a notification indicating that the second receiving queue is increased / decreased / remains unchanged.

[0089] Based on the above S1600, the priority of the second receiving queue indicated by the high-priority terminal device when encapsulating display data is higher than the priority of the second receiving queue indicated by the low-priority terminal device when encapsulating terminal device data.

[0090] It should be noted that the second receive queue adjustment notifications sent by the control device to different terminal devices are different. Furthermore, as explained above regarding S1300, since the priority of the first receive queue is higher than that of the second receive queue, regardless of how the terminal device adjusts the indicated second receive queue, the priority of the adjusted second receive queue is always lower than that of the first receive queue. This ensures that the pose data sent by the head-mounted display device to the control device is processed with the highest priority.

[0091] Based on the above S1600, after the terminal device sends display data to the control device, the control device first schedules the display data in the second receiving queue with higher priority, that is, it schedules the display data sent by the terminal device with higher priority first. In this way, it can be ensured that the display data sent by the terminal device with higher priority is processed first, that is, it can be ensured that the display data sent by the terminal device corresponding to the display area that the wearer is looking at first is processed first.

[0092] In one example, as Figure 2Based on the example shown, if it is determined from the pose data that the wearer is currently looking at display area 1 and will subsequently look at display area 2, then the priority of terminal device 1 corresponding to display area 1 is determined to be higher than the priority of terminal device 2 corresponding to display area 2, and the priority of terminal device 2 corresponding to display area 2 is higher than the priority of terminal device 3 corresponding to display area 3. Further, if the second receiving queue indicated by the second priority configuration information sent to terminal devices 1, 2, and 3 respectively in S1300 is the Best Effort queue, and the first receiving queue indicated by the first priority configuration information sent to the head-mounted display device in S1200 is the Voice queue, then a notification indicating that the second receiving queue is the Video queue is sent to terminal device 1, a notification indicating that the second receiving queue remains the Best Effort queue is sent to terminal device 2 (or, if the second receiving queue corresponding to terminal device 2 does not change before and after the adjustment, this notification is not sent), and a notification indicating that the second receiving queue is the Background queue is sent to terminal device 2. Based on this, as... Figure 4 As shown, after receiving pose data from the head-mounted display device, the control device stores the pose data in the Voice queue. After receiving display data 1 from terminal device 1, the control device stores display data 1 in the Video queue. After receiving display data 2 from terminal device 2, the control device stores display data 2 in the Best Effort queue. After receiving display data 3 from terminal device 3, the control device stores display data 3 in the Background queue.

[0093] Through the embodiments shown in S1400-S1600 above, the data receiving method for the split-type head-mounted device provided in this application can dynamically adjust the priority between terminal devices, thereby ensuring that the display data sent by the terminal device with higher priority is processed first. That is, the display data sent by the terminal device corresponding to the display area first looked at by the wearer is processed first. This can avoid the problem that the display area first looked at by the wearer cannot display the processed display data in time, and can further improve the user experience.

[0094] In one embodiment of this application, the data receiving method for the split-type head-mounted device provided in this application further includes the following steps S1700 to S1900.

[0095] S1700: Receive pose data sent by the head-mounted display device based on the first priority configuration information.

[0096] S1800. Determine the target display area based on the pose data.

[0097] The target display area is the display area that does not fall within the gaze trend of the wearer of the head-mounted device.

[0098] In the embodiments of this application, since the wearer's gaze trend can be determined based on pose data, it is possible to determine which display areas belong to the gaze trend and which do not. In the embodiments of this application, the display areas that do not belong to the wearer's gaze trend are denoted as target areas.

[0099] For example, continuing with the example of displaying display area 1, display area 2, and display area 3 sequentially from left to right, when the wearer is currently looking at display area 2, if the pose data indicates that the wearer's head is moving from left to right, then the wearer's gaze trend is determined to be looking at display area 2, which means they are looking at display area 3. In this case, the target display area is determined to be display area 1.

[0100] It is understandable that the display area belonging to the gaze trend is the display area that the wearer will be looking at in the next moment. The display area not belonging to the gaze trend is the display area that the wearer will not be looking at in the next moment.

[0101] S1900: Send a notification to the corresponding terminal device in the target area to stop displaying data transmission.

[0102] The head-mounted display device displays the corresponding display area of ​​the terminal device.

[0103] In this embodiment, since the target display area is not part of the wearer's gaze pattern, the wearer will not gaze at the target display area, and therefore, the target display area may not display an image. Thus, to avoid meaningless data transmission of display data by the terminal device corresponding to the target display area, the control device sends a notification to the terminal device corresponding to the target display area to stop data transmission. Upon receiving this notification, the terminal device stops transmitting display data. This reduces the power consumption of the terminal device. Simultaneously, since the control device does not need to process invalid display data, its power consumption is also reduced.

[0104] Based on the embodiments shown in S1700 to S1900 above, there are situations where the wearer is also preparing to look at the target display area. For this situation, the data receiving method for the split-type head-mounted device provided in this application further includes the following S2000 and S2100.

[0105] S2000: Based on the pose data, determine whether the target display area belongs to the display area on the wearer's gaze trend.

[0106] S2100, If so, send a notification to the terminal device corresponding to the target display area to start display data transmission.

[0107] In this embodiment, pose data can be used to determine whether the target display area belongs to the display area along the wearer's gaze trend. Specifically, if pose data indicates that the wearer is about to view the target display area, then the target display area is determined to belong to the display area along the wearer's gaze trend. Otherwise, the target display area remains in a state where it does not belong to the display area along the wearer's gaze trend.

[0108] If the target display area falls within the area the wearer's gaze trend aligns with, then the terminal device corresponding to that target display area needs to send display data to the control device again. The control device then processes the data and sends it to the head-mounted display device, which displays it in the corresponding display area. At this point, the control device sends a notification to the terminal device corresponding to the target display area to begin data transmission. Upon receiving this notification, the terminal device sends display data to the control device again, allowing the control device to display the image in the target display area. At this time, the target display area switches to a non-target display area.

[0109] Based on any of the above embodiments, if there are no receiving queues of different priorities on the control device side, the data receiving method for the split head-mounted device provided in this application embodiment further includes the following S2200.

[0110] S2200: Establish a first receiving queue that matches the first priority configuration information and a second receiving queue that matches the second priority configuration information.

[0111] It should be noted that traditional receiving queue establishment methods can be used to establish the first receiving queue and the second receiving queue. This application embodiment does not limit the establishment method of the first receiving queue or the second receiving queue.

[0112] This application provides a data receiving device 500 for a split-type head-mounted device. The split-type head-mounted device includes a control device and a head-mounted display device. The device 500 is applied to the control device, such as... Figure 5 As shown, it includes:

[0113] Module 510 is used to establish a first communication connection with the head-mounted display device and a second communication connection with the terminal device.

[0114] The first sending module 520 is used to send first priority configuration information to the head-mounted display device based on the first communication connection;

[0115] The second sending module 530 is used to send second priority configuration information to the terminal device based on the second communication connection;

[0116] The first priority configuration information is used to instruct the control device to use a first receiving queue when receiving pose data sent by the head-mounted display device, and the second priority configuration information is used to instruct the control device to use a second receiving queue when receiving display data sent by the terminal device, wherein the priority of the first receiving queue is higher than the priority of the second receiving queue.

[0117] The data receiving device for the split-type head-mounted device provided in this application embodiment enables the control device to prioritize the processing of pose data over display data. This avoids dizziness caused by the head-mounted display device displaying an image that doesn't match the user's head posture, or lag issues caused by the head-mounted display device, which can result from pose data not being processed in a timely manner.

[0118] In one embodiment of this application, there are at least two terminal devices. The data receiving device 500 for the split-type head-mounted device provided in this embodiment further includes:

[0119] The first receiving module is used to receive pose data sent by the head-mounted display device based on the first priority configuration information;

[0120] The first determining module is used to determine the priority among the terminal devices based on the pose data;

[0121] The third sending module is used to send a second receiving queue adjustment notification to the terminal devices according to the priority among the terminal devices, so that the priority of the second receiving queue used by the control device when receiving display data sent by a terminal device with a higher priority is higher than the priority of the second receiving queue used when receiving display data sent by a terminal device with a lower priority.

[0122] In one embodiment of this application, the data receiving device 500 for the split-type head-mounted device provided in this application further includes:

[0123] The second receiving module is used to receive pose data sent by the head-mounted display device based on the first priority configuration information;

[0124] The second determining module is used to determine a target area based on the pose data, wherein the target area is a display area that does not belong to the wearer's gaze trend of the head-mounted device;

[0125] The fourth sending module is used to send a notification to the terminal device corresponding to the target area to stop displaying data transmission;

[0126] The head-mounted display device displays the display area corresponding to the terminal device.

[0127] In one embodiment of this application, the data receiving device 500 for the split-type head-mounted device provided in this application further includes:

[0128] The third determining module is used to determine, based on the pose data, whether the target display area belongs to the display area on the wearer's gaze trend;

[0129] The fifth sending module is used to send a notification to the terminal device corresponding to the target display area to start displaying data transmission when the condition is met.

[0130] In one embodiment of this application, the establishment module 510 is further configured to:

[0131] A first receiving queue matching the first priority configuration information and a second receiving queue matching the second priority configuration information are established.

[0132] In one embodiment of this application, the first priority configuration information is a first DSCP configuration information, and the second priority configuration information is a second DSCP configuration information. The receiving queue corresponding to the DSCP value indicated by the first DSCP configuration information is different from the receiving queue corresponding to the second DSCP value, and the DSCP value indicated by the first DSCP configuration information is greater than the DSCP value indicated by the second DSCP configuration information.

[0133] This application also provides a control device 600 for a split-type head-mounted device, the control device 600 including the data receiving device 500 for the split-type head-mounted device as described in the above-described device embodiments; or...

[0134] like Figure 6 As shown, the control device 600 includes a memory 610 and a processor 620. The memory 610 is used to store computer instructions, and the processor 620 is used to retrieve the computer instructions from the memory 610 to execute the data receiving method of the split head-mounted device as described in any of the above method embodiments.

[0135] This application embodiment also provides a split-type head-mounted device, including: the control device as described in the above device embodiment;

[0136] And a head-mounted display device, configured to receive first priority configuration information sent by the control device, encapsulate pose data to be sent to the control device according to the first priority configuration information, and send the encapsulated pose data to the control device.

[0137] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of the first aspects.

[0138] This application may be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this application.

[0139] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0140] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0141] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing state information from the computer-readable program instructions. These electronic circuits can execute the computer-readable program instructions to implement various aspects of this application.

[0142] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0143] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0144] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0145] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be well known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0146] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this application is defined by the appended claims.

Claims

1. A data receiving method for a split-type head-mounted device, characterized in that, The split-type head-mounted device includes a control device and a head-mounted display device, and the method is applied to the control device, including: Establish a first communication connection with the head-mounted display device and a second communication connection with the terminal device; Based on the first communication connection, send first priority configuration information to the head-mounted display device; Based on the second communication connection, send second priority configuration information to the terminal device; The first priority configuration information is used to instruct the control device to use a first receiving queue when receiving pose data sent by the head-mounted display device, and the second priority configuration information is used to instruct the control device to use a second receiving queue when receiving display data sent by the terminal device, wherein the priority of the first receiving queue is higher than the priority of the second receiving queue.

2. The method according to claim 1, characterized in that, The terminal devices are at least two, and the method further includes: Receive pose data sent by the head-mounted display device based on the first priority configuration information; Based on the pose data, the priority among the terminal devices is determined; Based on the priority among the terminal devices, a second receiving queue adjustment notification is sent to the terminal devices so that the priority of the second receiving queue used by the control device when receiving display data sent by a terminal device with a higher priority is higher than the priority of the second receiving queue used when receiving display data sent by a terminal device with a lower priority.

3. The method according to claim 1, characterized in that, The method further includes: Receive pose data sent by the head-mounted display device based on the first priority configuration information; Based on the pose data, a target area is determined, wherein the target area is a display area that does not belong to the gaze trend of the wearer of the head-mounted device; Send a notification to the terminal device corresponding to the target area to stop displaying data transmission; The head-mounted display device displays the display area corresponding to the terminal device.

4. The method according to claim 3, characterized in that, The method further includes: Based on the pose data, determine whether the target area belongs to the display area on the wearer's gaze trend; In this case, a notification to start displaying data transmission is sent to the terminal device corresponding to the target area.

5. The method according to claim 1, characterized in that, The method further includes: A first receiving queue matching the first priority configuration information and a second receiving queue matching the second priority configuration information are established.

6. The method according to claim 1, characterized in that, The first priority configuration information is the first DSCP configuration information, and the second priority configuration information is the second DSCP configuration information. The receiving queue corresponding to the DSCP value indicated by the first DSCP configuration information is different from the receiving queue corresponding to the DSCP value indicated by the second DSCP configuration information, and the DSCP value indicated by the first DSCP configuration information is greater than the DSCP value indicated by the second DSCP configuration information.

7. A data receiving device for a split-type head-mounted device, characterized in that, The split-type head-mounted device includes a control device and a head-mounted display device, and the device is applied to the control device, including: A module is established to establish a first communication connection with the head-mounted display device and a second communication connection with the terminal device. The first sending module is configured to send first priority configuration information to the head-mounted display device based on the first communication connection; The second sending module is used to send second priority configuration information to the terminal device based on the second communication connection; The first priority configuration information is used to instruct the control device to use a first receiving queue when receiving pose data sent by the head-mounted display device, and the second priority configuration information is used to instruct the control device to use a second receiving queue when receiving display data sent by the terminal device, wherein the priority of the first receiving queue is higher than the priority of the second receiving queue.

8. A control device for a split-type head-mounted device, characterized in that, The control device includes a data receiving device for the split-type head-mounted device as described in claim 7; or... The control device includes a memory and a processor, the memory being used to store computer instructions, and the processor being used to retrieve the computer instructions from the memory to execute the data receiving method of the split head-mounted device as described in any one of claims 1-6.

9. A split-type head-mounted device, characterized in that, include: The control device as described in claim 8; And a head-mounted display device, configured to receive first priority configuration information sent by the control device, encapsulate pose data to be sent to the control device according to the first priority configuration information, and send the encapsulated pose data to the control device.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method according to any one of claims 1-6.

Images