Data processing method and apparatus, and wearable device

Abstract

The embodiment of the invention provides a data processing method and device and wearable equipment, and the method comprises the steps: obtaining target information which is used for reflecting the spatial state of first equipment; the first equipment is wearable equipment; determining at least one target second device from a plurality of second devices based on the target information, the plurality of second devices and the first device being in the same space; and in the first mode, outputting first display data to the at least one target second device.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of electronic technology, and in particular to a data processing method, apparatus and wearable device. Background Art

[0002] With the increasing number of smart devices, multi-screen interoperability across devices is becoming more common, such as projecting the display content of a mobile phone to a computer, projecting the display content of a tablet to a TV, etc.

[0003] XR (Extended Reality) devices expand or enhance the user's real-life experience through digital technology, bringing users a more realistic and vivid interactive experience. The demand for projecting computer display content to XR devices and mobile phone display content to XR devices is also increasing. Summary of the invention

[0004] In view of this, the embodiments of the present application at least provide a data processing method, apparatus and wearable device.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a data processing method, comprising: obtaining target information, wherein the target information is used to reflect the spatial state of a first device; the first device is a wearable device; based on the target information, determining at least one target second device from multiple second devices, and the multiple second devices are in the same space as the first device; in a first mode, outputting first display data to the at least one target second device.

[0007] In a second aspect, an embodiment of the present application provides a data processing device, the device comprising:

[0008] A first acquisition module is used to obtain target information, where the target information is used to reflect the spatial state of a first device; the first device is a wearable device;

[0009] A first determining module, configured to determine at least one target second device from a plurality of second devices based on the target information, wherein the plurality of second devices and the first device are in the same space;

[0010] The output module is used to output the first display data to the at least one target second device in the first mode.

[0011] In a third aspect, an embodiment of the present application provides a wearable device, the device comprising:

[0012] A display, used for displaying first display data;

[0013] A controller is used to obtain target information, wherein the target information is used to reflect the spatial state of a first device; the first device is a wearable device; based on the target information, at least one target second device is determined from multiple second devices, and the multiple second devices are in the same space as the first device; in a first mode, the first display data is output to the at least one target second device.

[0014] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0016] Figure 1 A schematic diagram of an implementation flow of a data processing method provided in an embodiment of the present application;

[0017] Figure 2 A schematic diagram of the implementation process of a method for projecting a screen of a VR device across multiple terminal devices provided in an embodiment of the present application;

[0018] Figure 3 A schematic diagram of establishing a connection between a multi-terminal device and a VR device provided in an embodiment of the present application;

[0019] Figure 4 A schematic diagram of a VR device displaying screen contents of multiple terminal devices provided in an embodiment of the present application;

[0020] Figure 5 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0021] Figure 6 A hardware entity schematic diagram of a wearable device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0023] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0024] It should be pointed out that the terms "first\second\third" involved in the embodiments of the present application are merely to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0025] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of those skilled in the art in the field to which the embodiments of the present application belong. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0026] The method provided in the embodiment of the present application can be executed by an electronic device, wherein the electronic device can be a wearable device, a laptop, a tablet computer, a desktop computer, a set-top box, a mobile device (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable gaming device) and other types of terminals, and can also be implemented as a server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0027] The method provided in the embodiment of the present application can also be executed through a hardware device simulated by software (i.e., the data processing device described below), which can run and operate the data processing method without actual hardware, thereby providing users with functions and experiences similar to real hardware.

[0028] The present application embodiment provides a data processing method, such as Figure 1 As shown, the data processing method includes the following steps S101 to S103, wherein:

[0029] Step S101, obtaining target information, where the target information is used to reflect the spatial state of a first device; the first device is a wearable device;

[0030] Here, the spatial state of the first device may be the position, direction, movement trajectory and other attributes of the first device in three-dimensional space. The wearable device may be an XR device worn on the user's head, including but not limited to a virtual reality (VR) device, an augmented reality (AR) device, and a mixed reality (MR) device. This application is illustrated by taking a VR device as an example. It can be understood that a VR device is a head-mounted display device used to bring a user into a virtual three-dimensional environment. Since the main function of a VR device is to simulate and create a virtual environment, it has certain limitations in observing real-world devices. However, by combining technologies such as mixed reality and augmented reality, VR devices can be fully utilized to assist in observing and analyzing the real world. For example, virtual elements are combined with real environments through mixed reality technology, allowing users to see virtual objects and information in the real world. Virtual information is superimposed on a view of the real world through augmented reality technology.

[0031] In some embodiments, if the user wears a wearable device, the spatial state of the first device may be related to the position, direction, rotation angle, etc. of the user's head. In other words, when the user's head rotates, the spatial state of the first device also changes.

[0032] In some embodiments, the built-in sensors of the wearable device can be used to obtain the target information of the wearable device. The sensors can be gyroscopes, accelerometers, magnetometers, etc. These sensors can monitor the movement, rotation, acceleration and other states of the wearable device in real time and generate corresponding data. By processing these data, key information such as the spatial position and direction of the wearable device can be obtained.

[0033] In some embodiments, in addition to the built-in sensors, an external positioning system can also be used to obtain the target information of the wearable device. The external positioning system can be an optical positioning system, a laser positioning system, or an ultrasonic positioning system. These systems monitor the position and direction of the device in real time by arranging sensors or transmitters around the wearable device.

[0034] Step S102, determining at least one target second device from a plurality of second devices based on the target information, wherein the plurality of second devices are in the same space as the first device;

[0035] Here, the second device may be other devices in the same space as the first device, such as a mobile phone, a computer, a tablet, a smart home device, etc. It should be noted that the number of the second devices may be one or more, depending on the actual scenario.

[0036] The target second device may be one or more devices selected from a plurality of second devices. The selection of the target second device may be based on a variety of factors such as a spatial relationship with the first device, functional requirements, user preferences, etc.

[0037] The multiple second devices and the first device being in the same space may mean that the first device and the multiple second devices are all located in the same physical space or environment. The same space may be of any size, from a small room to a large public place, such as a room, an office, etc. Next, the same space is described as a room.

[0038] In some implementations, a device located in the direction of the first device is determined from the second devices based on the position, direction, movement trajectory, etc. of the first device in the room space. Specifically, if the second device includes a mobile phone, a computer, a tablet, or a smart home device, and the mobile phone, computer, or tablet is located in the direction of the first device, the mobile phone, computer, or tablet is determined as the target second device.

[0039] Step S103: in the first mode, outputting first display data to the at least one target second device.

[0040] Here, the first mode may be a mode automatically triggered according to a specific condition. In different modes, the first device may exhibit different behaviors or functions.

[0041] In some embodiments, after determining the target second device, the first device will output first display data to the target second device in a first mode. The first display data may include images, videos, text, and other multimedia content to project the first display data in the first device to the target second device.

[0042] In some embodiments, the first display data may be data being displayed on the screen of the first device, or data running on the backend of the first device. For example, by completely copying the data being displayed on the screen of the first device to the target second device, the target second device displays exactly the same content as the screen of the first device. By projecting the data running on the backend of the first device to the target second device, the content displayed on the target second device is different from the content on the screen of the first device, but the same as the content running on the backend of the first device. It should be noted that the output process of the first display data may involve steps such as encoding, transmission, and decoding of data to ensure that the data can be correctly displayed or played on the target device.

[0043] In the embodiment of the present application, on the one hand, the output device of the first display data is adjusted according to the spatial state of the first device, thereby providing a more natural and smooth interactive experience. For example, when the user wears the first device and moves around the house, the screen in the first device can be automatically projected to the target second device within the user's field of vision without the need for manual operation by the user; on the other hand, among multiple second devices, only the device related to the target first device will be activated to receive and display data. For example, if the user's first device is pointing to the TV in the living room, the computer in the bedroom will not be activated, thereby saving energy and computing resources.

[0044] In some embodiments, the implementation of step S102 “determining at least one target second device from a plurality of second devices based on the target information” may include the following steps S111 and S112, wherein:

[0045] Step S111: if the target information indicates that the state of the first device in space has changed, determine a third device that the first device is currently facing from a plurality of second devices;

[0046] Here, the change in the state of the first device in space may be that the sensor in the first device detects that the state of the first device in space, such as position, direction or movement trajectory, has changed. This change may be that the user's head turns, the user turns around, the user's body moves, etc. changes the orientation of the first device. Of course, the eye data of the wearing user can also be tracked directly here. The change in the eye data will cause the spatial state of the first device to change. For example, the eye data will also change as the head turns. In this case, the eye data can reflect the spatial state of the first device; the change in the eye data will also cause the spatial state of the first device to remain unchanged. For example, keep the head still and only turn the eyeballs. In this case, the eye data cannot accurately reflect the spatial state of the first device, but it still belongs to the scope of the target information defined in this application. For example, the embodiment of this application can also determine the device that the user needs to project the screen based on the eye data.

[0047] In some embodiments, when the state of the first device changes, the first device selects a third device that the first device is currently facing from multiple second devices according to the current orientation of the first device (i.e., the direction of its line of sight or facing). Specifically, if the second devices in the same space as the first device are computer 1, computer 2, tablet 1, and mobile phone 1, and it is detected that the devices with the same line of sight as the current first device are computer 1, tablet 1, and mobile phone 1, then computer 1, tablet 1, and mobile phone 1 are determined as the third device.

[0048] The implementation of the above steps requires that the user can see the external environment when wearing the device, so that he can decide which device in the environment to interact with, that is, to decide which device to face. Compared with VR devices, AR devices and MR devices can show the external environment to users more clearly. Although the external environment displayed by VR devices based on cameras may be biased, compared with AR devices, they provide users with a richer and more immersive experience. However, no matter which wearable device is used, its implementation process belongs to the protection scope of this application.

[0049] Step S112: Determine a device among the third devices that belongs to the fourth device as a target second device; the fourth device is a device that is allowed to communicate with the first device.

[0050] Here, the fourth device is a device that is allowed to communicate with the first device, that is, the fourth device may have established a connection relationship with the first device in some manner (such as pairing, authorization, network connection, etc.).

[0051] In some implementations, after determining the third device, it is necessary to further determine whether the third device is a device that is allowed to communicate with the first device. If computer 1 and tablet 1 among the third devices are devices that are allowed to communicate with the first device, computer 1 and tablet 1 are determined as target second devices. The target second device is a device that will interact with the first device next, and the target second device is both located within the orientation range of the first device and is allowed to communicate with the first device.

[0052] In an embodiment of the present application, on the one hand, by analyzing the state changes of the first device in space (such as orientation, position, etc.), it is possible to more accurately determine which second devices are within the orientation range of the first device, which helps to improve the accuracy and efficiency of interaction between devices; on the other hand, among multiple second devices, only those devices determined as target second devices will be activated to receive and display data, which helps to avoid unnecessary waste of resources.

[0053] In some embodiments, the data processing method may further include the following step S121, wherein:

[0054] Step S121, obtaining device information, and locating a fourth device in the space based on the device information;

[0055] Here, the device information may include a unique identifier of the device (such as a MAC address), signal strength, location data, etc. Such information is usually obtained through wireless communication technology, such as Bluetooth or Ultra Wide Band (UWB) positioning technology.

[0056] In some embodiments, the collected device information is used to determine which devices are located in a specific space, and in particular, to find those devices referred to as fourth devices. The fourth device may be one or more specific devices that have certain specific attributes or functions that make them selected as targets for interaction with the first device.

[0057] In some embodiments, the information transmitted by Bluetooth may include the name and address of the Bluetooth device, which can be used to identify and locate the device. Similarly, UWB positioning technology calculates the distance by measuring the flight time of the signal from the transmitter to the receiver, thereby achieving high-precision spatial positioning.

[0058] In some embodiments, the location data in the device information (such as Bluetooth signal strength indicator (RSSI) or UWB measured distance) is compared with a preset threshold or map data to determine whether the device is located in a specified space. If the device information meets specific conditions (such as being located in a specific area, the signal strength exceeds a certain threshold, etc.), the device is considered to be a fourth device.

[0059] The implementation of step S112 “determining a device belonging to the fourth device in the third device as the target second device” may include the following steps S122 and S123, wherein:

[0060] Step S122, obtaining appearance information of a third device;

[0061] Here, in order to more accurately determine which third devices belong to the fourth device, it is necessary to collect appearance information of the third devices, which may include visual features such as shape, size, and color of the device.

[0062] In some embodiments, the size and distance information of the third device can be obtained by using the point cloud information of the third device collected by the depth sensor on the first device; the RGB image of the third device is collected by the red, green, and blue (RGB) camera on the first device to perform object recognition, so as to obtain the color information and device category of the third device.

[0063] Step S123: determining a target second device based on the appearance information and the device information of the fourth device.

[0064] In some implementations, the appearance information of the third device and the device information of the fourth device are used for matching to determine which third devices actually belong to the fourth device. This matching can be to compare the appearance information of the third device with the device information of the fourth device to find devices with similar appearance and the same size. Once the match is successful, these third devices are determined as target second devices.

[0065] In the embodiment of the present application, when determining the target second device, by comprehensively considering the appearance information of the third device and the device information of the fourth device, the device that meets the specific conditions or standards can be screened out more accurately. This matching process is not only based on the physical properties of the device, but also considers the function and compatibility of the device, thereby ensuring the correctness of the selected device.

[0066] In some embodiments, the implementation of step S123 “determining the target second device based on the appearance information and the device information of the fourth device” may include the following steps S131 and S132, wherein:

[0067] Step S131, when the third device is in the first position, if there is a fourth device in the first position and the device information of the fourth device matches the appearance information, determining that the device in the first position is the target second device;

[0068] In some implementations, if the third device is in the first position, check whether there is a fourth device in the first position, if there is a fourth device in the first position, further compare whether the device information of the fourth device matches the appearance information of the third device, if the device information of the fourth device matches the appearance information of the third device, then determine that the fourth device in the first position is the target second device. Next, take the third device being a computer and the fourth device being computer 1 as an example for explanation.

[0069] In some embodiments, if the computer is in the first position, check whether there is computer 1 in the first position; if there is computer 1 in the first position, further compare the shape, size, and color of computer 1 with the shape, size, and color of the computer; if the shape, size, and color of computer 1 matches the shape, size, and color of the computer, then determine that computer 1 in the first position is the target second device.

[0070] Step S132, when the third device is in the first orientation and the second orientation, if there is a fourth device in the first orientation and the device information of the fourth device matches the appearance information, and there is a fourth device in the second orientation and the device information of the fourth device matches the appearance information, determine that the devices in the first orientation and the second orientation are the target second devices.

[0071] In some implementations, if the third device is in the first position and the second position at the same time, it is checked whether there is a fourth device in the first position and the second position respectively. If there is a fourth device in both the first position and the second position, the device information of the fourth devices in the two positions is compared with the appearance information of the third device to see if they match. If the appearance information of the fourth devices in the two positions matches that of the third device, it is determined that the fourth devices in the two positions are both the target second devices. Next, the third device is a mobile phone and the fourth device is mobile phone 1 and mobile phone 2 as an example for explanation.

[0072] In some embodiments, if the mobile phone is in the first orientation and the second orientation at the same time, check whether mobile phone 1 and mobile phone 2 exist in the first orientation respectively; if mobile phone 1 and mobile phone 2 exist in both the first orientation and the second orientation, compare the shape, size, and color of mobile phone 1 and mobile phone 2 in the two orientations with the shape, size, and color of the mobile phone respectively; if the shape, size, and color of mobile phone 1 and mobile phone 2 match the shape, size, and color of the mobile phone, then mobile phone 1 and mobile phone 2 in the two orientations are both target second devices.

[0073] In the embodiment of the present application, regardless of whether the third device is in a single orientation or multiple orientations, it can be matched based on the appearance information and device information, thereby achieving accurate identification of the target second device, effectively reducing the misidentification rate, and improving the accuracy of device identification.

[0074] In some embodiments, the data processing method may further include the following steps S141 and S142, wherein:

[0075] Step S141, in the second mode, obtaining second display data output by the at least one target second device;

[0076] Here, the second mode may be a mode automatically triggered according to a specific condition. In different modes, the first device may exhibit different behaviors or functions.

[0077] In some embodiments, the first device may obtain the second display data output by the target second device in the second mode, and the second display data may include multimedia content such as images, videos, and texts, so as to project the first display data in the target second device to the first device. It should be noted that the output process of the second display data may involve steps such as encoding, transmission, and decoding of the data to ensure that the data can be correctly displayed or played on the first device.

[0078] Step S142: display based on the second display data.

[0079] In some implementations, once the first device acquires the second display data, it will display the second display data, so that the user can see the display content from the target second device.

[0080] In the embodiment of the present application, the first device can display the second display data obtained from the target second device in real time, reducing the intermediate links of data conversion and transmission and improving the speed and efficiency of information display.

[0081] In some embodiments, the implementation of step S142 “displaying based on the second display data” may include the following steps S151 to S153, wherein:

[0082] Step S151, obtaining a spatial position relationship of at least one target second device;

[0083] In some embodiments, the first device may obtain information about the relative position or layout of at least one target second device in space. These spatial position relationships may be determined based on parameters such as physical distance, direction, angle, etc. For example, if the target second device is a plurality of display devices, the relative positions between the plurality of display devices and the first device are obtained respectively, such as which display device is to the left of the first device, which display device is to the right of the first device, or the angles between the plurality of display devices and the first device.

[0084] Step S152, based on the spatial position relationship, determining a display area of ​​second display data of at least one target second device, the display area having a region relationship matching the spatial position relationship;

[0085] In some embodiments, the first device allocates a display area to each target second device according to the spatial position relationship of at least one target second device. These display areas have a regional relationship on the first device that matches the spatial position relationship. For example, if the target second device A is located on the left side of the first device, the second display data of the target second device A may be allocated to the left area of ​​the first device; if the target second device B is located on the right side of the first device, the second display data of the target second device B may be allocated to the right area of ​​the first device.

[0086] In some embodiments, the display area of ​​the second display data has a depth of field, and corresponds to the longitudinal distance of the target second device from the first device in the environment. For example, if the target second device A is located on the left side of the first device and is close, then the second display data of the target second device A will not only be allocated to the left area of ​​the first device, but the display area will have a shallower depth of field to intuitively indicate that it is closer to the first device. If the target second device B is located on the right side of the first device and is far away, then the second display data of the target second device B will be allocated to the right area of ​​the first device, and the display area will have a deeper depth of field to intuitively indicate that it is far away from the first device. Furthermore, due to the difference in depth of field, the projection screen size may be different for devices of the same size. Therefore, when creating a display area, in addition to considering the depth of field, it is also necessary to consider the size of the area.

[0087] Step S153: display second display data of at least one target second device in a corresponding display area.

[0088] In some implementations, the first device displays the corresponding second display data in the display area allocated to each target second device, so that the user can intuitively see the output content of each target second device on the display device.

[0089] In the embodiments of the present application, on the one hand, by matching the second display data of the target second device with its actual position in space, the user can intuitively see the output content of each device on the first device, which simplifies the user's operation and improves the user experience; on the other hand, the matching of the display area and the spatial position relationship reduces the need for the user to frequently switch or move his or her line of sight to view the second display data output by different target second devices, so that the user can view the second display data of all target second devices on a unified display interface, thereby improving work efficiency.

[0090] In some embodiments, the data processing method may further include the following steps S161 to S163, wherein:

[0091] Step S161, obtaining a resolution of at least one target second device;

[0092] Here, resolution can be the number of pixels on the device's screen, usually expressed as horizontal pixels × vertical pixels (for example, 1920 × 1080). Resolution is crucial for determining the data transmission frequency band later, because the higher the resolution, the larger the amount of data required to be transmitted.

[0093] In some implementations, after the first device establishes a communication connection with at least one target second device, the at least one target second device acquires the resolution information by reporting to the first device through communication.

[0094] Step S162, determining a data transmission frequency band of at least one target second device based on the resolution, wherein the data transmission frequency band has a frequency band relationship matching the resolution;

[0095] In some embodiments, after obtaining the resolution of the target second device, the first device matches the resolution with the data transmission frequency band based on the resolution of at least one target second device, and determines the data transmission frequency band of at least one target second device to ensure that data can be transmitted at a sufficient rate while avoiding excessive consumption of bandwidth or causing data transmission delays.

[0096] The data transmission frequency band in this embodiment is constructed based on the wireless connection established between devices, and the transmission protocol used to construct the wireless connection mainly includes but is not limited to the current mainstream wireless screen projection protocols, such as Miracast, AirPlay, DLNA, Chromecast and WiDi. These protocols provide a stable and efficient basis for wireless data transmission between devices, so that multimedia content such as images, videos, texts, etc. can be smoothly transmitted between the first device and the target second device. Different from the method of locating the fourth device through Bluetooth connection mentioned above, the selection of wireless connection and data transmission frequency band in this embodiment depends more on the resolution requirements of the target second device and the transmission characteristics supported by the wireless screen projection protocol, thereby achieving more flexible and efficient data transmission.

[0097] It should be noted that the higher the resolution of the target second device, the larger the amount of data required to be transmitted. Therefore, a high-resolution device may require a higher data transmission rate to ensure image quality, and will therefore be allocated a frequency band with a higher bandwidth.

[0098] Step S163: obtaining second display data transmitted by at least one target second device via the data transmission frequency band.

[0099] In some embodiments, after determining the data transmission frequency band, the first device begins to receive the second display data transmitted by the target second device through the previously determined data transmission frequency band. These data may include images, videos, texts, etc., which are transmitted in the form of digital signals. The received second display data may need to be decoded, decoded and compressed, or other processing steps in order to be correctly displayed on the display interface of the first device.

[0100] In the embodiments of the present application, on the one hand, by determining the data transmission frequency band according to the resolution of the target second device, it is possible to ensure that the data transmission rate matches the display capability of the target second device, thereby avoiding resource waste or transmission delay problems caused by excessively high or low transmission rates; on the other hand, selecting an appropriate data transmission frequency band can more effectively utilize limited bandwidth resources, thereby supporting simultaneous connection and data transmission of more devices.

[0101] In some embodiments, the implementation of step S163 “displaying based on the second display data” may include the following steps S171 to S173, wherein:

[0102] Step S171, when there is an idle data transmission frequency band in the first device, obtaining second display data transmitted by a target second device through the idle data transmission frequency band;

[0103] In some embodiments, when the first device has an idle data transmission frequency band, this means that the first device is capable of receiving new data from the target second device without interfering with other data transmissions. In this case, the first device uses this idle data transmission frequency band to receive the second display data transmitted by the target second device.

[0104] Step S172: if there is no idle data transmission frequency band in the first device, determine a target data transmission frequency band from the data transmission frequency bands, where the target data transmission frequency band represents the highest frequency band in the data transmission frequency bands;

[0105] In some embodiments, if the first device has no idle data transmission frequency band, this means that all current frequency bands are in use, or the bandwidth of the first device has been fully utilized. In this case, the highest frequency band in the data transmission frequency band is selected to receive new display data.

[0106] Step S173, obtaining second display data transmitted by the target second device via the target data transmission frequency band.

[0107] In some implementations, after determining the target data transmission frequency band, the first device receives the second display data transmitted by the target second device through the target data transmission frequency band.

[0108] In an embodiment of the present application, on the one hand, when the first device has an idle data transmission frequency band, giving priority to using the idle data transmission frequency band can maximize resource utilization and avoid wasting frequency band resources; on the other hand, when the first device does not have an idle data transmission frequency band, selecting the highest frequency band among the data transmission frequency bands as the target data transmission frequency band can ensure the data transmission rate and quality, thereby optimizing the overall performance.

[0109] With the increasing number of smart devices, multi-screen interoperability across devices is becoming more common, such as projecting the display content of a mobile phone to a computer, projecting the display content of a tablet to a TV, etc.

[0110] XR (Extended Reality) devices expand or enhance the user's real-life experience through digital technology, bringing users a more realistic and vivid interactive experience. The demand for projecting computer display content to XR devices and mobile phone display content to XR devices is also increasing.

[0111] Based on this, the embodiment of the present application proposes a method for projecting a screen of a VR device across multiple terminal devices, such as Figure 2 As shown, the screen projection method may include the following steps S201 to S211, wherein:

[0112] Step S201, the VR device scans the Bluetooth of surrounding devices and establishes a connection;

[0113] In some embodiments, the VR device establishes a connection with the smart device by scanning the Bluetooth of the surrounding smart devices, such as Figure 3 As shown, an Android phone 301 , an iOS phone 302 , an Android tablet 303 , and a Windows laptop 304 are connected to the VR device 300 , respectively.

[0114] Step S202, the connected device sends its own information (Bluetooth RSSI, IP address, device name, device category, etc.) to the VR device;

[0115] In some embodiments, after the connection is successful, the smart device sends the information obtained after the Bluetooth connection is successful to the VR device, the information including Bluetooth signal strength (Received Signal Strength Indication, RSSI), IP address, device name, device category, system category (such as Android 11, iOS15, Windows10), display resolution (such as 1080P, 2K, which will affect the transmission load), etc., among which the Bluetooth signal strength can be used for positioning.

[0116] Step S203, the user wearing the VR device approaches the selected target device;

[0117] Step S204, the depth camera of the VR device scans the target device to determine the type, size and distance of the target device;

[0118] In some embodiments, with the help of a depth sensor (Time of Flight (TOF) camera or binocular camera) on a VR device, point cloud information of surrounding devices can be collected, and then the size and distance information of surrounding smart devices can be obtained; through the RGB camera on the VR device, RGB images of surrounding devices are collected for object recognition, and the device category in the current scanning range can be determined, such as whether the surrounding device is a mobile phone, a tablet, or a notebook, etc.

[0119] Step S205, the VR device searches for a device that matches the category and size among the connected devices and selects it as the selected device;

[0120] In some embodiments, when a user wearing a VR device approaches a selected device, the VR device searches for the nearest device in the connected devices that matches the category and size as the selected device. For example, a VR user aims at the laptops around him and searches for devices belonging to the laptop category from the connected smart devices. The device with the largest RSSI or the closest distance is the currently selected device. If a connection is established based on UWB, positioning and selecting a device can be achieved by simply pointing the VR device toward a terminal that supports UWB.

[0121] Step S206, determining whether other devices are connected to the VR device and are casting screens;

[0122] Here, if yes, that is, other devices are connected to the VR device and are projecting the screen, go to step S207; if no, that is, no other devices are connected to the VR device and are projecting the screen, go to step S208.

[0123] Step S207, the VR device establishes a new connection with the selected device in the high frequency band;

[0124] In some embodiments, high-resolution, high-transmission-load content is transmitted over a high-speed connection established in the Wifi 6GHz band.

[0125] Step S208, determining whether there is an available Wifi frequency band;

[0126] Here, if yes, that is, there is an available Wifi frequency band, the process goes to step S209; if no, that is, there is no available Wifi frequency band, the process goes to step S210.

[0127] Step S209, the VR device establishes a connection with the selected device in an available Wifi frequency band;

[0128] In some embodiments, when new content is accessed, in order to avoid interfering with the previously established link (Wifi 6GHz band link), the new link can be switched to a clean Wifi 5GHz band to establish a connection with the smart device under this band; similarly, when a new low transmission load comes in, it can be switched to a clean Wifi 2.4GHz band.

[0129] Step S210, the VR device establishes a shared connection with the selected device in the high frequency band;

[0130] In some embodiments, when the number of connected devices exceeds 3, it is necessary to reuse the frequency bands that have established communication links to try to keep the load of each frequency band balanced. For example, if a new mobile device is added, if the current 6GHz load of the notebook is acceptable, then mobile phone 2 can be placed in the 6GHz frequency band, and so on, tablet 2 is placed in the 5GHz communication link, and mobile phone 3 is placed in the 2.4GHz communication link.

[0131] Step S211, projecting the screen content of the selected device onto the panel of the VR device.

[0132] In some embodiments, Figure 4 As shown, an Android phone 301, an iOS phone 302, an Android tablet 303, and a Windows laptop 304 respectively project their screens to a VR device 300, and the projection content 401 of the Android phone, the projection content 402 of the iOS phone, the projection content 403 of the Android tablet, and the projection content 404 of the Windows laptop are respectively displayed in different display areas of the VR device.

[0133] In some embodiments, when casting screens between devices of different systems, since most VR devices are currently Android systems, and Miracast is the mainstream screen casting protocol on Android and Windows platforms, if you are casting screens from an Android or Windows phone, tablet or computer to a VR device, you can select the corresponding channel and frequency band for communication in Miracast's Auto group owner (AGO) mode. When casting screens from an iOS smart device to a VR device, you can enable Airplay support in the VR device (Android devices can support the Airplay protocol by installing a client), and then the iOS device can cast screens to the VR device via Airplay.

[0134] In some embodiments, there are two ways to operate the content of each projection device in the VR virtual space: (1) Use the controller of the VR device to operate the content in the VR virtual scene. The coordinates in the three-dimensional space of the VR device can be converted to the coordinates in the two-dimensional space of the projected content, and then transmitted to the input module of each device terminal to achieve simulated input. (2) Input is performed through the input peripherals of each smart device, such as a mouse, Bluetooth keyboard, etc. If a laptop is connected to the VR device, the keyboard and touchpad of the laptop can be used directly for input interaction. Similarly, if it is a mobile phone, the touch screen of the mobile phone can still be used for interaction.

[0135] Compared with the prior art, the embodiments of the present application have the following advantages:

[0136] 1. There is no account restriction. Users do not need to log in to the same account between different devices to realize content flow.

[0137] 2. No platform restrictions, supports Android, iOS devices, mobile phones, tablets, computers and other devices to project screens to VR devices at the same time, and supports interaction.

[0138] 3. The number and types of supported terminals are greater, and screen projection from more than one device to a VR device is no longer limited.

[0139] Based on the above embodiments, the present application provides a data processing device, such as Figure 5 As shown, the data processing device 500 includes:

[0140] A first acquisition module 510 is used to obtain target information, where the target information is used to reflect the spatial state of a first device; the first device is a wearable device;

[0141] A first determination module 520, configured to determine at least one target second device from a plurality of second devices based on the target information, wherein the plurality of second devices are in the same space as the first device;

[0142] The output module 530 is configured to output the first display data to the at least one target second device in the first mode.

[0143] In some embodiments, the first determination module 520 includes: a first determination unit, used to determine, from multiple second devices, a third device to which the first device is currently heading if the target information represents that the state of the first device in space has changed; a second determination unit, used to determine a device among the third devices that belongs to a fourth device as a target second device; and the fourth device is a device that is allowed to communicate with the first device.

[0144] In some embodiments, the data processing device 500 also includes: a second acquisition module, used to obtain device information and locate a fourth device in the space based on the device information; the second determination unit includes: a first acquisition unit, used to obtain appearance information of a third device; a third determination unit, used to determine the target second device based on the appearance information and the device information of the fourth device.

[0145] In some embodiments, the third determination unit includes: a first determination subunit, used to determine that the device in the first orientation is the target second device if, when the third device is in the first orientation, there is a fourth device in the first orientation and the device information of the fourth device matches the appearance information; a second determination subunit, used to determine that the devices in the first orientation and the second orientation are the target second devices if, when the third device is in the first orientation and the second orientation, there is a fourth device in the first orientation and the device information of the fourth device matches the appearance information, and there is a fourth device in the second orientation and the device information of the fourth device matches the appearance information.

[0146] In some embodiments, the data processing apparatus 500 further includes: a third acquisition module, configured to obtain, in the second mode, second display data output by the at least one target second device; and a display module, configured to display based on the second display data.

[0147] In some embodiments, the display module includes: a second acquisition unit, used to obtain the spatial position relationship of at least one target second device; a fourth determination unit, used to determine the display area of ​​the second display data of at least one target second device based on the spatial position relationship, and the display area has a regional relationship matching the spatial position relationship; a display unit, used to display the second display data of at least one target second device in the corresponding display area.

[0148] In some embodiments, the data processing device 500 further includes: a fourth acquisition module, used to obtain the resolution of at least one target second device; a second determination module, used to determine the data transmission frequency band of at least one target second device based on the resolution, the data transmission frequency band having a frequency band relationship matching the resolution; and a fifth acquisition module, used to obtain second display data transmitted by at least one target second device through the data transmission frequency band.

[0149] In some embodiments, the fifth acquisition module includes: a third acquisition unit, used to obtain the second display data transmitted by the target second device through the idle data transmission frequency band when there is an idle data transmission frequency band in the first device; a fifth determination unit, used to determine the target data transmission frequency band from the data transmission frequency bands when there is no idle data transmission frequency band in the first device, the target data transmission frequency band representing the highest frequency band among the data transmission frequency bands; and a fourth acquisition unit, used to obtain the second display data transmitted by the target second device through the target data transmission frequency band.

[0150] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0151] It should be noted that in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0152] The present application embodiment provides a wearable device, such as Figure 6 As shown, the wearable device 600 includes:

[0153] Display 610, used to display first display data;

[0154] Controller 620 is used to obtain target information, where the target information is used to reflect the spatial state of a first device; the first device is a wearable device; based on the target information, at least one target second device is determined from multiple second devices, and the multiple second devices are in the same space as the first device; in a first mode, the first display data is output to the at least one target second device.

[0155] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and the computer program implements the above method when executed by a processor. The computer-readable storage medium may be transient or non-transient.

[0156] The embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be implemented specifically by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.

[0157] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0158] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the serial number of each step / process mentioned above does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The serial numbers of the embodiments of the present application mentioned above are for description only and do not represent the advantages and disadvantages of the embodiments.

[0159] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0160] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0161] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0162] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0163] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0164] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0165] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A data processing method, the method comprising: Obtaining target information, where the target information is used to reflect the spatial state of the first device; The first device is a wearable device; Based on the target information, determining at least one target second device from a plurality of second devices, where the plurality of second devices are in the same space as the first device; In the first mode, first display data is output to the at least one target second device.

2. The method according to claim 1, wherein determining at least one target second device from a plurality of second devices based on the target information comprises: If the target information indicates that the state of the first device in space has changed, determining a third device to which the first device is currently directed from a plurality of second devices; A device belonging to the fourth device among the third devices is determined as a target second device; the fourth device is a device that is allowed to communicate with the first device.

3. The method according to claim 2, further comprising: obtaining device information, and locating a fourth device in the space based on the device information; The step of determining a device belonging to the fourth device among the third devices as a target second device includes: Obtaining appearance information of a third device; Based on the appearance information and the device information of the fourth device, a target second device is determined.

4. The method according to claim 3, wherein determining the target second device based on the appearance information and the device information of the fourth device comprises: In the case where the third device is in the first position, if there is a fourth device in the first position and the device information of the fourth device matches the appearance information, determining that the device in the first position is the target second device; When the third device is in the first orientation and the second orientation, if there is a fourth device in the first orientation and the device information of the fourth device matches the appearance information, and there is a fourth device in the second orientation and the device information of the fourth device matches the appearance information, the devices in the first orientation and the second orientation are determined to be the target second devices.

5. The method according to any one of claims 1 to 4, further comprising: In the second mode, obtaining second display data output by the at least one target second device; Display is performed based on the second display data.

6. The method according to claim 5, wherein displaying based on the second display data comprises: Obtaining a spatial position relationship of at least one target second device; Based on the spatial position relationship, determining a display area of ​​second display data of at least one target second device, the display area having a region relationship matching the spatial position relationship; The second display data of at least one target second device is displayed in the corresponding display area.

7. The method according to claim 5, further comprising: obtaining a resolution of at least one target second device; Based on the resolution, determining a data transmission frequency band of at least one target second device, the data transmission frequency band having a frequency band relationship matching the resolution; Second display data transmitted by at least one target second device through the data transmission frequency band is obtained.

8. The method according to claim 7, wherein obtaining the second display data transmitted by at least one target second device through the data transmission frequency band comprises: In a case where the first device has an idle data transmission frequency band, obtaining second display data transmitted by a target second device through the idle data transmission frequency band; In a case where there is no idle data transmission frequency band in the first device, determining a target data transmission frequency band from the data transmission frequency bands, wherein the target data transmission frequency band represents a highest frequency band in the data transmission frequency bands; The second display data transmitted by the target second device through the target data transmission frequency band is obtained.

9. A data processing device, comprising: A first acquisition module, used to obtain target information, where the target information is used to reflect the spatial state of the first device; The first device is a wearable device; A first determining module, configured to determine at least one target second device from a plurality of second devices based on the target information, wherein the plurality of second devices and the first device are in the same space; The output module is used to output the first display data to the at least one target second device in the first mode.

10. A wearable device, comprising: A display, used for displaying first display data; A controller, configured to obtain target information, wherein the target information is used to reflect a spatial state of the first device; The first device is a wearable device; Based on the target information, determining at least one target second device from a plurality of second devices, where the plurality of second devices are in the same space as the first device; In the first mode, the first display data is output to the at least one target second device.

Images