Data processing method and device in mobile scenario

Abstract

A data processing method and device in a mobile scenario and a readable storage medium, which are used to perform calibration processing for cursor drift that may occur when using a pointing device in a mobile scenario, thereby ensuring the user's interactive experience. The method comprises: obtaining first data and second data of a first device (200) (901A); obtaining third data and fourth data of a second device (300) (901B); on the basis of the second data and the fourth data, obtaining first updated data (903), the first updated data being used to reflect the difference in inertial offset data of the second device (300) relative to the first device (200); on the basis of the first data and the third data, obtaining second updated data (904), the second updated data being used to reflect a measured actual relative spatial relationship of the second device (300) relative to the first device (200); and performing fusion processing on the second updated data on the basis of the first updated data, so as to obtain relative spatial data (905), the relative spatial data being used to reflect a target relative spatial relationship of the second device (300) relative to the first device (200).

Description

Data processing method and device in mobile scenario

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on March 26, 2024, with application number 202410356522.9 and application name "A data processing method and device in a mobile scenario", the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of terminal technology, and in particular to a data processing method and device in a mobile scenario. Background Art

[0004] As cars become increasingly intelligent, users can access more functions through in-car screens, such as online navigation, music playback, and video playback. This increased intelligence has also led to a gradual increase in the size of in-car screens. However, in-car screens are typically placed in the front row of the car. Even if a screen is installed in the middle or back row, the user is still far away from the screen during the ride, making it inconvenient for users to interact with the in-car screen.

[0005] A pointing remote control interacts with the ultra-wideband (UWB) on the vehicle's screen to calculate its posture and position relative to the screen. This determines the specific location of the vehicle's screen and displays the cursor at that location. The user experience of a pointing remote control is similar to that of an infrared laser pointer, and its interaction method is more in line with user habits. Therefore, a pointing remote control can enhance user comfort in interacting with the vehicle's screen.

[0006] However, in a mobile vehicle such as a car, if the mobile vehicle turns or bumps, the cursor may drift when using a pointing remote control. Summary of the Invention

[0007] The present application provides a data processing method and device in a mobile scenario, which is used to calibrate the cursor drift that may occur when using a pointing remote control in a mobile scenario, thereby ensuring the user's interactive experience.

[0008] The first aspect provides a data processing method in a mobile scenario. In this method, first data and second data of a first device are obtained; third data and fourth data of a second device are obtained; first update data is obtained based on the second data and the fourth data; the first update data is used to reflect the difference in inertial offset data of the second device relative to the first device; second update data is obtained based on the first data and the third data; the second update data is used to reflect the measured actual relative spatial relationship of the second device relative to the first device; the second update data is fused based on the first update data to obtain relative spatial data, and the relative spatial data is used to reflect the target relative spatial relationship of the second device relative to the first device.

[0009] The first data can be understood as the absolute spatial information of the first device measured by the first device; the second data can be understood as the relative spatial information of the first device measured by the first device, that is, the change in spatial information measured by the first device at the current measurement time compared to the spatial information at the previous measurement time. Similarly, the third data can be understood as the absolute spatial information of the second device measured by the second device; the fourth data can be understood as the relative spatial information of the second device measured by the second device, that is, the change in spatial information measured by the second device at the current measurement time compared to the spatial information at the previous measurement time.

[0010] The actual relative spatial relationship can be used to represent the relative spatial relationship between the first device and the second device, determined based on the absolute spatial information of the first device and the absolute spatial information of the second device. It will be appreciated that changes in the actual relative spatial relationship are not only caused by user operations on the second device, but also by inertial offset caused by the movement of the mobile carrier.

[0011] The target relative spatial relationship can be used to represent the relative spatial relationship between the first and second devices calculated by offsetting the change in the relative spatial relationship between the first and second devices, which is obtained based on the relative spatial information of the first and second devices. It should be understood that the target relative spatial relationship cannot be directly determined through measurement and must be calculated based on the actual relative spatial relationship and the change in the relative spatial relationship.

[0012] In this method, absolute and relative spatial information are measured by the first and second devices, respectively. Changes in the relative spatial relationship caused by inertial offset can be offset through calculation, thereby capturing changes in the relative spatial relationship caused by user manipulation. This method, based on the calculated relative spatial relationship of the target, allows pointing interactions between the first and second devices to be performed, reducing the probability and extent of abnormal cursor offset, thereby ensuring a consistent user experience for multi-device pointing interactions in mobile scenarios.

[0013] In one possible scenario, the first update data includes first posture update data and first position update data; the second update data includes second posture update data and second position update data. The relative spatial data includes third posture update data and third position update data. Fusion processing is performed on the second update data based on the first update data to obtain the relative spatial data, including: obtaining third posture update data based on the difference between the first posture update data and the second posture update data; and obtaining third position update data based on the difference between the first position update data and the second position update data.

[0014] In this scenario, based on the measured change in the relative spatial relationship between the first device and the second device, the actual relative spatial relationship is offset, so that the target relative spatial relationship between the first device and the second device determined by the user operation can be obtained more accurately, thereby improving the stability of the cursor display.

[0015] In one possible scenario, the second data includes first posture data and first position data; the fourth data includes second posture data and second position data. Obtaining first update data based on the second data and the fourth data includes: obtaining first posture update data based on the first posture data and the second posture data, the first posture update data being used to reflect a change in the posture of the second device relative to the first device due to inertia; and obtaining first position update data based on the first position data and the second position data, the first position update data being used to reflect a change in the position of the second device relative to the first device due to inertia.

[0016] In this scenario, by each device measuring its own relative spatial information, the change in the relative spatial relationship between the two devices can be accurately determined, thereby facilitating a more accurate calculation of the target relative spatial relationship between the two devices.

[0017] In one possible scenario, the first data includes third posture data and third position data; the third data includes fourth posture data and fourth position data. Obtaining second update data based on the first data and the third data includes: obtaining second posture update data based on the third posture data and the fourth posture data, the second posture update data being used to reflect actual posture changes of the second device relative to the first device; and obtaining second position update data based on the third position data and the fourth position data, the second position update data being used to reflect actual position changes of the second device relative to the first device.

[0018] In this scenario, by each device measuring its own absolute spatial information, the actual relative spatial relationship between the two devices can be accurately determined, which can facilitate a more accurate calculation of the target relative spatial relationship between the two devices.

[0019] In a possible scenario, the first device is a fixed device relative to the mobile carrier, and the second device is a mobile device relative to the mobile carrier.

[0020] In this scenario, when the user uses the second device to perform pointing operations on the first device, by calculating the changes in the spatial information of the second device caused by the user operation, the accuracy of multi-device pointing interaction can be guaranteed, and the user's experience of multi-device pointing interaction can be guaranteed.

[0021] In a possible scenario, the target relative spatial relationship is used to reflect the movement of the second device relative to the first device, where the movement is caused by a user operation.

[0022] In one possible scenario, the method further includes: determining a position of a display cursor of a target screen based on the relative spatial data; the target screen is the screen to which the second device is pointing. Optionally, the target screen is the first device. Alternatively, the target screen is a device different from the first device.

[0023] In this scenario, displaying the cursor based on the calculated relative spatial data can reduce the probability of abnormal cursor offset and the degree of cursor offset.

[0024] In a possible scenario, before obtaining the first data and the second data of the first device, the method further includes: performing time synchronization between the first device and the second device.

[0025] In this scenario, by performing time synchronization between the first device and the second device, the synchronization of measurements performed by the first device and the second device can be guaranteed, thereby ensuring the accuracy of calculation of the relative spatial relationship of the targets.

[0026] In one possible scenario, the first data is measured by the UWB module of the first device, and the second data is measured by the inertial measurement unit (IMU) chip system of the first device; the third data is measured by the UWB module of the second device, and the fourth data is measured by the IMU chip system of the second device.

[0027] In this scenario, by integrating the UWB module and IMU chip system in the first device, and also integrating the UWB module and IMU chip system in the second device, the relative spatial relationship of the target can be calculated, the probability of abnormal cursor offset can be reduced, and the degree of cursor offset can be reduced, thereby ensuring the user experience of multi-device pointing interaction in mobile scenarios.

[0028] A second aspect provides a data processing system in a mobile scenario, which may include a first device and a second device. The system is used to implement the method described in the first aspect and its possible scenarios.

[0029] In one possible scenario, a first device obtains first data and second data from the first device; a second device obtains third data and fourth data from the second device. Furthermore, the first device sends the first data and second data to the second device. Based on this, the second device obtains first update data based on the second data and the fourth data; the first update data is used to reflect the difference in inertial offset data of the second device relative to the first device; the second device obtains second update data based on the first data and the third data; the second update data is used to reflect the measured actual relative spatial relationship of the second device relative to the first device; the second device fuses the second update data based on the first update data to obtain relative spatial data, which is used to reflect the target relative spatial relationship of the second device relative to the first device.

[0030] In another possible scenario, the first device obtains the first data and second data of the first device; the second device obtains the third data and fourth data of the second device. Furthermore, the second device sends the third data and fourth data to the first device. Based on this, the first device obtains first update data based on the second data and the fourth data; the first update data is used to reflect the difference in inertial offset data of the second device relative to the first device; the first device obtains second update data based on the first data and the third data; the second update data is used to reflect the measured actual relative spatial relationship of the second device relative to the first device; the first device fuses the second update data based on the first update data to obtain relative spatial data, and the relative spatial data is used to reflect the target relative spatial relationship of the second device relative to the first device.

[0031] In another possible scenario, the system may also include a third device. The first device obtains the first data and the second data of the first device; the second device obtains the third data and the fourth data of the second device. In addition, the first device sends the first data and the second data to the third device, and the second device sends the third data and the fourth data to the third device. Based on this, the third device obtains the first update data according to the second data and the fourth data; the first update data is used to reflect the difference in inertial offset data of the second device relative to the first device; the third device obtains the second update data according to the first data and the third data; the second update data is used to reflect the measured actual relative spatial relationship of the second device relative to the first device; the third device fuses the second update data based on the first update data to obtain relative spatial data, and the relative spatial data is used to reflect the target relative spatial relationship of the second device relative to the first device. Optionally, the third device can be other devices or a cloud server.

[0032] Optionally, the system may also include one or more vehicle-mounted screens.

[0033] A third aspect provides a device comprising multiple functional modules; the multiple functional modules interact with each other to implement the method described in the first aspect and its possible scenarios. The multiple functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the multiple functional modules can be arbitrarily combined or divided based on the specific implementation.

[0034] The fourth aspect provides a device comprising at least one processor and at least one memory, wherein the at least one memory stores a program, and when the device runs the program, the at least one processor executes the method described in the first aspect and its possible scenarios. Optionally, the device can be understood as a first device, and the first device can receive third data and fourth data from a second device. Alternatively, the device can also be understood as a second device, and the second device can receive first data and second data from the first device. Alternatively, the device can also be understood as a third device, and the third device can receive first data and second data from the first device, and receive third data and fourth data from the second device.

[0035] The fifth aspect further provides a program product, which, when run on a device, enables the device to execute the method described in any one of the above aspects and its possible scenarios.

[0036] The sixth aspect also provides a readable storage medium, in which a program is stored. When the program is executed by a device, the device executes the method described in any one of the above aspects and its possible scenarios.

[0037] The seventh aspect also provides a chip, which is used to read the program stored in the memory and execute the method described in any of the above aspects and its possible scenarios.

[0038] An eighth aspect further provides a chip system, comprising a processor configured to support a device in implementing the method described in any of the above aspects and its possible scenarios. In one possible scenario, the chip system further comprises a memory configured to store the necessary programs and data. The chip system may be composed of a chip or may include a chip and other discrete components. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] FIG1 is a schematic diagram showing an application scenario to which an embodiment of the present application is applicable;

[0040] FIG2 is a schematic diagram of a hardware structure of a first device 200 provided in an embodiment of the present application;

[0041] FIG3 is a schematic diagram of a hardware structure of a second device 300 provided in an embodiment of the present application;

[0042] FIG4A is a schematic diagram of establishing a UWB coordinate system of the first device 200 according to an embodiment of the present application;

[0043] FIG4B is a second schematic diagram of establishing a UWB coordinate system of the first device 200 provided in an embodiment of the present application;

[0044] FIG5 is a schematic diagram of establishing a geographic coordinate system according to an embodiment of the present application;

[0045] FIG6 is a schematic diagram of establishing a remote control coordinate system according to an embodiment of the present application;

[0046] FIG7A is a schematic diagram of establishing a screen coordinate system according to an embodiment of the present application;

[0047] FIG7B is a schematic diagram of establishing a display coordinate system according to an embodiment of the present application;

[0048] FIG8 is a schematic diagram of the attitude angles of the b-frame and the e-frame provided in an embodiment of the present application;

[0049] FIG9 is a flow chart of a data processing method in a mobile scenario provided by an embodiment of the present application;

[0050] FIG10 is a schematic diagram of a time synchronization process of a data processing method applied to a mobile scenario according to an embodiment of the present application;

[0051] FIG11 is a schematic diagram of the azimuth angle of a data processing method in a mobile scenario provided by an embodiment of the present application;

[0052] FIG12 is another azimuth diagram of a data processing method in a mobile scenario provided by an embodiment of the present application;

[0053] FIG13 is a schematic diagram of an interface of a data processing method in a mobile scenario provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and examples.

[0055] The method provided in the embodiments of the present application can be applied to interactive scenarios involving multiple devices pointing in mobile scenarios, such as scenarios involving riding in a car or riding in an airplane. The following embodiments of the present application use the mobile scenario of riding in a car as an example for description. For other mobile scenarios, please refer to the scenario of riding in a car. It is understood that the application scenarios of the embodiments of the present application can include scenarios involving multiple devices interacting with each other on a mobile carrier, such as using a remote control to point at an in-car screen in a mobile car scenario.

[0056] For example, FIG1 shows a schematic diagram of an application scenario applicable to an embodiment of the present application. Referring to 101 in FIG1 , the application scenario may include a vehicle-mounted screen 10 and a remote control 20. It should be understood that 101 in FIG1 is only an example in which the vehicle-mounted screen 10 is the main control screen included in the car, and the embodiment of the present application is not limited to this. For example, the vehicle-mounted screen 10 may also be a co-pilot screen, a middle row screen, or a rear row screen. In addition, 101 in FIG1 is only an example of one vehicle-mounted screen, and more vehicle-mounted screens may be included in the present application, and there is no specific limitation on this.

[0057] In some embodiments, one or more UWB modules may be installed on the vehicle screen 10 and the remote control 20. For example, one UWB module may be installed on the vehicle screen 10 shown as 101 in FIG1 , and one UWB module may be installed on the remote control 20. The UWB module may be used to provide UWB communication, distance measurement, and other capabilities for the vehicle screen, and generally includes a UWB chip and antenna.

[0058] Then, the UWB coordinate system of the vehicle-mounted screen 10 is constructed according to the position of the UWB module on the vehicle-mounted screen 10, and the UWB coordinate system of the remote control 20 can also be constructed. Based on the UWB coordinate system of the vehicle-mounted screen 10, the interactive calculation between the vehicle-mounted screen 10 and the remote control 20 is performed, so that the relative posture information and relative position information of the remote control 20 relative to the vehicle-mounted screen 10 can be obtained. Then, based on the relative posture information and relative position information of the remote control 20 relative to each vehicle-mounted screen 10 (only one vehicle-mounted screen 10 is shown in Figure 1), it can be determined which vehicle-mounted screen 10 the remote control 20 is pointing to, and the specific position on the pointed vehicle-mounted screen 10 can be determined, so that the cursor of the remote control 20 can be displayed at the corresponding position of the corresponding vehicle-mounted screen 10. It can be understood that the user experience of the remote control 20 is similar to that of an infrared laser pen, and its interactive method is more in line with the user's usage habits.

[0059] In addition, the remote controller 20 is generally equipped with one or more IMU chip systems, which can sense the shaking of the human hand. The IMU chip system may include a gyroscope and an accelerometer, or the IMU chip system may include a gyroscope, an accelerometer, and a magnetometer.

[0060] However, in a mobile scenario, when the mobile carrier moves or rotates, the cursor may drift due to factors such as inertia. For example, when a car is driving and encounters a turn or bump, the cursor of the remote control 20 may drift on the in-vehicle screen 10.

[0061] In view of this, an embodiment of the present application provides a data processing method and device in a mobile scenario. In this method, a first device is provided on a mobile carrier, and the first device may include an IMU chip system and a UWB module; wherein, the first device can be understood as a fixed device relative to the mobile carrier and the vehicle-mounted screen. The second device also includes an IMU chip system and a UWB module; wherein, the second device can be understood as a mobile device relative to the mobile carrier and the vehicle-mounted screen, such as a remote control. In this way, through the operation of the IMU chip system and UWB module of each of the first device and the second device, the actual relative spatial relationship measured between the two devices can be taken into account. The change in the relative spatial relationship between the two devices caused by inertia can be calculated, and the change in the target relative spatial relationship between the two devices caused by user operation can be obtained, thereby determining the display result of the cursor based on the target relative spatial relationship, which can reduce the probability of cursor drift and improve the user experience of multi-device pointing interaction in mobile scenarios.

[0062] Among them, the first device can be a vehicle-mounted screen, and the IMU chip system and the UWB module can be integrated in the vehicle-mounted screen; or, the first device can also be a separate device independent of the vehicle-mounted screen, and the IMU chip system and the UWB module are integrated in the separate device. In this case, the first device can be installed relative to the vehicle-mounted screen, for example, next to the vehicle-mounted screen, on the back panel, etc.

[0063] It should be understood that the data processing method in a mobile scenario provided by the embodiment of the present application can be applied to electronic devices that need to be controlled by a remote controller in a mobile scenario, such as a car screen, a car smart device, or other smart devices that may appear in mobile scenarios such as car scenarios in the future. In addition, exemplary embodiments of electronic devices include but are not limited to devices equipped with Or devices with other operating systems.

[0064] The following uses the first device 200 as an example of a separate device independent of the vehicle-mounted screen to introduce the structure of the device to which the embodiments of the present application are applicable.

[0065] As shown in Figure 2 , the first device 200 may include a processor 210, an internal memory 220, a power management module 240, a power supply 241, a communication module 250, a UWB module 260, an IMU chip system 270, etc. Alternatively, although not shown in Figure 2 , the first device 200 may further include an external memory interface, a touch panel, etc. It will be appreciated that when the first device 200 is an in-vehicle screen, the in-vehicle screen may further include an audio module, a speaker, buttons, a display, etc.

[0066] The processor 210 may include one or more processing units. For example, the processor 210 may include an application processor (AP), a modem processor, a controller, and memory. Optionally, depending on the different functions of the first device 200, the processor 210 may also include a graphics processing unit (GPU), an image signal processor (ISP), a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors. The controller may be the nerve center and command center of the first device 200. The controller may generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. The processor 210 may also be provided with memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a high-speed cache memory. This memory may store instructions or data that have just been used or are recycled by the processor 210. If the processor 210 needs to use the instruction or data again, it can directly call it from the memory. Repeated access is avoided, the waiting time of the processor 210 is reduced, and the efficiency of the system is improved. In the embodiment of the present application, the processor 210 can be used to process the communication information or measurement information output by the UWB module 260, and can also be used to calculate the position information and posture information of the vehicle screen 10, and can also be used to obtain the position information and posture information of the remote control 20. It can also be used to perform fusion processing of the measurement data of the UWB module 260 and the measurement data of the IMU chip system 270, and calculate the relative position confidence and relative posture information of the remote control 20 relative to the first device 200 or the vehicle screen, so as to determine the cursor display position on the vehicle screen, etc.

[0067] The internal memory 220 can be used to store programs, which include instructions. The processor 210 executes various functional applications and data processing of the first device 200 by running the instructions stored in the internal memory 220. The internal memory 220 may include a program storage area and a data storage area. The program storage area may store an operating system and software code for at least one application. The data storage area may store data generated during the use of the first device 200. In addition, the internal memory 220 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a general-purpose flash memory, etc.

[0068] Power management module 240 is configured to receive input from power source 241. Power source 241 can be a battery or a vehicle's power supply system. Power management module 240 receives power from the battery and / or the vehicle to provide the energy required for normal operation of processor 210, internal memory 220, communication module 250, and UWB module 260.

[0069] The communication module 250 can provide wireless communication functions applied to the first device 200, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. The communication module 250 can be one or more devices integrating at least one wireless communication function.

[0070] The communication module 250 can receive electromagnetic waves via antenna a, frequency modulate and filter the electromagnetic wave signals, and send the processed signals to the processor 210. The communication module 250 can also receive signals to be transmitted from the processor 210, frequency modulate and amplify them, and convert them into electromagnetic waves for radiation via antenna a.

[0071] The UWB module 260 can provide UWB communication, ranging, etc. applied to the first device 200. The UWB module 260 may include an antenna module b, which includes at least three antennas. Among them, three antennas can be used to achieve three-dimensional positioning of the first device 200. In another possible design, the first device 200 may also include multiple UWB modules 260. For example, when including three UWB modules 260, each UWB module 260 may include an antenna. In this way, the first device 200 may also include at least three antennas, thereby achieving three-dimensional positioning of the first device 200.

[0072] As shown in Figure 2, antenna module b may include three antennas. All three antennas are integrated into the UWB module 260 of the first device 200, or may be connected to the UWB module 260 of the first device 200. For example, the three antennas may be used to receive UWB signals from other devices, such as the second device 300.

[0073] Optionally, the IMU chip system 270 may include a gyroscope and an accelerometer. Alternatively, the IMU chip system 270 may include a gyroscope, an accelerometer, and a magnetometer. The gyroscope can be used to measure the change in attitude at the current measurement time relative to the previous measurement time; the accelerometer can be used to measure absolute attitude information at the current measurement time; and the magnetometer can be used to measure the strength and direction of the magnetic field.

[0074] It is understandable that the components shown in FIG2 do not constitute a specific limitation on the first device 200. The first device 200 may also include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently.

[0075] The following uses the second device 300 as a remote control as an example to introduce the structure of the device to which the embodiments of the present application are applicable.

[0076] As shown in FIG3 , the second device 300 may include a processor 310 , an internal memory 330 , a power management module 340 , a battery 341 , a communication module 350 , a UWB module 360 ​​, an IMU chip system 370 , and the like.

[0077] The functions of the hardware structure shown in FIG3 that are the same as those in FIG2 can be found in the description of FIG2 and will not be repeated here. The difference from FIG2 is that the power management module 340 included in the remote control generally obtains energy from the battery 341.

[0078] It is understandable that the components shown in FIG3 do not constitute a specific limitation on the second device 300. The second device 300 may also include more or fewer components than shown, or combine certain components, or split certain components, or arrange the components differently.

[0079] It should be understood that in the embodiments of the present application, "at least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c, or a, b and c, where a, b, c can be single or multiple. "Multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0080] In addition, it should be understood that, in the description of this application, words such as "first" and "second" are only used for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

[0081] It should be understood that the hardware structure of the first device can be as shown in Figure 2, where the program in the device is stored in the internal memory 220, and the processor 210 can run the program stored in the internal memory 220 to execute the process of the data processing method in a mobile scenario provided by the embodiment of the present application. It should be understood that the hardware structure of the second device can be as shown in Figure 3, where the software program in the device is stored in the internal memory 330, and the processor 310 can run the program stored in the internal memory 330 to execute the process of the data processing method in a mobile scenario provided by the embodiment of the present application.

[0082] In order to facilitate understanding of the data processing method in a mobile scenario provided by the present application, the implementation process of the method provided by the present application is introduced below in combination with the contents shown in Figures 4A to 12.

[0083] The following first explains the technical terms that may be involved in the embodiments of this application, including:

[0084] (1) UWB coordinate system of the first device 200 (also referred to as "e-system" in the present embodiment)

[0085] As shown in FIG4A , a schematic diagram of establishing a UWB coordinate system of the first device 200 provided in an embodiment of the present application is provided. As shown in FIG4A , the first device 200 can be a separate device independent of the vehicle screen and can be installed above the vehicle screen, for example, near the vehicle screen in the top area of ​​the car. In addition, as shown in FIG4A , the vehicle screen can be located between the front seats and in front of the middle or rear seats. The present application does not limit the installation location of the vehicle screen. For example, it can also be a main control screen located in the front row of the car.

[0086] Based on the content introduced in Figure 2, the UWB module of the first device 200 generally adopts a multi-antenna structure, such as a three-antenna structure. Refer to Figure 4B, which shows the distribution diagram of two three-antenna structures. Based on the three-antenna structure, any antenna in the three-antenna structure can be selected as the origin of the e-system. As shown in Figure 4B, antenna 0 is selected as the origin of the e-system; wherein, antenna 0 is the lower left antenna located in the three-antenna structure. As shown in Figure 4B, the direction parallel to the horizontal antenna line and pointing to the left edge of the antenna panel can be selected as the X-axis direction of the e-system; the direction of the antenna panel normal can be selected as the Y-axis direction of the e-system; based on the right-hand rule, the Z-axis direction of the e-system can be determined to be perpendicular to the X-axis. e 0Y e The plane points in the upward direction.

[0087] It should be noted that the embodiments of the present application do not limit the origin, X-axis direction, Y-axis direction, and Z-axis direction selected when establishing the e-coordinate system. A coordinate system may be established based on the position of the first device 200 and with any fixed reference object as the origin. The e-coordinate system can be used to reflect the relative position and posture of the remote control relative to the UWB module of the first device 200.

[0088] (2) Geographic Coordinate System (also referred to as “G-system” in this application embodiment)

[0089] Figure 5 shows a schematic diagram of establishing a geographic coordinate system according to an embodiment of the present application. Referring to Figure 5 , similar to the origin selected for the E-system, antenna 0 can also be selected as the origin of the G-system. Alternatively, other fixed reference objects can be selected as the origin of the G-system.

[0090] Optionally, when the IMU chip system 270 included in the first device 200 includes an accelerometer, a gyroscope, and a magnetometer, the geographical directions of east, north, and sky of the current location can be measured by the IMU chip system 270. Then, the geographical direction of east can be selected as the X-axis direction of the g-system, and the geographical direction of north can be selected as the Y-axis direction of the g-system. Based on the right-hand rule, the Z-axis direction of the g-system can be determined to be perpendicular to the X-axis. g 0Y g The plane points in the upward direction.

[0091] Alternatively, when the first device 200 includes only an accelerometer and a gyroscope but no magnetometer, since it can only measure the geographical direction of the sky but cannot measure the geographical directions of east and north, the Z-axis direction of the g-system can be defined as being consistent with the geographical direction of the sky, and the X-axis direction and the Y-axis direction of the g-system are defined as X and Y. g 0Y g Any two perpendicular directions chosen on a plane that conform to the right-hand rule.

[0092] It should be noted that the embodiments of the present application do not limit the origin, X-axis direction, Y-axis direction, and Z-axis direction selected when establishing the G-coordinate system. A coordinate system may be established based on the position of the first device 200 and with any fixed reference object as the origin. It is understood that since the G-coordinate system generally does not change over a short period of time, the G-coordinate system can be used to facilitate calculation of the relative posture information and relative position information of the second device relative to the first device.

[0093] (3) Remote Control Coordinate System (also referred to as “B System” in this embodiment)

[0094] As shown in Figure 6, it is a schematic diagram of establishing the remote control coordinate system provided by the embodiment of the present application. As shown in Figure 6, the center of the remote control can be selected as the origin of the remote control coordinate system; or, if the position of the IMU chip system of the remote control is stored, the center of the IMU chip system can also be selected as the origin of the remote control coordinate system. As shown in Figure 6, the short side direction of the front of the remote control can be selected as the X-axis direction of the b system; and the long side direction of the front of the remote control can be selected as the Y-axis direction of the b system; based on the right-hand rule, it can be determined that the Z-axis direction of the b system is perpendicular to the X-axis. b 0Y b The plane points upward

[0095] It should be noted that the embodiments of the present application do not limit the origin, X-axis direction, Y-axis direction, and Z-axis direction selected when establishing the B-coordinate system. A coordinate system may be established based on the second device 300 and using any fixed reference object as its origin. The fixed reference object may also be, for example, the center of gravity of the second device 300. The remote control coordinate system can be used to reflect remote control posture information and can be combined with the first device 200 to reflect the relative posture information of the remote control relative to the vehicle-mounted screen.

[0096] (4) Screen Coordinate System (hereinafter referred to as “S-system”) and Display Coordinate System (hereinafter referred to as “D-system”)

[0097] As shown in FIG7A, a schematic diagram of establishing the screen coordinate system provided in an embodiment of the present application is shown. As shown in FIG7A, the center of the top edge of the vehicle screen can be selected as the origin of the s-system. As shown in FIG7A, the left direction of the top edge of the vehicle screen can be selected as the X-axis direction of the s-system; and the direction perpendicular to the positive direction of the vehicle screen can be selected as the Y-axis direction of the s-system. Based on the right-hand rule, the Z-axis direction of the s-system can be determined to be perpendicular to the X-axis direction. s 0Y s The direction in which the plane points upwards can also be understood as the vertical edge of the car screen pointing upwards.

[0098] It should be noted that the embodiments of this application do not limit the origin, X-axis direction, Y-axis direction, and Z-axis direction selected when establishing the S-coordinate system. A coordinate system may be established based on the vehicle screen and with any fixed reference object as its origin. The fixed reference object may be, for example, the leftmost point at the top of the vehicle screen. The screen coordinate system can be used to reflect the posture information of the vehicle screen and can be combined with the remote control to reflect the relative posture information of the remote control with respect to the vehicle screen.

[0099] Figure 7B shows a schematic diagram of establishing a display coordinate system according to an embodiment of the present application. The d-axis is a two-dimensional coordinate system, and the upper left corner of the vehicle screen can be selected as the origin of the d-axis. As shown in Figure 7B , the right side of the top edge of the vehicle screen can be selected as the x-axis direction of the d-axis, and the direction perpendicular to the vertical direction of the vehicle screen can be selected as the y-axis direction of the d-axis.

[0100] It should be noted that the embodiments of this application do not limit the origin, X-axis direction, and Y-axis direction selected when establishing the d-coordinate system. A coordinate system may be established based on the vehicle screen and with any fixed reference object as the origin. The fixed reference object may also be, for example, the rightmost point at the top of the vehicle screen. The display coordinate system can be used to determine the display position of the remote control cursor on the vehicle screen.

[0101] For example, the origin of the s system and the origin of the d system can also be selected as the same origin.

[0102] (5) Attitude angle

[0103] The attitude angle is determined by the rotation relationship between the two coordinate systems. Taking the transformation of the remote control coordinate system (b system) relative to the UWB coordinate system (e system) as an example, the rotation relationship between the b system and the e system can be expressed by the Euler angle, which includes the heading angle ψ, the pitch angle and roll angle θ. Refer to FIG8, which is a schematic diagram of the attitude angles of the b-frame and the e-frame provided in the embodiment of the present application.

[0104] The heading angle ψ is the Y direction of the b system b Axis X in e-system e OY e Projection on the surface, and Y e The angle between the axes is positive when the remote controller yaws to the right;

[0105] Pitch angle Y of b series b Axis and X in e system e OY e The angle between the two faces is positive when the remote control is raised;

[0106] The roll angle θ is the Z of the b frame b Axis and contains moving vector Y b The angle between the plumb planes of the axes is positive when the remote control tilts to the right.

[0107] In addition, the relative attitude angle of the two coordinate systems can also be expressed by the attitude matrix, as shown in the following formula 1, which can be used to express the attitude matrix of the b system in the e system:

[0108] The relative attitude angles of two coordinate systems can be used to determine the attitude of one device relative to another. For example, the attitude of a remote control relative to a car screen can be determined, thereby determining the display position of the remote control cursor on the car screen.

[0109] For example, the conversion between the b-frame and the e-frame can also be achieved by first converting the b-frame to the g-frame, and then converting from the g-frame to the e-frame. This application does not limit the specific conversion method between the two coordinate systems. It is understood that the attitude angles between other coordinate systems, such as the conversion of the UWB coordinate system (e-frame) relative to the geographic coordinate system (g-frame), can be referred to the introduction shown in Figure 8, and this application will not repeat them.

[0110] FIG9 is a flow chart of a data processing method in a mobile scenario provided by an embodiment of the present application. Referring to FIG9 , the method can be applied to a multi-device pointing interaction scenario in a mobile scenario, which may include at least a first device 200 and a second device 300. The method may include the following steps:

[0111] In step 901A, the first device 200 measures and obtains first data through the UWB module 260 , and measures and obtains second data through the IMU chip system 270 .

[0112] As shown in FIG. 2 , the first device 200 may include a UWB module 260 and an IMU chip system 270 . Therefore, after the first device 200 is powered on, the UWB module 260 and the IMU chip system 270 may start measurement.

[0113] Optionally, the UWB module 260 and the IMU chip system 270 may be integrated hardware structures or chips, or may be separate hardware structures or chips. The specific composition of the UWB module 260 and the IMU chip system 270 is not limited in the embodiments of the present application.

[0114] Optionally, based on the introduction of the e-system in the aforementioned embodiment, the first data and the second data can be represented based on the e-system. It can be understood that the first relative spatial relationship between the UWB module 260 and the IMU chip system 270 is fixed and known; wherein the first relative spatial relationship includes a first relative position relationship and a first relative posture relationship. In this way, based on the first data measured by the UWB module 260, and the first data can be represented by the e-system; and based on the relatively fixed first relative spatial relationship between the UWB module 260 and the IMU chip system 270, the second data can also be represented by the e-system.

[0115] Alternatively, based on the introduction to the g-frame in the aforementioned embodiment, the first data and the second data can also be represented based on the g-frame. It is understandable that the second relative spatial relationship of the UWB module 260 relative to the geographic coordinate system is fixed and known; wherein the second relative spatial relationship includes a second relative position relationship and a second relative attitude relationship; the third relative spatial relationship of the IMU chip system 270 relative to the geographic coordinate system is also fixed and known; wherein the third relative spatial relationship includes a third relative position relationship and a third relative attitude relationship. In this way, the first data and the second data can also be represented by the g-frame.

[0116] Alternatively, in other optional examples, the first data and the second data may be represented based on other coordinate systems. In addition, the first data and the second data may also be represented by different coordinate systems, which is not limited in this application.

[0117] In step 901B, the second device 300 obtains third data by measuring through the UWB module 360 ​​, and obtains fourth data by measuring through the IMU chip system 370 .

[0118] As shown in FIG3 , the second device 300 includes a UWB module 360 ​​and an IMU chip system 370 , so after the second device 300 is powered on, the UWB module 360 ​​and the IMU chip system 370 can start measurement.

[0119] Optionally, the UWB module 360 ​​and the IMU chip system 370 may be integrated hardware structures or chips, or may be separate hardware structures or chips. The specific composition of the UWB module 360 ​​and the IMU chip system 370 is not limited in the embodiments of the present application.

[0120] Optionally, based on the introduction of the B system in the aforementioned embodiment, the third data and the fourth data can be represented based on the B system. It can be understood that the fourth relative spatial relationship between the UWB module 360 ​​and the IMU chip system 370 is fixed and known; wherein, the fourth relative spatial relationship includes a fourth relative position relationship and a fourth relative posture relationship. Considering that the position of the second device is relatively unstable, the first priority can be selected to represent the third data and the fourth data based on the B system, but the embodiment of the present application is not limited to representing the third data and the fourth data based on the B system.

[0121] Step 902: The first device 200 sends a first message to the second device 300; wherein the first message includes first data and second data.

[0122] Exemplarily, the first message may be sent using, but not limited to, the following communication methods: UWB, Bluetooth, infrared, NFC, star flash, Wi-Fi, and other communication methods.

[0123] It should be noted that, in the embodiment of the present application, the second device 300 is used as an example to process the first data, the second data, the third data, and the fourth data, but the processing is not limited to the second device 300. For example, the first device 200 can also be used for processing. In this example, the second device 300 can send the third data and the fourth data to the first device. For another example, a third device other than the first device 200 and the second device 300 can also be used for processing. In this example, the first device 200 can send the first data and the second data to the third device, and the second device 300 can send the third data and the fourth data to the third device. Optionally, the third device can be a device or a cloud server, which is not limited in this application.

[0124] In step 903, the second device 300 calculates first updated data of the second device 300 relative to the first device 200 based on the second data and the fourth data. The first updated data is used to reflect the update of the relative spatial relationship between the first device 200 and the second device 300 due to the movement of the mobile carrier. It will be understood that due to the movement of the mobile carrier, based on the principle of inertial navigation, the first device 200 and the second device 300 generate inertial offset data. Since the first device 200 and the second device 300 are located at different positions on the mobile carrier, the degree of inertial offset varies, resulting in a difference between the inertial offset data, i.e., the first updated data.

[0125] It can be understood that based on the second data measured in real time by the first device 200, the position and posture of the first device 200 relative to the g-system can be updated, that is, the inertial offset data of the first device 200 can be obtained. Based on the fourth data measured in real time by the second device 300, the position and posture of the second device 300 relative to the g-system can be updated, that is, the inertial offset data of the second device 300 can be obtained. In a mobile scenario, due to the influence of factors such as the movement, rotation, and bumps of the mobile carrier, the update degree of the position and / or posture of the first device 200 and the second device 300 is different, that is, there is a difference in the inertial offset data of the second device relative to the first device. For example, in a scenario where a car is rotating, the first device 200 is closer to the front of the car than the second device 300, so the inertial offset degree of the first device 200 is greater than the inertial offset degree of the second device 300.

[0126] In addition, it can be understood that in this scenario, the data measured by the IMU chip system can be used to indicate the update of the position and / or attitude of the device due to the inertial navigation principle, and this update of the position and / or attitude will cause cursor drift. Therefore, by accurately obtaining the first update data, it is possible to calculate a more realistic relative spatial relationship between the first device and the second device based on the first update data, so that after the fusion processing of step 905, the change in the relative spatial relationship caused by the influence of factors such as the movement, rotation, and bumps of the mobile carrier can be offset, thereby reducing the occurrence of cursor drift and ensuring the user experience of multi-device pointing interaction.

[0127] In the embodiments of the present application, the first update data mainly includes first posture update data and first position update data. In the following embodiments, taking the second device 300 as a remote control and the update data represented based on the g-system as an example, the first update data calculated may include first posture update data calculated based on the posture update algorithm and first position update data calculated based on the position update algorithm.

[0128] The calculation process of the attitude update algorithm and the position update algorithm can be found in the following:

[0129] (1) Posture update algorithm, used to obtain updated posture data of the remote controller.

[0130] Assume that the attitude rotation matrix of the g system relative to the b system at time m is expressed as It can be obtained by the following formula 2-1:

[0131] In formula 2-1, time m can be understood as the time corresponding to the mth measurement, and time m-1 can be understood as the time corresponding to the m-1th measurement. It is used to indicate the attitude rotation matrix of the g system relative to the b system at time m-1. It has been calculated at time m-1 and is a known number.

[0132] In formula 2-1 It is used to indicate the attitude rotation matrix of the g-frame at time m relative to the g-frame at time m-1. In addition, in the mobile scenario, since the geographical location of the mobile carrier changes little in a short period of time, it can be considered that the g-frame has no rotation, so Where I represents the identity matrix. It can also be expressed as the following formula 2-2:

[0133] In Formula 2-1 or Formula 2-2 It is used to indicate the attitude rotation matrix of the b system at time m relative to the b system at time m-1. Generally depends on the rotation vector Φ output by the remote controller's IMU chip system at time m m , for example based on Φ m , It can be expressed as the following formula 2-3:

[0134] Among them, the and Φ m The relationship between Φ and Φ can be found in the related art, and this application will not elaborate on this. m Under the 2-sample condition, it can be expressed as follows:

[0135] Formula 2-4 introduces Φ m The method of obtaining Δθ can be found in the introduction of related technologies, and this application will not elaborate on this. m (1) and Δθ m (2) and The angular increment in time. And Δθ m is Δθ m (1) and Δθ m The total increment of (2) can be expressed as the following formula 2-5: Δθ m =Δθ m (1)+Δθ m (2) Formula 2-5

[0136] Δθ in Formula 2-4 and Formula 2-5 m (1) and Δθ m (2) It can be measured by the IMU chip system of the remote control.

[0137] (2) Position update algorithm, used to obtain updated position data of the remote control.

[0138] Assume that the position rotation matrix of the g system relative to the b system at time m is expressed as It can be expressed by the following formula 3-1:

[0139] T in Formula 3-1 m It is used to represent the time difference between time m and time m-1. For example, the time difference between the time corresponding to the mth measurement and the time corresponding to the m-1th measurement can be 1 second, 10 seconds, etc. Indicates that the time obtained is the middle time between time m and time m-1 (that is, The speed value at the time.

[0140] In formula 3-1 It can be expressed by the following formula 3-2:

[0141] Among them, formula 3-2 is used to express The speed value at time m can be obtained by averaging the speed value at time m and the speed value at time m-1.

[0142] In addition, in Formula 3-2 It can be calculated using the speed update algorithm. The speed update algorithm is used to obtain the remote controller's speed information to better determine the updated position information. The speed update algorithm formula can be shown in the following formula 3-2-1:

[0143] Among them, in formula 3-2-1 The rotation matrix used to represent the acceleration vector of the remote control b system relative to the g system at time m can be measured by the IMU chip system of the remote control; wherein, the IMU chip system generally includes an accelerometer, and the acceleration vector can be measured by the accelerometer. It can be expressed as the following formula 3-2-2:

[0144] In formula 3-2-2, based on the introduction in the previous embodiment, therefore It can also be expressed as the following formula 3-2-3:

[0145] Among them, Formula 3-2-2 or Formula 3-2-3 is used to express the expression of the acceleration vector. Please refer to the introduction in the relevant technology, and this application will not elaborate on it.

[0146] Based on the above introduction to the attitude update algorithm and position update algorithm, we can not only calculate the attitude rotation matrix of the remote control and the position rotation matrix The posture rotation matrix of the first device 200 can also be calculated and the position rotation matrix Then, the first update data of the second device 300 relative to the first device 200 can be calculated. The first update data may include: the first posture update data of the second device 300 relative to the first device 200 and the first position update data of the second device 300 relative to the first device 200 in, It can be expressed by the following formula 4-1: It can be expressed by the following formula 4-2:

[0147] The first posture update data shown in Formula 4-1 and the first position update data shown in Formula 4-2 are caused by factors such as the movement of the mobile carrier. Therefore, after the second posture update data and the second position update data included in the second update data are fused in step 905, position cancellation and posture cancellation can be achieved.

[0148] Therefore, through the measurements of the IMU chip system included in the first device 200 and the IMU chip system included in the second device 300, it is possible to consider the deviation in the inertial offset of the two devices due to the movement of the mobile carrier, which in turn causes the cursor offset. The cursor offset can then be offset through position cancellation and posture cancellation to ensure that the cursor display can more accurately reflect the user operation.

[0149] In step 904, the second device 300 calculates second updated data of the second device 300 relative to the first device 200 based on the first data and the third data; the second updated data is used to reflect the update of the relative spatial relationship between the first device 200 and the second device 300. The second updated data includes second posture update data and second position update data. It can be understood that the second updated data is an update of the relative spatial relationship between the first device 200 and the second device 300 caused by the user operation and the movement of the mobile carrier. It can also be understood that the second updated data is used to reflect the actual relative spatial relationship between the first device 200 and the second device 300, and is obtained through measurement.

[0150] The calculation of the second update data in the embodiment of the present application will not be described in detail, and reference may be made to the calculation process of the first update data in step 903 .

[0151] It should be noted that the embodiment of the present application does not limit the execution order of step 903 and step 904.

[0152] In step 905, the second device 300 fuses the first updated data and the second updated data to calculate relative spatial data. The relative spatial data reflects the target relative spatial relationship between the first device 200 and the second device 300. The target relative spatial relationship includes a target relative position relationship and a target relative posture relationship. The target relative position relationship reflects the relative position of the first device 200 and the second device 300 in space, and the target relative posture relationship reflects the relative posture of the first device 200 and the second device 300 in space.

[0153] Exemplarily, the relative spatial data can be represented by third posture update data and third position update data; wherein, the third posture update data is obtained after offsetting the second posture update data based on the first posture update data, and the third position update data is obtained after offsetting the second position update data based on the first position update data.

[0154] In the embodiment of the present application, the fusion processing can eliminate the influence of the first update data on the second update data, thereby obtaining an update that can reflect the relative spatial relationship between the first device 200 and the second device 300 caused by the user operation. It can also be understood that the relative spatial data is used to reflect the target relative spatial relationship calculated in the embodiment of the present application, and is calculated, not measured.

[0155] In addition, UWB measurements can be directly converted to and location The first update data and the second update data can be fused using a Kalman filter or other method to obtain the accurate posture and position of the remote control in the UWB coordinate system.

[0156] Step 906: The second device 300 sends the relative spatial data to the vehicle screen.

[0157] In one possible scenario, the first device 200 is an independent device relative to the vehicle screen, and the second device 300 can send the relative spatial data to the vehicle screen.

[0158] In another possible scenario, the first device 200 can also be a vehicle-mounted screen, which can also be understood as the IMU chip system and the UWB module integrated in the vehicle-mounted screen. In this scenario, the second device 300 can send relative spatial data to the first device 200, that is, the vehicle-mounted screen.

[0159] Step 907: The vehicle screen displays a cursor according to the relative space data.

[0160] For example, the vehicle screen can convert the relative spatial data into the s system for representation first, and then into the d system for representation, so that the calculated cursor position of the second device 300 can be projected and displayed on the vehicle screen.

[0161] As another example, the position of the cursor displayed on the vehicle screen can be calculated in the UWB coordinate system based on the range and posture of the vehicle screen in the UWB coordinate system and the posture and position of the remote control in the UWB coordinate system.

[0162] In the method provided in the embodiment of the present application, by deploying a UWB module and an IMU chip system on the first device 200, real-time measurement of the spatial data of the first device 200 can be achieved; and by simultaneously deploying a UWB module and an IMU chip system on the second device 300, real-time measurement of the spatial data of the second device 300 can also be achieved. In this way, the absolute spatial information measured by the UWB modules of the first device 200 and the second device 300 can be used to obtain second updated data between the first device 200 and the second device 300. The second updated data is caused by various factors such as the movement of the mobile carrier and user operations. In addition, the relative spatial information measured by the IMU chip systems of the first device 200 and the second device 300 can also be used to obtain first updated data between the first device 200 and the second device 300. The first updated data is caused by inertial factors such as the movement of the mobile carrier. In this way, through the fusion processing of the first update data and the second update data, the first update data included in the second update data can be offset, so that the change in the relative spatial relationship of the second device 300 with respect to the first device 200 caused by the user operation can be obtained, and the probability of abnormal cursor offset can be reduced, and the degree of cursor offset can be reduced, thereby ensuring the user experience of multi-device pointing interaction in a mobile scenario.

[0163] Based on the process shown in Figure 9, Figure 10 is a schematic diagram of the time synchronization process of a data processing method applied to a mobile scenario provided by an embodiment of the present application. This process can be executed before step 902 and may include the following steps:

[0164] Step 1001: The first device 200 sends a synchronization frame to the second device 300 and starts timing.

[0165] Step 1002: The second device 300 receives a synchronization frame from the first device 200 and starts timing.

[0166] Step 1003: The second device 300 sends a return frame to the first device 200 and stops timing to obtain a timing result T2.

[0167] Step 1004: The first device 200 receives a return frame from the second device 300 and stops timing to obtain a timing result T1.

[0168] Optionally, in step 1005A, when the return frame includes T2, the first device 200 may determine the data transmission delay T based on T1 and T2. delay ; which can be expressed by the following formula 5:

[0169] Alternatively, in step 1005B, when the return frame does not include T2, the first device 200 may send T1 to the second device 300.

[0170] Step 1005C: After the second device 300 receives T1 sent from the first device 200, it can determine the data transmission delay T based on T1 and T2. delay .

[0171] Step 1006: The first device 200 and the second device 300 communicate with each other based on T delay It is understood that the first device 200 or the second device 300, whichever has a faster time, can synchronize time based on T delay Delay processing.

[0172] Through the process shown in Figure 10, the delay caused by system scheduling and air interface transmission during data transmission can be reduced, thereby improving the processing accuracy of the process shown in Figure 9 and ensuring the processing effect of cursor drift.

[0173] Taking the second device 300 as a remote control as an example, as shown in Figure 11, it is a scene in which the remote control is held still while driving, and the intercepted data is data when the car makes a 90° turn or so while driving. As shown in interface 1101 in Figure 11, the azimuths of the remote control side and the first device side are respectively solved. It can be seen from interface 1101 that there is a 90° offset in the azimuth. As shown in interface 1102 in Figure 11, it is the azimuth of the remote control side and the first device side obtained after data processing using the method provided in an embodiment of the present application. It can be seen from interface 1102 that the 90° offset is offset. As shown in interface 1103 in Figure 11, it is a stretched diagram on the vertical axis compared to interface 1102. It can also be seen from interface 1103 that the 90° azimuth is offset.

[0174] Still taking the second device 300 as a remote control as an example, FIG12 is another azimuth diagram of a data processing method in a mobile scenario provided by an embodiment of the present application. As shown in FIG12 , this is a scenario in which the remote control periodically points left and right during driving, and the intercepted data is data when the car makes a 90° left and right turn while driving. As shown in interface 1201 in FIG12 , the azimuth angles of the remote control side and the first device side are respectively solved. From interface 1201, it can be seen that not only are there multiple 90° offsets in the azimuth angle, but there are also small fluctuations in the azimuth angle. As shown in interface 1202 in FIG12 , the azimuth angles of the remote control side and the first device side obtained after data processing using the method provided in an embodiment of the present application, it can be seen from interface 1202 that the multiple 90° azimuth offsets are offset, and it can be seen that after the offset, the left and right shaking of the remote control relative to the first device is clearly restored.

[0175] Still taking the second device 300 as a remote control as an example, Figure 13 is a schematic diagram of the interface of a data processing method in a mobile scenario provided by an embodiment of the present application. Figure 13 shows a scenario in which the remote control is held motionless during driving. As shown in interface 1301 in Figure 13, there was cursor drift before the method provided by the embodiment of the present application was adopted. As shown in interface 1302 in Figure 13, after the method provided by the embodiment of the present application was adopted, the degree of cursor drift was reduced, and the cursor can be relatively stable on the in-vehicle screen. Therefore, through the method provided by the embodiment of the present application, the pointing effect in response to the user's operation can be more accurately reflected, ensuring the pointing interaction experience of multiple devices.

[0176] Based on the above embodiments, the present application also provides a first device, which includes multiple functional modules; the multiple functional modules interact with each other to implement the functions performed by the first device in each method described in the embodiments of the present application. The multiple functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the multiple functional modules can be arbitrarily combined or divided based on the specific implementation. For example, execute steps 901A and 902 performed by the first device 200 in the embodiment shown in Figure 9. Alternatively, execute steps 1001, 1004, 1005A, and 1006 performed by the first device 200 in the embodiment shown in Figure 10, or execute steps 1001, 1004, 1005B, and 1006 performed by the first device 200 in the embodiment shown in Figure 10.

[0177] Based on the above embodiments, the present application further provides a first device, which includes at least one processor and at least one memory, wherein the at least one memory stores a program, and when the first device runs the program, the at least one processor performs the functions performed by the first device in each method described in the embodiments of the present application. For example, steps 901A and 902 performed by the first device 200 in the embodiment shown in Figure 9 are executed. Alternatively, steps 1001, 1004, 1005A, and 1006 performed by the first device 200 in the embodiment shown in Figure 10 are executed, or steps 1001, 1004, 1005B, and 1006 performed by the first device 200 in the embodiment shown in Figure 10 are executed.

[0178] Based on the above embodiments, the present application also provides a second device, which includes multiple functional modules; the multiple functional modules interact with each other to implement the functions performed by the second device in each method described in the embodiments of the present application. The multiple functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the multiple functional modules can be arbitrarily combined or divided based on the specific implementation. For example, execute steps 901B, 902 to 906 performed by the second device 300 in the embodiment shown in Figure 9. Alternatively, execute steps 1002, 1003, and 1006 performed by the second device 300 in the embodiment shown in Figure 10, or execute steps 1002, 1003, 1005B, 1005C, and 1006 performed by the second device 300 in the embodiment shown in Figure 10.

[0179] Based on the above embodiments, the present application also provides a second device, which includes at least one processor and at least one memory, wherein the at least one memory stores a program, and when the second device runs the program, the at least one processor performs the functions performed by the second device in each method described in the embodiments of the present application. For example, step 901B, step 902 to step 906 performed by the second device 300 in the embodiment shown in Figure 9 are executed. Alternatively, steps 1002, step 1003, and step 1006 performed by the second device 300 in the embodiment shown in Figure 10 are executed, or steps 1002, step 1003, step 1005B, step 1005C, and step 1006 performed by the second device 300 in the embodiment shown in Figure 10 are executed.

[0180] Based on the above embodiments, the present application further provides a data processing system for a mobile scenario. The data processing system for a mobile scenario may include the first device and the second device described in the above embodiments. Optionally, the data processing system for a mobile scenario may further include a third device. Alternatively, the data processing system for a mobile scenario may further include an in-vehicle screen.

[0181] Based on the above embodiments, the present application also provides a program product, which includes: a program (also called code, or instructions), which, when executed, enables a device or computer to execute the methods described in the embodiments of the present application.

[0182] Based on the above embodiments, the present application also provides a readable storage medium, in which a program is stored. When the program is executed by a device or a computer, the device or the computer executes the methods described in the embodiments of the present application.

[0183] Based on the above embodiments, the present application further provides a chip, which is used to read the program stored in the memory to implement the various methods described in the embodiments of the present application.

[0184] Based on the above embodiments, the present application provides a chip system, which includes a processor for supporting a device to implement the various methods described in the embodiments of the present application. In one possible design, the chip system also includes a memory, which is used to store necessary programs and data. The chip system can be composed of chips, or it can include chips and other discrete devices. Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects.

[0185] Obviously, those skilled in the art may make various modifications and variations to this application without departing from the scope of protection of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A data processing method in a mobile scenario, characterized in that: The method comprises: Acquire first data and second data of the first device; Acquire third data and fourth data of the second device; Obtaining first updated data according to the second data and the fourth data; wherein the first updated data is used to reflect a difference in inertial offset data of the second device relative to the first device; Obtaining second updated data based on the first data and the third data; the second updated data is used to reflect the measured actual relative spatial relationship of the second device with respect to the first device; The second updated data is fused based on the first updated data to obtain relative spatial data, where the relative spatial data is used to reflect a target relative spatial relationship between the second device and the first device.

2. The method according to claim 1, characterized in that The first update data includes first posture update data and first position update data; the second update data includes second posture update data and second position update data; the relative space data includes third posture update data and third position update data; The second updated data is fused based on the first updated data to obtain relative spatial data, including: Obtaining the third posture update data according to a difference between the first posture update data and the second posture update data; The third location update data is obtained according to a difference between the first location update data and the second location update data.

3. The method according to claim 1 or 2, characterized in that The second data includes first posture data and first position data; the fourth data includes second posture data and second position data; The obtaining of first updated data according to the second data and the fourth data includes: Obtaining first posture update data according to the first posture data and the second posture data, where the first posture update data is used to reflect a posture change of the second device relative to the first device due to inertia; First position update data is obtained according to the first position data and the second position data, where the first position update data is used to reflect a position change of the second device relative to the first device due to inertia.

4. The method according to any one of claims 1 to 3, characterized in that The first data includes third posture data and third position data; the third data includes fourth posture data and fourth position data; The obtaining of second updated data according to the first data and the third data includes: Obtaining second posture update data according to the third posture data and the fourth posture data, where the second posture update data is used to reflect an actual posture change of the second device relative to the first device; Second location update data is obtained according to the third location data and the fourth location data, where the second location update data is used to reflect actual location changes of the second device relative to the first device.

5. The method according to any one of claims 1 to 4, characterized in that The first device is a fixed device relative to the mobile carrier, and the second device is a mobile device relative to the mobile carrier.

6. The method according to claim 5, characterized in that The target-relative spatial relationship is used to reflect the movement of the second device relative to the first device, where the movement is caused by a user operation.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Based on the relative spatial data, a position of a display cursor of a target screen is determined; the target screen is the screen pointed to by the second device.

8. The method according to any one of claims 1 to 7, characterized in that Before obtaining the first data and the second data of the first device, the method further includes: Perform time synchronization between the first device and the second device.

9. The method according to any one of claims 1 to 8, characterized in that The first data is measured by an ultra-wideband (UWB) module of the first device, and the second data is measured by an inertial measurement unit (IMU) chip system of the first device; The third data is measured by the UWB module of the second device, and the fourth data is measured by the IMU chip system of the second device.

10. A device, characterized in that The system comprises at least one processor coupled to at least one memory, and the at least one processor is configured to read a program stored in the at least one memory to execute the method according to any one of claims 1 to 9.

11. A readable storage medium, characterized in that: The readable storage medium stores instructions, which, when executed on a device, enable the device to execute the method according to any one of claims 1 to 9.

Images