Data processing method and equipment in mobile scene

Abstract

The invention provides a data processing method and equipment in a mobile scene, which are used for carrying out calibration processing on cursor drift possibly occurring when directional equipment is used in the mobile scene, so that the interactive experience of a user can be guaranteed. The method comprises the following steps: acquiring first data and second data of first equipment; acquiring third data and fourth data of the second equipment; obtaining first update data according to the second data and the fourth data; the first update data is used for reflecting a difference value of inertia offset data of the second equipment relative to the first equipment; obtaining second update data according to the first data and the third data; the second update data is used for reflecting the measured actual relative spatial relationship of the second equipment relative to the first equipment; and performing fusion processing on the second update data based on the first update data to obtain relative spatial data, the relative spatial data being used for reflecting a target relative spatial relationship of the second device relative to the first device.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a data processing method and device in a mobile scenario. Background Art

[0002] As cars become increasingly intelligent, users can access more functions through in-car screens, such as online navigation, music playback, and video playback. This increased intelligence has also led to a gradual increase in the size of in-car screens. However, in-car screens are typically placed in the front row of the car. Even if a screen is installed in the middle or back row, the user is still far away from the screen during the ride, making it inconvenient for users to interact with the in-car screen.

[0003] A pointing remote control interacts with the ultra-wideband (UWB) on the vehicle's screen to calculate its posture and position relative to the screen. This determines the specific location of the vehicle's screen and displays the cursor at that location. The user experience of a pointing remote control is similar to that of an infrared laser pointer, and its interaction method is more in line with user habits. Therefore, a pointing remote control can enhance user comfort in interacting with the vehicle's screen.

[0004] However, in a mobile vehicle such as a car, if the mobile vehicle turns or bumps, the cursor may drift when using a pointing remote control. Summary of the Invention

[0005] The present application provides a data processing method and device in a mobile scenario, which is used to calibrate the cursor drift that may occur when using a pointing remote control in a mobile scenario, thereby ensuring the user's interactive experience.

[0006] The first aspect provides a data processing method in a mobile scenario. In this method, first data and second data of a first device are obtained; third data and fourth data of a second device are obtained; first update data is obtained based on the second data and the fourth data; the first update data is used to reflect the difference in inertial offset data of the second device relative to the first device; second update data is obtained based on the first data and the third data; the second update data is used to reflect the measured actual relative spatial relationship of the second device relative to the first device; the second update data is fused based on the first update data to obtain relative spatial data, and the relative spatial data is used to reflect the target relative spatial relationship of the second device relative to the first device.

[0007] The first data can be understood as the absolute spatial information of the first device measured by the first device; the second data can be understood as the relative spatial information of the first device measured by the first device, that is, the change in spatial information measured by the first device at the current measurement time compared to the spatial information at the previous measurement time. Similarly, the third data can be understood as the absolute spatial information of the second device measured by the second device; the fourth data can be understood as the relative spatial information of the second device measured by the second device, that is, the change in spatial information measured by the second device at the current measurement time compared to the spatial information at the previous measurement time.

[0008] The actual relative spatial relationship can be used to represent the relative spatial relationship between the first device and the second device, determined based on the absolute spatial information of the first device and the absolute spatial information of the second device. It will be appreciated that changes in the actual relative spatial relationship are not only caused by user operations on the second device, but also by inertial offset caused by the movement of the mobile carrier.

[0009] The target relative spatial relationship can be used to represent the relative spatial relationship between the first and second devices calculated by offsetting the change in the relative spatial relationship between the first and second devices, which is obtained based on the relative spatial information of the first and second devices. It should be understood that the target relative spatial relationship cannot be directly determined through measurement and must be calculated based on the actual relative spatial relationship and the change in the relative spatial relationship.

[0010] In this method, absolute and relative spatial information are measured by the first and second devices, respectively. Changes in the relative spatial relationship caused by inertial offset can be offset through calculation, thereby capturing changes in the relative spatial relationship caused by user manipulation. This method, based on the calculated relative spatial relationship of the target, allows pointing interactions between the first and second devices to be performed, reducing the probability and extent of abnormal cursor offset, thereby ensuring a consistent user experience for multi-device pointing interactions in mobile scenarios.

[0011] In one possible scenario, the first update data includes first posture update data and first position update data; the second update data includes second posture update data and second position update data. The relative spatial data includes third posture update data and third position update data. Fusion processing is performed on the second update data based on the first update data to obtain the relative spatial data, including: obtaining third posture update data based on the difference between the first posture update data and the second posture update data; and obtaining third position update data based on the difference between the first position update data and the second position update data.

[0012] In this scenario, based on the measured change in the relative spatial relationship between the first device and the second device, the actual relative spatial relationship is offset, so that the target relative spatial relationship between the first device and the second device determined by the user operation can be obtained more accurately, thereby improving the stability of the cursor display.

[0013] In one possible scenario, the second data includes first posture data and first position data; the fourth data includes second posture data and second position data. Obtaining first update data based on the second data and the fourth data includes: obtaining first posture update data based on the first posture data and the second posture data, the first posture update data being used to reflect a change in the posture of the second device relative to the first device due to inertia; and obtaining first position update data based on the first position data and the second position data, the first position update data being used to reflect a change in the position of the second device relative to the first device due to inertia.

[0014] In this scenario, by each device measuring its own relative spatial information, the change in the relative spatial relationship between the two devices can be accurately determined, thereby facilitating a more accurate calculation of the target relative spatial relationship between the two devices.

[0015] In one possible scenario, the first data includes third posture data and third position data; the third data includes fourth posture data and fourth position data. Obtaining second update data based on the first data and the third data includes: obtaining second posture update data based on the third posture data and the fourth posture data, the second posture update data being used to reflect actual posture changes of the second device relative to the first device; and obtaining second position update data based on the third position data and the fourth position data, the second position update data being used to reflect actual position changes of the second device relative to the first device.

[0016] In this scenario, by each device measuring its own absolute spatial information, the actual relative spatial relationship between the two devices can be accurately determined, which can facilitate a more accurate calculation of the target relative spatial relationship between the two devices.

[0017] In a possible scenario, the first device is a fixed device relative to the mobile carrier, and the second device is a mobile device relative to the mobile carrier.

[0018] In this scenario, when the user uses the second device to perform pointing operations on the first device, by calculating the changes in the spatial information of the second device caused by the user operation, the accuracy of multi-device pointing interaction can be guaranteed, and the user's experience of multi-device pointing interaction can be guaranteed.

[0019] In a possible scenario, the target relative spatial relationship is used to reflect the movement of the second device relative to the first device, where the movement is caused by a user operation.

[0020] In one possible scenario, the method further includes: determining a position of a display cursor of a target screen based on the relative spatial data; the target screen is the screen to which the second device is pointing. Optionally, the target screen is the first device. Alternatively, the target screen is a device different from the first device.

[0021] In this scenario, displaying the cursor based on the calculated relative spatial data can reduce the probability of abnormal cursor offset and the degree of cursor offset.

[0022] In a possible scenario, before obtaining the first data and the second data of the first device, the method further includes: performing time synchronization between the first device and the second device.

[0023] In this scenario, by performing time synchronization between the first device and the second device, the synchronization of measurements performed by the first device and the second device can be guaranteed, thereby ensuring the accuracy of calculation of the relative spatial relationship of the targets.

[0024] In one possible scenario, the first data is measured by the UWB module of the first device, and the second data is measured by the inertial measurement unit (IMU) chip system of the first device; the third data is measured by the UWB module of the second device, and the fourth data is measured by the IMU chip system of the second device.

[0025] In this scenario, by integrating the UWB module and IMU chip system in the first device, and also integrating the UWB module and IMU chip system in the second device, the relative spatial relationship of the target can be calculated, the probability of abnormal cursor offset can be reduced, and the degree of cursor offset can be reduced, thereby ensuring the user experience of multi-device pointing interaction in mobile scenarios.

[0026] A second aspect provides a data processing system in a mobile scenario, which may include a first device and a second device. The system is used to implement the method described in the first aspect and its possible scenarios.

[0027] In one possible scenario, a first device obtains first data and second data from the first device; a second device obtains third data and fourth data from the second device. Furthermore, the first device sends the first data and second data to the second device. Based on this, the second device obtains first update data based on the second data and the fourth data; the first update data is used to reflect the difference in inertial offset data of the second device relative to the first device; the second device obtains second update data based on the first data and the third data; the second update data is used to reflect the measured actual relative spatial relationship of the second device relative to the first device; the second device fuses the second update data based on the first update data to obtain relative spatial data, which is used to reflect the target relative spatial relationship of the second device relative to the first device.

[0028] In another possible scenario, the first device obtains the first data and second data of the first device; the second device obtains the third data and fourth data of the second device. Furthermore, the second device sends the third data and fourth data to the first device. Based on this, the first device obtains first update data based on the second data and the fourth data; the first update data is used to reflect the difference in inertial offset data of the second device relative to the first device; the first device obtains second update data based on the first data and the third data; the second update data is used to reflect the measured actual relative spatial relationship of the second device relative to the first device; the first device fuses the second update data based on the first update data to obtain relative spatial data, and the relative spatial data is used to reflect the target relative spatial relationship of the second device relative to the first device.

[0029] In another possible scenario, the system may also include a third device. The first device obtains the first data and the second data of the first device; the second device obtains the third data and the fourth data of the second device. In addition, the first device sends the first data and the second data to the third device, and the second device sends the third data and the fourth data to the third device. Based on this, the third device obtains the first update data according to the second data and the fourth data; the first update data is used to reflect the difference in inertial offset data of the second device relative to the first device; the third device obtains the second update data according to the first data and the third data; the second update data is used to reflect the measured actual relative spatial relationship of the second device relative to the first device; the third device fuses the second update data based on the first update data to obtain relative spatial data, and the relative spatial data is used to reflect the target relative spatial relationship of the second device relative to the first device. Optionally, the third device can be other devices or a cloud server.

[0030] Optionally, the system may also include one or more vehicle-mounted screens.

[0031] A third aspect provides a device comprising multiple functional modules; the multiple functional modules interact with each other to implement the method described in the first aspect and its possible scenarios. The multiple functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the multiple functional modules can be arbitrarily combined or divided based on the specific implementation.

[0032] The fourth aspect provides a device comprising at least one processor and at least one memory, wherein the at least one memory stores a program, and when the device runs the program, the at least one processor executes the method described in the first aspect and its possible scenarios. Optionally, the device can be understood as a first device, and the first device can receive third data and fourth data from a second device. Alternatively, the device can also be understood as a second device, and the second device can receive first data and second data from the first device. Alternatively, the device can also be understood as a third device, and the third device can receive first data and second data from the first device, and receive third data and fourth data from the second device.

[0033] The fifth aspect further provides a program product, which, when run on a device, enables the device to execute the method described in any one of the above aspects and its possible scenarios.

[0034] The sixth aspect also provides a readable storage medium, in which a program is stored. When the program is executed by a device, the device executes the method described in any one of the above aspects and its possible scenarios.

[0035] The seventh aspect also provides a chip, which is used to read the program stored in the memory and execute the method described in any of the above aspects and its possible scenarios.

[0036] An eighth aspect further provides a chip system, comprising a processor configured to support a device in implementing the method described in any of the above aspects and its possible scenarios. In one possible scenario, the chip system further comprises a memory configured to store the necessary programs and data. The chip system may be composed of a chip or may include a chip and other discrete components. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram showing an application scenario to which the embodiments of the present application are applicable;

[0038] Figure 2 A schematic diagram of the hardware structure of the first device 200 provided in an embodiment of the present application;

[0039] Figure 3 A schematic diagram of the hardware structure of the second device 300 provided in an embodiment of the present application;

[0040] Figure 4A This is one of the schematic diagrams for establishing the UWB coordinate system of the first device 200 provided in an embodiment of the present application;

[0041] Figure 4B The second schematic diagram of establishing the UWB coordinate system of the first device 200 provided in an embodiment of the present application;

[0042] Figure 5 A schematic diagram of establishing a geographic coordinate system provided in an embodiment of the present application;

[0043] Figure 6 A schematic diagram of establishing a remote control coordinate system according to an embodiment of the present application;

[0044] Figure 7A A schematic diagram of establishing a screen coordinate system provided in an embodiment of the present application;

[0045] Figure 7B A schematic diagram of establishing a display coordinate system provided in an embodiment of the present application;

[0046] Figure 8 A schematic diagram of the attitude angles of the b-frame and the e-frame provided in an embodiment of the present application;

[0047] Figure 9 A flowchart of a data processing method in a mobile scenario provided by an embodiment of the present application;

[0048] Figure 10A schematic diagram of a time synchronization process for a data processing method in a mobile scenario provided by an embodiment of the present application;

[0049] Figure 11 A schematic diagram of the azimuth angle of a data processing method in a mobile scenario provided by an embodiment of the present application;

[0050] Figure 12 A schematic diagram of another azimuth angle of a data processing method in a mobile scenario provided by an embodiment of the present application;

[0051] Figure 13 A schematic diagram of an interface of a data processing method in a mobile scenario provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and examples.

[0053] The method provided in the embodiments of the present application can be applied to interactive scenarios involving multiple devices pointing in mobile scenarios, such as scenarios involving riding in a car or riding in an airplane. The following embodiments of the present application use the mobile scenario of riding in a car as an example for description. For other mobile scenarios, please refer to the scenario of riding in a car. It is understood that the application scenarios of the embodiments of the present application can include scenarios involving multiple devices interacting with each other on a mobile carrier, such as using a remote control to point at an in-car screen in a mobile car scenario.

[0054] For example, Figure 1 A schematic diagram showing an application scenario to which the embodiments of the present application are applicable. Figure 1 As shown in 101, the application scenario may include a vehicle screen 10 and a remote controller 20. It should be understood that Figure 1 The number 101 is only an example of the vehicle screen 10 being the main control screen included in the car, and the embodiment of the present application is not limited thereto. For example, the vehicle screen 10 can also be a co-pilot screen, a middle row screen or a rear row screen. In addition, Figure 1 The number 101 is just an example of one vehicle-mounted screen. This application may also include more vehicle-mounted screens, and there is no specific limitation on this.

[0055] In some embodiments, one or more UWB modules may be installed on the vehicle screen 10 and the remote controller 20. Figure 1 A UWB module is installed on the vehicle screen 10 shown in 101, and a UWB module is installed on the remote control 20. Among them, the UWB module can be used to provide UWB communication, distance measurement and other capabilities of the vehicle screen, and generally includes a UWB chip and an antenna.

[0056] Then, the UWB coordinate system of the vehicle-mounted screen 10 is constructed according to the position of the UWB module on the vehicle-mounted screen 10, and the UWB coordinate system of the remote control 20 can also be constructed. Based on the UWB coordinate system of the vehicle-mounted screen 10, the interactive calculation between the vehicle-mounted screen 10 and the remote control 20 is performed, so that the relative posture information and relative position information of the remote control 20 relative to the vehicle-mounted screen 10 can be obtained. Then, based on the remote control 20 relative to each vehicle-mounted screen 10 ( Figure 1 Only one in-vehicle screen 10 is shown in the figure. This allows the user to determine which in-vehicle screen 10 the remote controller 20 is pointing to, as well as the specific location on the in-vehicle screen 10 being pointed to. This allows the cursor of the remote controller 20 to be displayed at the corresponding location on the in-vehicle screen 10. It is understood that the user experience of the remote controller 20 is similar to that of an infrared laser pointer, and its interactive mode is more in line with user habits.

[0057] In addition, the remote controller 20 is generally equipped with one or more IMU chip systems, which can sense the shaking of the human hand. The IMU chip system may include a gyroscope and an accelerometer, or the IMU chip system may include a gyroscope, an accelerometer, and a magnetometer.

[0058] However, in a mobile scenario, when the mobile carrier moves or rotates, the cursor may drift due to factors such as inertia. For example, when a car is driving and encounters a turn or bump, the cursor of the remote control 20 may drift on the in-vehicle screen 10.

[0059] In view of this, an embodiment of the present application provides a data processing method and device in a mobile scenario. In this method, a first device is provided on a mobile carrier, and the first device may include an IMU chip system and a UWB module; wherein, the first device can be understood as a fixed device relative to the mobile carrier and the vehicle-mounted screen. The second device also includes an IMU chip system and a UWB module; wherein, the second device can be understood as a mobile device relative to the mobile carrier and the vehicle-mounted screen, such as a remote control. In this way, through the operation of the IMU chip system and UWB module of each of the first device and the second device, the actual relative spatial relationship measured between the two devices can be taken into account. The change in the relative spatial relationship between the two devices caused by inertia can be calculated, and the change in the target relative spatial relationship between the two devices caused by user operation can be obtained, thereby determining the display result of the cursor based on the target relative spatial relationship, which can reduce the probability of cursor drift and improve the user experience of multi-device pointing interaction in mobile scenarios.

[0060] Among them, the first device can be a vehicle-mounted screen, and the IMU chip system and the UWB module can be integrated in the vehicle-mounted screen; or, the first device can also be a separate device independent of the vehicle-mounted screen, and the IMU chip system and the UWB module are integrated in the separate device. In this case, the first device can be installed relative to the vehicle-mounted screen, for example, next to the vehicle-mounted screen, on the back panel, etc.

[0061] It should be understood that the data processing method in a mobile scenario provided by the embodiment of the present application can be applied to electronic devices that need to be controlled by a remote controller in a mobile scenario, such as a car screen, a car smart device, or other smart devices that may appear in mobile scenarios such as car scenarios in the future. In addition, exemplary embodiments of electronic devices include but are not limited to devices equipped with Or devices with other operating systems.

[0062] The following uses the first device 200 as an example of a separate device independent of the vehicle-mounted screen to introduce the structure of the device to which the embodiments of the present application are applicable.

[0063] like Figure 2 As shown, the first device 200 may include a processor 210, an internal memory 220, a power management module 240, a power supply 241, a communication module 250, a UWB module 260, an IMU chip system 270, etc. Alternatively, although Figure 2 Not shown in the figure, the first device 200 may further include an external memory interface, a touch panel, etc. It is understood that when the first device 200 is a car screen, the car screen may further include an audio module, a speaker, buttons, a display screen, etc.

[0064] The processor 210 may include one or more processing units. For example, the processor 210 may include an application processor (AP), a modem processor, a controller, and memory. Optionally, depending on the different functions of the first device 200, the processor 210 may also include a graphics processing unit (GPU), an image signal processor (ISP), a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors. The controller may be the nerve center and command center of the first device 200. The controller may generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. The processor 210 may also include memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a high-speed cache memory. This memory may store instructions or data that have just been used or are recycled by the processor 210. If the processor 210 needs to use the instruction or data again, it can directly call it from the memory. Repeated access is avoided, the waiting time of the processor 210 is reduced, and the efficiency of the system is improved. In the embodiment of the present application, the processor 210 can be used to process the communication information or measurement information output by the UWB module 260, and can also be used to calculate the position information and posture information of the vehicle screen 10, and can also be used to obtain the position information and posture information of the remote control 20. It can also be used to perform fusion processing of the measurement data of the UWB module 260 and the measurement data of the IMU chip system 270, and calculate the relative position confidence and relative posture information of the remote control 20 relative to the first device 200 or the vehicle screen, so as to determine the cursor display position on the vehicle screen, etc.

[0065] The internal memory 220 can be used to store programs, which include instructions. The processor 210 executes various functional applications and data processing of the first device 200 by running the instructions stored in the internal memory 220. The internal memory 220 may include a program storage area and a data storage area. The program storage area may store an operating system and software code for at least one application. The data storage area may store data generated during the use of the first device 200. In addition, the internal memory 220 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a general-purpose flash memory, etc.

[0066] Power management module 240 is configured to receive input from power source 241. Power source 241 can be a battery or a vehicle's power supply system. Power management module 240 receives power from the battery and / or the vehicle to provide the energy required for normal operation of processor 210, internal memory 220, communication module 250, and UWB module 260.

[0067] The communication module 250 can provide wireless communication functions applied to the first device 200, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc. The communication module 250 can be one or more devices integrating at least one wireless communication function.

[0068] The communication module 250 can receive electromagnetic waves via antenna a, frequency modulate and filter the electromagnetic wave signals, and send the processed signals to the processor 210. The communication module 250 can also receive signals to be transmitted from the processor 210, frequency modulate and amplify them, and convert them into electromagnetic waves for radiation via antenna a.

[0069] The UWB module 260 can provide UWB communication, ranging, etc. applied to the first device 200. The UWB module 260 may include an antenna module b, which includes at least three antennas. Among them, three antennas can be used to achieve three-dimensional positioning of the first device 200. In another possible design, the first device 200 may also include multiple UWB modules 260. For example, when including three UWB modules 260, each UWB module 260 may include an antenna. In this way, the first device 200 may also include at least three antennas, thereby achieving three-dimensional positioning of the first device 200.

[0070] like Figure 2 As shown, antenna module b may include three antennas. All three antennas are integrated into the UWB module 260 of the first device 200, or may be connected to the UWB module 260 of the first device 200. For example, the three antennas may be used to receive UWB signals from other devices, such as the second device 300.

[0071] Optionally, the IMU chip system 270 may include a gyroscope and an accelerometer. Alternatively, the IMU chip system 270 may include a gyroscope, an accelerometer, and a magnetometer. The gyroscope can be used to measure the change in attitude at the current measurement time relative to the previous measurement time; the accelerometer can be used to measure absolute attitude information at the current measurement time; and the magnetometer can be used to measure the strength and direction of the magnetic field.

[0072] It is understandable that Figure 2 The components shown do not constitute a specific limitation on the first device 200. The first device 200 may also include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently.

[0073] The following uses the second device 300 as a remote control as an example to introduce the structure of the device to which the embodiments of the present application are applicable.

[0074] like Figure 3 As shown, the second device 300 may include a processor 310, an internal memory 330, a power management module 340, a battery 341, a communication module 350, a UWB module 360, an IMU chip system 370, and the like.

[0075] Figure 3 Shown in Figure 2 For functions with the same hardware structure, please refer to Figure 2 The introduction in , will not be repeated here. Figure 2 The difference is that the battery management module 340 included in the remote control generally obtains energy from the battery 341.

[0076] It is understandable that Figure 3 The components shown do not constitute a specific limitation on the second device 300. The second device 300 may also include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently.

[0077] It should be understood that in the embodiments of the present application, "at least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c, or a, b and c, where a, b, c can be single or multiple. "Multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0078] In addition, it should be understood that, in the description of this application, words such as "first" and "second" are only used for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

[0079] It should be understood that the hardware structure of the first device can be as follows Figure 2 As shown, the program in the device is stored in the internal memory 220, and the processor 210 can run the program stored in the internal memory 220 to execute the process of the data processing method in a mobile scenario provided by the embodiment of the present application. It should be understood that the hardware structure of the second device can be as follows Figure 3 As shown, the software program in the device is stored in the internal memory 330, and the processor 310 can run the program stored in the internal memory 330 to execute the process of a data processing method in a mobile scenario provided by an embodiment of the present application.

[0080] In order to facilitate understanding of the data processing method in a mobile scenario provided by this application, the following Figures 4A to 12 The content shown introduces the implementation process of the method provided by this application.

[0081] The following first explains the technical terms that may be involved in the embodiments of this application, including:

[0082] (1) UWB coordinate system of the first device 200 (also referred to as "e-system" in the present embodiment)

[0083] like Figure 4A As shown in FIG, a schematic diagram of establishing the UWB coordinate system of the first device 200 provided in an embodiment of the present application is provided. Figure 4A As shown, the first device 200 can be a separate device independent of the vehicle screen, and can be installed above the vehicle screen, for example, near the vehicle screen in the top area of ​​the car. Figure 4A As shown, the vehicle-mounted screen can be located in the middle of the front seats and in front of the middle or rear seats. This application does not limit the installation position of the vehicle-mounted screen. For example, it can also be a main control screen located in the front row of the car.

[0084] based on Figure 2 The UWB module of the first device 200 generally adopts a multi-antenna structure, such as a three-antenna structure, see Figure 4B As shown in the figure, the distribution diagrams of the two three-antenna structures are shown. Based on the three-antenna structure, any antenna in the three-antenna structure can be selected as the origin of the e system. Figure 4B As shown, antenna 0 is selected as the origin of the e system; where antenna 0 is located at the lower left antenna of the three-antenna structure. Figure 4BAs shown, the direction parallel to the horizontal antenna line and pointing to the left edge of the antenna panel can be selected as the X-axis direction of the e-system; the direction of the normal line of the antenna panel can be selected as the Y-axis direction of the e-system; based on the right-hand rule, the Z-axis direction of the e-system can be determined to be perpendicular to the X-axis direction. e 0Y e The plane points in the upward direction.

[0085] It should be noted that the embodiments of the present application do not limit the origin, X-axis direction, Y-axis direction, and Z-axis direction selected when establishing the e-coordinate system. A coordinate system may be established based on the position of the first device 200 and with any fixed reference object as the origin. The e-coordinate system can be used to reflect the relative position and posture of the remote control relative to the UWB module of the first device 200.

[0086] (2) Geographic Coordinate System (also referred to as “G-system” in this application embodiment)

[0087] like Figure 5 FIG2 is a schematic diagram of establishing a geographic coordinate system according to an embodiment of the present application. Figure 5 As shown in FIG, similar to the origin selected for the e-system, antenna 0 can also be selected as the origin of the g-system. Alternatively, other fixed reference objects can be selected as the origin of the g-system.

[0088] Optionally, when the IMU chip system 270 included in the first device 200 includes an accelerometer, a gyroscope, and a magnetometer, the geographical directions of east, north, and sky of the current location can be measured by the IMU chip system 270. Then, the geographical direction of east can be selected as the X-axis direction of the g-system, and the geographical direction of north can be selected as the Y-axis direction of the g-system. Based on the right-hand rule, the Z-axis direction of the g-system can be determined to be perpendicular to the X-axis. g 0Y g The plane points in the upward direction.

[0089] Alternatively, when the first device 200 includes only an accelerometer and a gyroscope but no magnetometer, since it can only measure the geographical direction of the sky but cannot measure the geographical directions of east and north, the Z-axis direction of the g-system can be defined as being consistent with the geographical direction of the sky, and the X-axis direction and the Y-axis direction of the g-system are defined as X and Y. g 0Y g Any two perpendicular directions chosen on a plane that conform to the right-hand rule.

[0090] It should be noted that the embodiments of the present application do not limit the origin, X-axis direction, Y-axis direction, and Z-axis direction selected when establishing the G-coordinate system. A coordinate system may be established based on the position of the first device 200 and with any fixed reference object as the origin. It is understood that since the G-coordinate system generally does not change over a short period of time, the G-coordinate system can be used to facilitate calculation of the relative posture information and relative position information of the second device relative to the first device.

[0091] (3) Remote Control Coordinate System (also referred to as “B System” in this embodiment)

[0092] like Figure 6 As shown in FIG, a schematic diagram of establishing a remote control coordinate system according to an embodiment of the present application is provided. Figure 6 As shown, the center of the remote control can be selected as the origin of the remote control coordinate system; or, if the position of the remote control's IMU chip system is stored, the center of the IMU chip system can also be selected as the origin of the remote control coordinate system. Figure 6 As shown, the short side of the remote control can be selected as the X-axis direction of the b system; and the long side of the remote control can be selected as the Y-axis direction of the b system; based on the right-hand rule, the Z-axis direction of the b system can be determined to be perpendicular to the X-axis direction. b 0Y b The plane points upward

[0093] It should be noted that the embodiments of the present application do not limit the origin, X-axis direction, Y-axis direction, and Z-axis direction selected when establishing the B-coordinate system. A coordinate system may be established based on the second device 300 and using any fixed reference object as its origin. The fixed reference object may also be, for example, the center of gravity of the second device 300. The remote control coordinate system can be used to reflect remote control posture information and can be combined with the first device 200 to reflect the relative posture information of the remote control relative to the vehicle-mounted screen.

[0094] (4) Screen Coordinate System (hereinafter referred to as “S-system”) and Display Coordinate System (hereinafter referred to as “D-system”)

[0095] like Figure 7A As shown in FIG, it is a schematic diagram of establishing the screen coordinate system provided by the embodiment of the present application. Figure 7A As shown in , the center of the top edge of the car screen can be selected as the origin of the s system. Figure 7A As shown, the left direction of the top edge of the car screen can be selected as the X-axis direction of the s-system; and the direction perpendicular to the positive direction of the car screen can be selected as the Y-axis direction of the s-system; based on the right-hand rule, the Z-axis direction of the s-system can be determined to be perpendicular to the X-axis direction. s 0Y s The direction in which the plane points upwards can also be understood as the vertical edge of the car screen pointing upwards.

[0096] It should be noted that the embodiments of this application do not limit the origin, X-axis direction, Y-axis direction, and Z-axis direction selected when establishing the S-coordinate system. A coordinate system may be established based on the vehicle screen and with any fixed reference object as its origin. The fixed reference object may be, for example, the leftmost point at the top of the vehicle screen. The screen coordinate system can be used to reflect the posture information of the vehicle screen and can be combined with the remote control to reflect the relative posture information of the remote control with respect to the vehicle screen.

[0097] like Figure 7B As shown in FIG, it is a schematic diagram of establishing the display coordinate system provided by the embodiment of the present application. The d system is a two-dimensional coordinate system, and the upper left corner of the vehicle screen can be selected as the origin of the d system. Figure 7B As shown, the right direction of the top edge of the car screen can be selected as the X-axis direction of the d-series; and the direction perpendicular to the vertical direction of the car screen can be selected as the Y-axis direction of the d-series.

[0098] It should be noted that the embodiments of this application do not limit the origin, X-axis direction, and Y-axis direction selected when establishing the d-coordinate system. A coordinate system may be established based on the vehicle screen and with any fixed reference object as the origin. The fixed reference object may also be, for example, the rightmost point at the top of the vehicle screen. The display coordinate system can be used to determine the display position of the remote control cursor on the vehicle screen.

[0099] For example, the origin of the s system and the origin of the d system can also be selected as the same origin.

[0100] (5) Attitude angle

[0101] The attitude angle is determined by the rotation relationship between the two coordinate systems. Taking the transformation of the remote control coordinate system (b system) relative to the UWB coordinate system (e system) as an example, the rotation relationship between the b system and the e system can be expressed by the Euler angle, which includes the heading angle ψ, the pitch angle and the roll angle θ. Figure 8 , is a schematic diagram of the attitude angles of the b-system and the e-system provided in the embodiment of the present application.

[0102] The heading angle ψ is the Y direction of the b system b Axis X in e-system e OY e Projection on the surface, and Y e The angle between the axes is positive when the remote controller yaws to the right;

[0103] Pitch angle Y of b series b Axis and X in e system e OY e The angle between the two faces is positive when the remote control is raised;

[0104] The roll angle θ is the Z of the b frame b Axis and contains moving vector Y b The angle between the plumb planes of the axes is positive when the remote control tilts to the right.

[0105] In addition, the relative attitude angle of the two coordinate systems can also be expressed by the attitude matrix, as shown in the following formula 1, which can be used to express the attitude matrix of the b system in the e system:

[0106]

[0107] The relative attitude angles of two coordinate systems can be used to determine the attitude of one device relative to another. For example, the attitude of a remote control relative to a car screen can be determined, thereby determining the display position of the remote control cursor on the car screen.

[0108] For example, the conversion between the b-frame and the e-frame can also be achieved by first converting the b-frame to the g-frame, and then converting from the g-frame to the e-frame. This application does not limit the specific conversion method between the two coordinate systems. It is understood that the attitude angles between other coordinate systems, such as the conversion of the UWB coordinate system (e-frame) relative to the geographic coordinate system (g-frame), can be referred to. Figure 8 The introduction content shown is not repeated in this application.

[0109] Figure 9 This is a flow chart of a data processing method in a mobile scenario provided by an embodiment of the present application. Figure 9 As shown, the method can be applied to a multi-device pointing interaction scenario in a mobile scenario, and the scenario can include at least a first device 200 and a second device 300. The method can include the following steps:

[0110] In step 901A, the first device 200 measures and obtains first data through the UWB module 260 , and measures and obtains second data through the IMU chip system 270 .

[0111] like Figure 2 As shown, the first device 200 may include a UWB module 260 and an IMU chip system 270 , so after the first device 200 is powered on, the UWB module 260 and the IMU chip system 270 may start measurement.

[0112] Optionally, the UWB module 260 and the IMU chip system 270 may be integrated hardware structures or chips, or may be separate hardware structures or chips. The specific composition of the UWB module 260 and the IMU chip system 270 is not limited in the embodiments of the present application.

[0113] Optionally, based on the introduction of the e-system in the aforementioned embodiment, the first data and the second data can be represented based on the e-system. It can be understood that the first relative spatial relationship between the UWB module 260 and the IMU chip system 270 is fixed and known; wherein the first relative spatial relationship includes a first relative position relationship and a first relative posture relationship. In this way, based on the first data measured by the UWB module 260, and the first data can be represented by the e-system; and based on the relatively fixed first relative spatial relationship between the UWB module 260 and the IMU chip system 270, the second data can also be represented by the e-system.

[0114] Alternatively, based on the introduction to the g-frame in the aforementioned embodiment, the first data and the second data can also be represented based on the g-frame. It is understandable that the second relative spatial relationship of the UWB module 260 relative to the geographic coordinate system is fixed and known; wherein the second relative spatial relationship includes a second relative position relationship and a second relative attitude relationship; the third relative spatial relationship of the IMU chip system 270 relative to the geographic coordinate system is also fixed and known; wherein the third relative spatial relationship includes a third relative position relationship and a third relative attitude relationship. In this way, the first data and the second data can also be represented by the g-frame.

[0115] Alternatively, in other optional examples, the first data and the second data may be represented based on other coordinate systems. In addition, the first data and the second data may also be represented by different coordinate systems, which is not limited in this application.

[0116] In step 901B, the second device 300 obtains third data by measuring through the UWB module 360 ​​, and obtains fourth data by measuring through the IMU chip system 370 .

[0117] like Figure 3 As shown, the second device 300 includes a UWB module 360 ​​and an IMU chip system 370 , so after the second device 300 is powered on, the UWB module 360 ​​and the IMU chip system 370 can start measurement.

[0118] Optionally, the UWB module 360 ​​and the IMU chip system 370 may be integrated hardware structures or chips, or may be separate hardware structures or chips. The specific composition of the UWB module 360 ​​and the IMU chip system 370 is not limited in the embodiments of the present application.

[0119] Optionally, based on the introduction of the B system in the aforementioned embodiment, the third data and the fourth data can be represented based on the B system. It can be understood that the fourth relative spatial relationship between the UWB module 360 ​​and the IMU chip system 370 is fixed and known; wherein, the fourth relative spatial relationship includes a fourth relative position relationship and a fourth relative posture relationship. Considering that the position of the second device is relatively unstable, the first priority can be selected to represent the third data and the fourth data based on the B system, but the embodiment of the present application is not limited to representing the third data and the fourth data based on the B system.

[0120] Step 902: The first device 200 sends a first message to the second device 300; wherein the first message includes first data and second data.

[0121] Exemplarily, the first message may be sent using, but not limited to, the following communication methods: UWB, Bluetooth, infrared, NFC, star flash, Wi-Fi, and other communication methods.

[0122] It should be noted that, in the embodiment of the present application, the second device 300 is used as an example to process the first data, the second data, the third data, and the fourth data, but the processing is not limited to the second device 300. For example, the first device 200 can also be used for processing. In this example, the second device 300 can send the third data and the fourth data to the first device. For another example, a third device other than the first device 200 and the second device 300 can also be used for processing. In this example, the first device 200 can send the first data and the second data to the third device, and the second device 300 can send the third data and the fourth data to the third device. Optionally, the third device can be a device or a cloud server, which is not limited in this application.

[0123] In step 903, the second device 300 calculates first updated data of the second device 300 relative to the first device 200 based on the second data and the fourth data. The first updated data is used to reflect the update of the relative spatial relationship between the first device 200 and the second device 300 due to the movement of the mobile carrier. It will be understood that due to the movement of the mobile carrier, based on the principle of inertial navigation, the first device 200 and the second device 300 generate inertial offset data. Since the first device 200 and the second device 300 are located at different positions on the mobile carrier, the degree of inertial offset varies, resulting in a difference between the inertial offset data, i.e., the first updated data.

[0124] It can be understood that based on the second data measured in real time by the first device 200, the position and posture of the first device 200 relative to the g-system can be updated, that is, the inertial offset data of the first device 200 can be obtained. Based on the fourth data measured in real time by the second device 300, the position and posture of the second device 300 relative to the g-system can be updated, that is, the inertial offset data of the second device 300 can be obtained. In a mobile scenario, due to the influence of factors such as the movement, rotation, and bumps of the mobile carrier, the update degree of the position and / or posture of the first device 200 and the second device 300 is different, that is, there is a difference in the inertial offset data of the second device relative to the first device. For example, in a scenario where a car is rotating, the first device 200 is closer to the front of the car than the second device 300, so the inertial offset degree of the first device 200 is greater than the inertial offset degree of the second device 300.

[0125] In addition, it can be understood that in this scenario, the data measured by the IMU chip system can be used to indicate the update of the position and / or attitude of the device due to the inertial navigation principle, and this update of the position and / or attitude will cause cursor drift. Therefore, by accurately obtaining the first update data, it is possible to calculate a more realistic relative spatial relationship between the first device and the second device based on the first update data, so that after the fusion processing of step 905, the change in the relative spatial relationship caused by the influence of factors such as the movement, rotation, and bumps of the mobile carrier can be offset, thereby reducing the occurrence of cursor drift and ensuring the user experience of multi-device pointing interaction.

[0126] In the embodiments of the present application, the first update data mainly includes first posture update data and first position update data. In the following embodiments, taking the second device 300 as a remote control and the update data represented based on the g-system as an example, the first update data calculated may include first posture update data calculated based on the posture update algorithm and first position update data calculated based on the position update algorithm.

[0127] The calculation process of the attitude update algorithm and the position update algorithm can be found in the following:

[0128] (1) Posture update algorithm, used to obtain updated posture data of the remote controller.

[0129] Assume that the attitude rotation matrix of the g system relative to the b system at time m is expressed as It can be obtained by the following formula 2-1:

[0130]

[0131] In formula 2-1, time m can be understood as the time corresponding to the mth measurement, and time m-1 can be understood as the time corresponding to the m-1th measurement. It is used to indicate the attitude rotation matrix of the g system relative to the b system at time m-1. It has been calculated at time m-1 and is a known number.

[0132] In formula 2-1 It is used to indicate the attitude rotation matrix of the g-frame at time m relative to the g-frame at time m-1. In addition, in the mobile scenario, since the geographical location of the mobile carrier changes little in a short period of time, it can be considered that the g-frame has no rotation, so Where I represents the identity matrix. It can also be expressed as the following formula 2-2:

[0133]

[0134] In Formula 2-1 or Formula 2-2 It is used to indicate the attitude rotation matrix of the b system at time m relative to the b system at time m-1. Generally depends on the rotation vector Φ output by the remote controller's IMU chip system at time m m , for example based on Φ m , It can be expressed as the following formula 2-3:

[0135]

[0136] Among them, the and Φ m The relationship between them can be found in the introduction of the relevant technology, and this application will not elaborate on it.

[0137] In formula 2-3, Φ m Under the 2-sample condition, it can be expressed as follows:

[0138]

[0139] Formula 2-4 introduces Φ m The method of obtaining Δθ can be found in the introduction of related technologies, and this application will not elaborate on this. m (1) and Δθ m (2) and The angular increment in time. And Δθ m is Δθ m (1) and Δθ m The total increment of (2) can be expressed as the following formula 2-5:

[0140] Δθ m =Δθ m (1)+Δθ m (2)Formula 2-5

[0141] Δθ in Formula 2-4 and Formula 2-5 m (1) and Δθ m (2) It can be measured by the IMU chip system of the remote control.

[0142] (2) Position update algorithm, used to obtain updated position data of the remote control.

[0143] Assume that the position rotation matrix of the g system relative to the b system at time m is expressed as It can be expressed by the following formula 3-1:

[0144]

[0145] T in Formula 3-1 m It is used to represent the time difference between time m and time m-1. For example, the time difference between the time corresponding to the mth measurement and the time corresponding to the m-1th measurement can be 1 second, 10 seconds, etc. Indicates that the time obtained is the middle time between time m and time m-1 (that is, The speed value at the time.

[0146] In formula 3-1 It can be expressed by the following formula 3-2:

[0147]

[0148] Among them, formula 3-2 is used to express The speed value at time m can be obtained by averaging the speed value at time m and the speed value at time m-1.

[0149] In addition, in Formula 3-2 It can be calculated using the speed update algorithm. The speed update algorithm is used to obtain the remote controller's speed information to better determine the updated position information. The speed update algorithm formula can be shown in the following formula 3-2-1:

[0150]

[0151] Among them, in formula 3-2-1 The rotation matrix used to represent the acceleration vector of the remote control b system relative to the g system at time m can be measured by the IMU chip system of the remote control; wherein, the IMU chip system generally includes an accelerometer, and the acceleration vector can be measured by the accelerometer. It can be expressed as the following formula 3-2-2:

[0152]

[0153] In formula 3-2-2, based on the introduction in the previous embodiment, therefore It can also be expressed as the following formula 3-2-3:

[0154]

[0155] Among them, Formula 3-2-2 or Formula 3-2-3 is used to express the expression of the acceleration vector. Please refer to the introduction in the relevant technology, and this application will not elaborate on it.

[0156] Based on the above introduction to the attitude update algorithm and position update algorithm, we can not only calculate the attitude rotation matrix of the remote control and the position rotation matrix The posture rotation matrix of the first device 200 can also be calculated and the position rotation matrix Then, the first update data of the second device 300 relative to the first device 200 can be calculated. The first update data may include: the first posture update data of the second device 300 relative to the first device 200 and the first position update data of the second device 300 relative to the first device 200 in, It can be expressed by the following formula 4-1: It can be expressed by the following formula 4-2:

[0157]

[0158]

[0159] The first posture update data shown in Formula 4-1 and the first position update data shown in Formula 4-2 are caused by factors such as the movement of the mobile carrier. Therefore, after the second posture update data and the second position update data included in the second update data are fused in step 905, position cancellation and posture cancellation can be achieved.

[0160] Therefore, through the measurements of the IMU chip system included in the first device 200 and the IMU chip system included in the second device 300, it is possible to consider the deviation in the inertial offset of the two devices due to the movement of the mobile carrier, which in turn causes the cursor offset. The cursor offset can then be offset through position cancellation and posture cancellation to ensure that the cursor display can more accurately reflect the user operation.

[0161] In step 904, the second device 300 calculates second updated data of the second device 300 relative to the first device 200 based on the first data and the third data; the second updated data is used to reflect the update of the relative spatial relationship between the first device 200 and the second device 300. The second updated data includes second posture update data and second position update data. It can be understood that the second updated data is an update of the relative spatial relationship between the first device 200 and the second device 300 caused by the user operation and the movement of the mobile carrier. It can also be understood that the second updated data is used to reflect the actual relative spatial relationship between the first device 200 and the second device 300, and is obtained through measurement.

[0162] The calculation of the second update data in the embodiment of the present application will not be described in detail, and reference may be made to the calculation process of the first update data in step 903 .

[0163] It should be noted that the embodiment of the present application does not limit the execution order of step 903 and step 904.

[0164] In step 905, the second device 300 fuses the first updated data and the second updated data to calculate relative spatial data. The relative spatial data reflects the target relative spatial relationship between the first device 200 and the second device 300. The target relative spatial relationship includes a target relative position relationship and a target relative posture relationship. The target relative position relationship reflects the relative position of the first device 200 and the second device 300 in space, and the target relative posture relationship reflects the relative posture of the first device 200 and the second device 300 in space.

[0165] Exemplarily, the relative spatial data can be represented by third posture update data and third position update data; wherein, the third posture update data is obtained after offsetting the second posture update data based on the first posture update data, and the third position update data is obtained after offsetting the second position update data based on the first position update data.

[0166] In the embodiment of the present application, the fusion processing can eliminate the influence of the first update data on the second update data, thereby obtaining an update that can reflect the relative spatial relationship between the first device 200 and the second device 300 caused by the user operation. It can also be understood that the relative spatial data is used to reflect the target relative spatial relationship calculated in the embodiment of the present application, and is calculated, not measured.

[0167] In addition, UWB measurements can be directly converted to and location The first update data and the second update data can be fused using a Kalman filter or other method to obtain the accurate posture and position of the remote control in the UWB coordinate system.

[0168] Step 906: The second device 300 sends the relative spatial data to the vehicle screen.

[0169] In one possible scenario, the first device 200 is an independent device relative to the vehicle screen, and the second device 300 can send the relative spatial data to the vehicle screen.

[0170] In another possible scenario, the first device 200 can also be a vehicle-mounted screen, which can also be understood as the IMU chip system and the UWB module integrated in the vehicle-mounted screen. In this scenario, the second device 300 can send relative spatial data to the first device 200, that is, the vehicle-mounted screen.

[0171] Step 907: The vehicle screen displays a cursor according to the relative space data.

[0172] For example, the vehicle screen can convert the relative spatial data into the s system for representation first, and then into the d system for representation, so that the calculated cursor position of the second device 300 can be projected and displayed on the vehicle screen.

[0173] As another example, the position of the cursor displayed on the vehicle screen can be calculated in the UWB coordinate system based on the range and posture of the vehicle screen in the UWB coordinate system and the posture and position of the remote control in the UWB coordinate system.

[0174] In the method provided in the embodiment of the present application, by deploying a UWB module and an IMU chip system on the first device 200, real-time measurement of the spatial data of the first device 200 can be achieved; and by simultaneously deploying a UWB module and an IMU chip system on the second device 300, real-time measurement of the spatial data of the second device 300 can also be achieved. In this way, the absolute spatial information measured by the UWB modules of the first device 200 and the second device 300 can be used to obtain second updated data between the first device 200 and the second device 300. The second updated data is caused by various factors such as the movement of the mobile carrier and user operations. In addition, the relative spatial information measured by the IMU chip systems of the first device 200 and the second device 300 can also be used to obtain first updated data between the first device 200 and the second device 300. The first updated data is caused by inertial factors such as the movement of the mobile carrier. In this way, through the fusion processing of the first update data and the second update data, the first update data included in the second update data can be offset, so that the change in the relative spatial relationship of the second device 300 with respect to the first device 200 caused by the user operation can be obtained, and the probability of abnormal cursor offset can be reduced, and the degree of cursor offset can be reduced, thereby ensuring the user experience of multi-device pointing interaction in a mobile scenario.

[0175] based on Figure 9 The process shown, Figure 10 This is a flow chart of a time synchronization method for a data processing method in a mobile scenario provided by an embodiment of the present application. This process can be performed before step 902 and may include the following steps:

[0176] Step 1001: The first device 200 sends a synchronization frame to the second device 300 and starts timing.

[0177] Step 1002: The second device 300 receives a synchronization frame from the first device 200 and starts timing.

[0178] Step 1003: The second device 300 sends a return frame to the first device 200 and stops timing to obtain a timing result T2.

[0179] Step 1004: The first device 200 receives a return frame from the second device 300 and stops timing to obtain a timing result T1.

[0180] Optionally, in step 1005A, when the return frame includes T2, the first device 200 may determine the data transmission delay T based on T1 and T2. delay ; which can be expressed by the following formula 5:

[0181]

[0182] Alternatively, in step 1005B, when the return frame does not include T2, the first device 200 may send T1 to the second device 300.

[0183] Step 1005C: After the second device 300 receives T1 sent from the first device 200, it can determine the data transmission delay T based on T1 and T2. delay .

[0184] Step 1006: The first device 200 and the second device 300 communicate with each other based on T delay It is understood that the first device 200 or the second device 300, whichever has a faster time, can synchronize time based on T delay Delay processing.

[0185] pass Figure 10 The process shown in FIG can reduce the delay caused by system scheduling and air interface transmission during data transmission, thereby improving the Figure 9 The processing accuracy of the process shown ensures the processing effect of cursor drift.

[0186] Take the second device 300 as a remote controller as an example. Figure 11 As shown in , the remote controller is held still while driving, and the intercepted data is the data when the car makes a 90° turn. Figure 11 As shown in the interface 1101 in the figure, the azimuth angles of the remote control side and the first device side are respectively calculated. It can be seen from the interface 1101 that the azimuth angles are offset by 90°. Figure 11 As shown in the interface 1102 in the figure, the azimuth angles of the remote control side and the first device side are obtained after data processing using the method provided in the embodiment of the present application. It can be seen from the interface 1102 that the 90° offset is offset. Figure 11 As shown in the interface 1103, it is a stretched diagram on the vertical axis compared to the interface 1102. It can also be seen from the interface 1103 that the 90° azimuth angle is offset.

[0187] Still taking the second device 300 as a remote controller as an example, Figure 12 This is another azimuth diagram of a data processing method in a mobile scenario provided by an embodiment of the present application. Figure 12 As shown in FIG, the remote control is periodically pointed left and right during driving, and the intercepted data is the data when the car makes a 90° turn left and right during driving. Figure 12 As shown in the interface 1201, the azimuth angles of the remote control side and the first device side are respectively calculated. From the interface 1201, it can be seen that the azimuth angles have multiple 90° offsets and the azimuth angles also have small fluctuations. Figure 12As shown in the interface 1202, it is the azimuth angle of the remote control side and the first device side obtained after data processing using the method provided in the embodiment of the present application. It can be seen from the interface 1202 that multiple 90° azimuth angle offsets are offset, and it can be seen that after the offset, the left and right shaking of the remote control relative to the first device is clearly restored.

[0188] Still taking the second device 300 as a remote controller as an example, Figure 13 A schematic diagram of an interface of a data processing method in a mobile scenario provided in an embodiment of the present application. Figure 13 The scene shown is that the remote controller is held still while driving. Figure 13 As shown in the interface 1301 in FIG, there is cursor drift before the method provided in the embodiment of the present application is adopted, such as Figure 13 As shown in interface 1302, after adopting the method provided in the embodiment of the present application, the degree of cursor drift is reduced, and the cursor can be relatively stable on the in-vehicle screen. Therefore, the method provided in the embodiment of the present application can more accurately reflect the pointing effect in response to user operations, ensuring a multi-device pointing interaction experience.

[0189] Based on the above embodiments, the present application further provides a first device, which includes multiple functional modules; the multiple functional modules interact with each other to implement the functions performed by the first device in each method described in the embodiments of the present application. The multiple functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the multiple functional modules can be arbitrarily combined or divided based on the specific implementation. Figure 9 In the embodiment shown, the first device 200 performs steps 901A and 902. Alternatively, Figure 10 In the embodiment shown, the first device 200 performs steps 1001, 1004, 1005A, and 1006, or performs Figure 10 In the illustrated embodiment, the first device 200 performs steps 1001, 1004, 1005B, and 1006.

[0190] Based on the above embodiments, the present application further provides a first device, which includes at least one processor and at least one memory, wherein the at least one memory stores a program, and when the first device runs the program, the at least one processor executes the functions executed by the first device in each method described in the embodiments of the present application. Figure 9 In the embodiment shown, the first device 200 performs steps 901A and 902. Alternatively, Figure 10 In the embodiment shown, the first device 200 performs steps 1001, 1004, 1005A, and 1006, or performs Figure 10In the illustrated embodiment, the first device 200 performs steps 1001, 1004, 1005B, and 1006.

[0191] Based on the above embodiments, the present application also provides a second device, which includes multiple functional modules; the multiple functional modules interact with each other to implement the functions performed by the second device in each method described in the embodiments of the present application. The multiple functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the multiple functional modules can be arbitrarily combined or divided based on the specific implementation. Figure 9 In the embodiment shown, the second device 300 performs steps 901B, 902 to 906. Alternatively, Figure 10 In the embodiment shown, the second device 300 performs steps 1002, 1003, and 1006, or performs Figure 10 In the illustrated embodiment, the second device 300 performs steps 1002 , 1003 , 1005B, 1005C, and 1006 .

[0192] Based on the above embodiments, the present application further provides a second device, which includes at least one processor and at least one memory, wherein the at least one memory stores a program, and when the second device runs the program, the at least one processor executes the functions executed by the second device in each method described in the embodiments of the present application. Figure 9 In the embodiment shown, the second device 300 performs steps 901B, 902 to 906. Alternatively, Figure 10 In the embodiment shown, the second device 300 performs steps 1002, 1003, and 1006, or performs Figure 10 In the illustrated embodiment, the second device 300 performs steps 1002 , 1003 , 1005B, 1005C, and 1006 .

[0193] Based on the above embodiments, the present application further provides a data processing system for a mobile scenario. The data processing system for a mobile scenario may include the first device and the second device described in the above embodiments. Optionally, the data processing system for a mobile scenario may further include a third device. Alternatively, the data processing system for a mobile scenario may further include an in-vehicle screen.

[0194] Based on the above embodiments, the present application also provides a program product, which includes: a program (also called code, or instructions), which, when executed, enables a device or computer to execute the methods described in the embodiments of the present application.

[0195] Based on the above embodiments, the present application also provides a readable storage medium, in which a program is stored. When the program is executed by a device or a computer, the device or the computer executes the methods described in the embodiments of the present application.

[0196] Based on the above embodiments, the present application further provides a chip, which is used to read the program stored in the memory to implement the various methods described in the embodiments of the present application.

[0197] Based on the above embodiments, the present application provides a chip system, which includes a processor for supporting a device to implement the various methods described in the embodiments of the present application. In one possible design, the chip system also includes a memory, which is used to store necessary programs and data. The chip system can be composed of chips, or it can include chips and other discrete devices. Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects.

[0198] Obviously, those skilled in the art may make various modifications and variations to this application without departing from the scope of protection of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A data processing method in a mobile scenario, characterized in that: The method comprises: Acquire first data and second data of the first device; Acquire third data and fourth data of the second device; Obtaining first updated data according to the second data and the fourth data; wherein the first updated data is used to reflect a difference in inertial offset data of the second device relative to the first device; Obtaining second updated data based on the first data and the third data; the second updated data is used to reflect the measured actual relative spatial relationship of the second device with respect to the first device; The second updated data is fused based on the first updated data to obtain relative spatial data, where the relative spatial data is used to reflect a target relative spatial relationship between the second device and the first device.

2. The method according to claim 1, characterized in that The first update data includes first posture update data and first position update data; the second update data includes second posture update data and second position update data; the relative space data includes third posture update data and third position update data; The second updated data is fused based on the first updated data to obtain relative spatial data, including: Obtaining the third posture update data according to a difference between the first posture update data and the second posture update data; The third location update data is obtained according to a difference between the first location update data and the second location update data.

3. The method according to claim 1 or 2, characterized in that The second data includes first posture data and first position data; the fourth data includes second posture data and second position data; The obtaining of first updated data according to the second data and the fourth data includes: Obtaining first posture update data according to the first posture data and the second posture data, where the first posture update data is used to reflect a posture change of the second device relative to the first device due to inertia; First position update data is obtained according to the first position data and the second position data, where the first position update data is used to reflect a position change of the second device relative to the first device due to inertia.

4. The method according to any one of claims 1 to 3, characterized in that The first data includes third posture data and third position data; the third data includes fourth posture data and fourth position data; The obtaining of second updated data according to the first data and the third data includes: Obtaining second posture update data according to the third posture data and the fourth posture data, where the second posture update data is used to reflect an actual posture change of the second device relative to the first device; Second location update data is obtained according to the third location data and the fourth location data, where the second location update data is used to reflect actual location changes of the second device relative to the first device.

5. The method according to any one of claims 1 to 4, characterized in that The first device is a fixed device relative to the mobile carrier, and the second device is a mobile device relative to the mobile carrier.

6. The method according to claim 5, characterized in that The target-relative spatial relationship is used to reflect the movement of the second device relative to the first device, where the movement is caused by a user operation.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Based on the relative spatial data, a position of a display cursor of a target screen is determined; the target screen is the screen pointed to by the second device.

8. The method according to any one of claims 1 to 7, characterized in that Before obtaining the first data and the second data of the first device, the method further includes: Perform time synchronization between the first device and the second device.

9. The method according to any one of claims 1 to 8, characterized in that The first data is measured by an ultra-wideband (UWB) module of the first device, and the second data is measured by an inertial measurement unit (IMU) chip system of the first device; The third data is measured by the UWB module of the second device, and the fourth data is measured by the IMU chip system of the second device.

10. A device, characterized in that The system comprises at least one processor coupled to at least one memory, and the at least one processor is configured to read a program stored in the at least one memory to execute the method according to any one of claims 1 to 9.

11. A readable storage medium, characterized in that: The readable storage medium stores instructions, which, when executed on a device, enable the device to execute the method according to any one of claims 1 to 9.

Images