Vehicle driving accompanying method and device

Abstract

The invention provides a vehicle driving accompanying method and device. The method comprises the steps that a first vehicle driving accompanying device receives a first data stream which can be used for bearing video data of a first vehicle at a first time and driving data of the first vehicle at the first time, and then sends a second data stream which can be used for bearing the video data and the driving data, the second data stream is related to the first data stream, the video data can be used for restoring a video picture of the first vehicle at the first time, and the driving data can be used for restoring a driving state of the first vehicle at the first time. By synchronously transmitting the video data of the first vehicle at the first time and the driving data of the first vehicle at the first time, synchronous restoration of the video picture and the driving state of the first vehicle at the first time can be realized, so that a remote user can experience the driving process of the first vehicle personally on the scene; the remote user can share the whole journey synchronously with the driver of the first vehicle.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on March 15, 2024, with application number 202410302014.2 and application name “A vehicle driving accompaniment method and device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of vehicle networking technology, and in particular to a vehicle driving accompaniment method and device. Background Art

[0004] Communicating with family or friends while driving (e.g., chatting on the phone) is a fundamental and important use case. This has evolved from using mobile phones for individual voice chats, to using mobile phones for WeChat video chats, and now using in-car video apps and external cameras for on-the-go chats. However, current remote chats using external cameras only offer two-dimensional experiences: audio and camera input. Essentially, users are still stuck in the era of phone software, unable to better communicate while driving. Therefore, how to enable the other party (or remote user) to experience the actual driving process and communicate with the vehicle user while the vehicle is in motion has become an urgent issue that needs to be addressed. Summary of the Invention

[0005] The present application provides a vehicle driving companion method and device for enabling a remote user to experience the vehicle driving process in an immersive way.

[0006] In a first aspect, the present application provides a vehicle driving companion method, which can be applied to the electric vehicle field or any other connected vehicle field. Optionally, the method can be executed by a first vehicle driving companion device or a module of the first vehicle driving companion device (such as a processor, processing unit, chip, chip system, or circuit). It should be understood that the method can also be implemented by a logical node, logic module, or software that can implement all or part of the functions of the first vehicle driving companion device. Exemplarily, the following uses the first vehicle driving companion device executing the vehicle driving companion method as an example. The method can include the following steps: the first vehicle driving companion device receives a first data stream, wherein the first data stream is used to carry video data of a first vehicle at a first time and driving data of the first vehicle at a first time. Thereafter, the first vehicle driving companion device sends a second data stream, wherein the second data stream can be used to carry video data of the first vehicle at a first time and driving data of the first vehicle at a first time. The second data stream is related to the first data stream, and the video data of the first vehicle at the first time is used to restore the video image of the first vehicle at the first time, and the driving data of the first vehicle at the first time is used to restore the driving state of the first vehicle at the first time.

[0007] In this method, by synchronously transmitting the video data of the first vehicle at the first time and the driving data of the first vehicle at the first time, the video image and driving status of the first vehicle at the first time can be synchronously restored (or it can be understood as ensuring that the presentation of the video image is consistent with the presentation of the driving status or maintaining synchronous matching), so that the remote user (or remote user) can experience the driving process of the first vehicle in an immersive way, so that the remote user can synchronously share the entire journey with the driver of the first vehicle, further enabling the remote user to have a good companionship and communication experience, and the remote user can also communicate well with the driver of the first vehicle in the process of sharing the vehicle journey with the driver.

[0008] In a possible implementation, the first vehicle driving companion device sends the second data stream, including: the first vehicle driving companion device may send the second data stream to the first terminal device.

[0009] In the above implementation method, by sending the video data of the first vehicle at the first time and the driving data of the first vehicle at the first time to the first terminal device (such as a smart phone or tablet computer, etc.), the first terminal device can present the video image of the first vehicle at the first time based on the video data, and display the driving status of the first vehicle at the first time or restore the driving process of the first vehicle at the first time based on the driving data.

[0010] In one possible implementation, the first vehicle driving companion device sends a second data stream, including: the first vehicle driving companion device can render the video data of the first vehicle at the first time to obtain video rendering data, and can convert the driving data of the first vehicle at the first time to obtain motion simulation data. After that, the first vehicle driving companion device can send the second data stream to the second terminal device, wherein the second data stream may include video rendering data and motion simulation data, the video rendering data can be used by the head-mounted device included in the second terminal device to restore the video picture of the first vehicle at the first time, and the motion simulation data can be used by the simulated seat included in the second terminal device to restore the driving state of the first vehicle at the first time.

[0011] In the above implementation method, by sending the video rendering data after the video data is rendered and the motion simulation data after the driving data is converted to the second terminal device, this can facilitate the head-mounted device in the second terminal device to restore the video image of the first vehicle at the first time in a timely and effective manner according to the video rendering data, and can facilitate the simulated seat in the second terminal device to restore the driving state (or can be understood as the motion state) of the first vehicle at the first time in a timely and effective manner according to the motion simulation data.

[0012] In one possible implementation, the driving data of the first vehicle at the first time may include at least one of the following: movement data of the first vehicle in a first direction, movement data of the first vehicle in a second direction, or movement data of the first vehicle in a third direction; wherein the first direction, the second direction, and the third direction are different;

[0013] The first vehicle driving accompanying device converts the driving data of the first vehicle at a first time to obtain motion simulation data, including:

[0014] The first vehicle driving companion device may convert the motion data of the first vehicle in the first direction to obtain first motion simulation data, wherein the first motion simulation data may be used to restore (or simulate) the motion state of the first vehicle at the first time and in the first direction; or

[0015] The first vehicle driving companion device may convert the motion data of the first vehicle in the second direction to obtain second motion simulation data, wherein the second motion simulation data may be used to restore the motion state of the first vehicle at the first time and in the second direction; or

[0016] The first vehicle driving accompanying device can convert the motion data of the first vehicle in the third direction to obtain third motion simulation data, wherein the third motion simulation data can be used to restore the motion state of the first vehicle at the first time and in the third direction.

[0017] In the above implementation, by converting the motion data of the first vehicle in different directions into corresponding motion simulation data, the second terminal device can timely and effectively restore the motion state of the first vehicle in different directions, providing the remote user with an immersive experience of the first vehicle's motion changes in different directions. Furthermore, this implementation can simply and efficiently convert the complex motion data of the first vehicle in different directions into the motion simulation data of the first vehicle in different directions, and can enable the second terminal device to provide the remote user with an immersive driving experience (or what can be called a motion experience, i.e., the remote user can immersively experience the motion changes of the first vehicle in different directions) based on the motion simulation data of the first vehicle in different directions.

[0018] In one possible implementation, the first vehicle driving companion device converts the motion data of the first vehicle in the first direction to obtain first motion simulation data, including: the first vehicle driving companion device can determine the first level value corresponding to the motion data of the first vehicle in the first direction, and can determine the first numerical value used to indicate the first direction. Afterwards, the first vehicle driving companion device can determine the first motion simulation data based on the first level value and the first numerical value.

[0019] In the above implementation, by converting the motion data of the first vehicle in the first direction into corresponding motion simulation data, the second terminal device can promptly and effectively restore the motion state of the first vehicle in the first direction, providing the remote user with an immersive experience of the first vehicle's motion changes in the first direction. Furthermore, this implementation can simply and efficiently convert the complex motion data of the first vehicle in the first direction into the simulation data of the first vehicle's motion in the first direction through simple data mapping (e.g., mapping direction to corresponding direction values ​​or mapping motion data to corresponding level values) and data calculations.

[0020] In one possible implementation, the first vehicle driving companion device converts the motion data of the first vehicle in the second direction to obtain second motion simulation data, including: the first vehicle driving companion device can determine the second level value corresponding to the motion data of the first vehicle in the second direction, and can determine the second numerical value used to indicate the second direction. Afterwards, the first vehicle driving companion device can determine the second motion simulation data based on the second level value and the second numerical value.

[0021] In the above implementation, by converting the motion data of the first vehicle in the second direction into corresponding motion simulation data, the second terminal device can promptly and effectively restore the motion state of the first vehicle in the second direction, providing the remote user with an immersive experience of the first vehicle's motion changes in the second direction. Furthermore, this implementation can simply and efficiently convert the complex motion data of the first vehicle in the second direction into the simulated motion data of the first vehicle in the second direction through simple data mapping and data calculations.

[0022] In one possible implementation, the first vehicle driving companion device converts the motion data of the first vehicle in the third direction to obtain third motion simulation data, including: the first vehicle driving companion device can determine the third level value corresponding to the motion data of the first vehicle in the third direction, and can determine the third numerical value used to indicate the third direction. Thereafter, the first vehicle driving companion device can determine the third motion simulation data based on the third level value and the third numerical value.

[0023] In the above implementation, by converting the motion data of the first vehicle in the third direction into corresponding motion simulation data, the second terminal device can promptly and effectively restore the motion state of the first vehicle in the third direction, providing the remote user with an immersive experience of the first vehicle's motion changes in the third direction. Furthermore, this implementation can simply and efficiently convert the complex motion data of the first vehicle in the third direction into the simulation data of the first vehicle in the third direction through simple data mapping and data calculation.

[0024] In one possible implementation, the method also includes: the first vehicle driving accompaniment device can send a delay parameter, wherein the delay parameter can be used to synchronize the video image of the first vehicle at the first time with the driving status of the first vehicle at the first time, and the delay parameter is determined by training the artificial intelligence AI model based on the video data samples and the driving data samples.

[0025] In the above implementation, by sending the delay parameter to the terminal device (such as the first terminal device or the second terminal device), the video picture effect presented on the terminal device side can be kept consistent with the driving status.

[0026] In a possible implementation, the first data stream may further carry at least one of the following: an offset in the first data stream of the driving data at the first time or the first vehicle at the first time.

[0027] In the above implementation, by including the first time in the first data stream, the first vehicle's driving companion device can accurately determine the acquisition time of the video data and driving data, thereby facilitating the synchronous transmission of the video data and driving data by the first vehicle's driving companion device, thereby ensuring that the video image and driving status of the first vehicle at the first time can be synchronously restored. Furthermore, by including the offset of the driving data within the first data stream in the first data stream, the first vehicle's driving companion device can facilitate timely and efficient parsing of the first data stream.

[0028] In one possible implementation, when the first data stream also carries the offset of the driving data of the first vehicle at the first time in the first data stream, the method also includes: the first vehicle driving accompaniment device can parse the first data stream according to the offset to obtain the video data of the first vehicle at the first time and the driving data of the first vehicle at the first time.

[0029] In the above implementation, by carrying the offset of the driving data in the first data stream in the first data stream, the first vehicle driving companion device can parse the first data stream in a timely and effective manner, thereby obtaining the video data and driving data of the first vehicle at the first time.

[0030] In a possible implementation manner, the video data of the first vehicle at the first time includes at least one of the following: vehicle exterior video data of the first vehicle at the first time.

[0031] In the above implementation method, by uploading the vehicle's external video data, the vehicle's external scene can be rendered and restored, so that the remote user can have a good full-scene visual experience (that is, the actual external information of the first vehicle is presented to the remote user).

[0032] In one possible implementation, the driving status of the first vehicle at the first time may include at least one of the following: the position information of the first vehicle, the driving speed of the first vehicle, the relative position of the first vehicle and other vehicles, the acceleration of the first vehicle, the attitude angle of the first vehicle, or whether the first vehicle is in an automatic driving state.

[0033] In a second aspect, the present application provides a vehicle driving companion method, which can be applied to the field of electric vehicles or any connected vehicle field. Optionally, the method can be executed by a second vehicle driving companion device or a module of the second vehicle driving companion device (such as a processor, processing unit, chip, chip system, or circuit, etc.). It should be understood that the method can also be implemented by a logical node, logic module, or software that can implement all or part of the functions of the second vehicle driving companion device. Exemplarily, the following example uses the second vehicle driving companion device to execute the vehicle driving companion method. The method may include the following steps: the second vehicle driving companion device can splice the video data of the first vehicle at a first time and the driving data of the first vehicle at a first time to obtain a first data stream, wherein the video data of the first vehicle at the first time is collected by an image acquisition device located on the first vehicle at the first time, and the driving data of the first vehicle at the first time is collected by a chassis sensor device located on the first vehicle at the first time. Afterwards, the second vehicle driving companion device can send the first data stream.

[0034] In this method, by splicing video data and driving data to form combined data (i.e., a first data stream), synchronous transmission (or synchronous uploading or synchronous sending) of the video data and driving data can be easily and conveniently achieved. This allows for synchronous restoration of the video image and driving status of the first vehicle at the first moment, thereby enabling a remote user to immersively experience the driving process of the first vehicle, allowing the remote user to synchronously share the entire journey with the driver of the first vehicle, further enhancing the remote user's companionship and communication experience. Furthermore, the remote user can also effectively communicate with the driver of the first vehicle while sharing the vehicle journey.

[0035] In one possible implementation, the second vehicle driving companion device splices the video data of the first vehicle at the first time and the driving data of the first vehicle at the first time to obtain a first data stream, including: the second vehicle driving companion device can encode the video data of the first vehicle at the first time according to the set video encoding standard to obtain video encoding data, and then the second vehicle driving companion device can splice the video encoding data and the driving data of the first vehicle at the first time to obtain the first data stream.

[0036] In the above implementation, encoding the video data can reduce the amount of data transmitted, thereby reducing communication overhead. In addition, by simply splicing the video encoding data and the driving data, the video encoding data and the driving data can be effectively uploaded synchronously.

[0037] In a possible implementation, the first data stream may further carry at least one of the following: an offset in the first data stream of the driving data at the first time or the first vehicle at the first time.

[0038] The technical effects that can be achieved by this implementation method can refer to the technical effects of the corresponding implementation method in the first aspect above, and will not be repeated here.

[0039] In a possible implementation manner, the video data of the first vehicle at the first time includes at least one of the following: vehicle exterior video data of the first vehicle at the first time.

[0040] The technical effects that can be achieved by this implementation method can refer to the technical effects of the corresponding implementation method in the first aspect above, and will not be repeated here.

[0041] On the third aspect, the present application provides a vehicle driving companion device, which has the ability to implement the functions involved in the first to second aspects above. For example, the vehicle driving companion device includes modules or units or means corresponding to the operations involved in the first to second aspects above. The functions or units or means can be implemented through software, or can be implemented through hardware, or can be implemented by executing corresponding software through hardware.

[0042] In one possible implementation, the vehicle driving companion device includes a transceiver module (or a communication module, transceiver unit, or communication unit, for sending and receiving data) and a processing module (or a processing unit). The transceiver module can be used to send and receive signals to enable communication between the vehicle driving companion device and other devices. For example, the transceiver unit is used to send data to the cloud. The processing module can be used to perform certain internal operations of the vehicle driving companion device. The functions performed by the transceiver module and the processing module can correspond to the operations described in the first and second aspects above.

[0043] In one possible implementation, the vehicle driving companion device includes a processor, which can be coupled to a memory. The memory can store the necessary computer programs or instructions for implementing the functions described in the first and second aspects. The processor can execute the computer programs or instructions stored in the memory. When executed, the computer programs or instructions cause the vehicle driving companion device to implement the method described in any possible implementation of the first and second aspects.

[0044] In one possible implementation, the vehicle driving companion device includes a processor and a memory. The memory can store the necessary computer programs or instructions for implementing the functions described in the first and second aspects above. The processor can execute the computer programs or instructions stored in the memory. When executed, the computer programs or instructions enable the vehicle driving companion device to implement the method described in any possible implementation of the first and second aspects above.

[0045] In one possible implementation, the vehicle driving companion device includes a processor and an interface circuit, wherein the processor is used to communicate with other devices through the interface circuit and execute the method in any possible implementation of the first to second aspects above.

[0046] It is understandable that in the third aspect above, the processor can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor can be a general-purpose processor, which is implemented by reading the software code stored in the memory. In addition, the above processors can be one or more, and the memories can be one or more. The memory can be integrated with the processor, or the memory and the processor can be set separately. In the specific implementation process, the memory can be integrated with the processor on the same chip, or can be set on different chips respectively. The embodiment of the present application does not limit the type of memory and the setting method of the memory and the processor.

[0047] In a fourth aspect, the present application provides a vehicle comprising an image acquisition device, a chassis sensing device, and a second vehicle driving companion device for performing the method of any possible implementation of the second aspect. The image acquisition device is configured to acquire (or may be referred to as acquiring) video data of the vehicle at a first time, and the chassis sensing device is configured to acquire driving data of the vehicle at the first time.

[0048] In a fifth aspect, the present application provides a cloud comprising a first vehicle driving companion device for executing the method in any possible implementation of the first aspect above.

[0049] In a sixth aspect, the present application provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are run on a computer, the computer executes the method in any possible implementation of the first aspect or the method in any possible implementation of the second aspect.

[0050] In the seventh aspect, the present application provides a computer-readable storage medium, which stores a computer program or instruction. When the computer program or instruction is executed by a computer, the computer executes the method in any possible implementation of the first aspect or the method in any possible implementation of the second aspect.

[0051] In an eighth aspect, the present application provides a chip, which may include a processor and a memory (or the chip is coupled to the memory), wherein the processor executes program instructions in the memory to cause the chip to perform the method in any possible implementation of the first aspect or the method in any possible implementation of the second aspect. "Coupled" refers to the direct or indirect connection of two components to each other, such as coupling may refer to an electrical connection between two components.

[0052] In a ninth aspect, the present application further provides a chip system, which includes a processor for supporting a computer device to implement the method in any possible implementation of the first aspect or the method in any possible implementation of the second aspect. In one possible implementation, the chip system also includes a memory for storing programs and data necessary for the computer device. The chip system can be composed of a chip, or it can include a chip and other discrete devices.

[0053] Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A schematic diagram exemplarily illustrates a possible application scenario provided by an embodiment of the present application;

[0055] Figure 2 A schematic diagram illustrating a functional module structure of a vehicle provided in an embodiment of the present application is exemplified;

[0056] Figure 3 A schematic diagram exemplarily illustrates a process flow of a vehicle driving companionship method provided by an embodiment of the present application;

[0057] Figure 4 A schematic diagram illustrating the direction of motion data provided by an embodiment of the present application is exemplified;

[0058] Figure 5 A schematic diagram illustrating the formation of a combination of data provided in an embodiment of the present application is exemplified;

[0059] Figure 6 A schematic diagram illustrating a synchronous experience of VR glasses and a simulated seat provided in an embodiment of the present application is exemplified;

[0060] Figure 7 A schematic structural diagram of a vehicle driving companion device provided by an embodiment of the present application is exemplarily shown;

[0061] Figure 8 A schematic structural diagram of another vehicle driving companion device provided in an embodiment of the present application is exemplarily shown. DETAILED DESCRIPTION

[0062] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0063] The following describes the application scenarios to which the vehicle driving companionship method provided by this application is applicable. It should be noted that these descriptions are for the purpose of facilitating understanding by those skilled in the art and do not limit the scope of protection claimed by this application.

[0064] Figure 1 The following is a schematic diagram showing a possible application scenario to which the embodiment of the present application is applicable. Figure 1 As shown, the application scenario may include a vehicle 100, a cloud 200, and a terminal device 300. For example, the terminal device 300 includes a head-mounted device and a simulated seat (or may be called a sports seat or a sports simulated seat). Exemplarily, the head-mounted device may include but is not limited to a virtual reality (VR) device or an augmented reality (AR) device. For example, a VR device may be such as VR glasses or a VR helmet, and an AR device may be such as AR glasses or an AR helmet. Optionally, the terminal device 300 may also be a smartphone, a tablet computer (Pad), a customer-premises equipment (CPE), a laptop computer, or a computer with wireless transceiver capabilities.

[0065] The vehicle 100 is a vehicle capable of collecting images of the surrounding environment and remote communication. For example, the vehicle 100 is provided with sensors that can collect information about the environment in which the vehicle 100 is located (including information about the vehicle's surroundings), or can also collect driving data of the vehicle 100 itself (such as vehicle position, speed, deceleration, acceleration, or attitude angles (such as pitch angle, roll angle, or heading angle)). It should be understood that the vehicle 100 can send environmental information collected by the sensors (such as video data outside the vehicle) to the cloud 200, or can also send vehicle driving data collected by the sensors to the cloud 200.

[0066] For example, the sensor may be an image acquisition device, which may include but is not limited to: a visible light camera, a depth camera (i.e., a three-dimensional camera), a fisheye camera, a monocular camera, a binocular camera, a near-infrared camera, a video camera, a cockpit camera, a driving recorder (i.e., a video recording terminal), a reversing image camera, or a depth camera. For example, taking the image acquisition device as a fisheye camera, fisheye cameras may be respectively set in the front, rear, left, and right directions of the vehicle 100 to realize the collection of environmental information in the front, rear, left, and right directions of the vehicle 100. The field of view angle of the four fisheye cameras may all be greater than 180 degrees, thereby realizing the all-round capture of the environmental information surrounding the vehicle 100. It should be understood that the larger the field of view angle of the sensor, the larger the range that the sensor can perceive.

[0067] For another example, the sensor may be a radar device. The radar device may transmit electromagnetic wave signals through an antenna and receive echo signals obtained by the reflection of the electromagnetic wave signals by the target, amplify and down-convert the echo signals, and obtain information such as the relative distance, relative speed, and angle between the vehicle 100 and the target. For example, the radar device may be at least one of a lidar sensor or a millimeter-wave radar sensor. The millimeter-wave radar sensor may use radio signals to sense targets in the surrounding environment of the vehicle 100. In some embodiments, in addition to sensing targets, the millimeter-wave radar sensor may also be used to sense the speed and / or direction of travel of the target. The lidar may use lasers to sense targets in the environment in which the vehicle 100 is located.

[0068] In an embodiment of the present application, the sensor may further include a global positioning system (GPS), an inertial measurement unit (IMU), and a brake for modifying the position and / or orientation of the sensor. Among them, the GPS can be any sensor used to estimate the geographic location of the vehicle 100. For example, the GPS may include a transceiver that estimates the position of the vehicle 100 relative to the earth based on satellite positioning data. The IMU can be used to sense changes in the position and orientation of the vehicle 100 based on inertial acceleration and any combination thereof. In some embodiments, the combination of sensors in the IMU may include an accelerometer sensor and a gyroscope sensor.

[0069] The cloud 200 is configured to forward and / or process relevant data from the vehicle 100 (e.g., external video data and vehicle driving data). For example, the cloud 200 may be responsible for rendering the surrounding environment information (e.g., external video data) from the vehicle 100 and transmitting the rendered surrounding environment information to the terminal device 300. It may also be responsible for converting vehicle driving data and transmitting the converted vehicle driving data to the terminal device 300. Optionally, the cloud 200 may also be responsible for authenticating the vehicle 100 and the terminal device 300. It should be understood that the cloud may also be referred to as a cloud server, vehicle cloud server, vehicle cloud, vehicle cloud, cloud, vehicle server, cloud server, cloud controller, or Internet of Vehicles server.

[0070] For example, the cloud 200 may include an authentication module, a data processing module, and a data transceiver module (or may be called a data transmission module). Among them, the authentication module is responsible for the authentication of the vehicle 100 and the terminal device 300. The data processing module is used to parse and process relevant data from the vehicle 100. For example, the data processing module can be used to render the surrounding environment information from the vehicle 100, or it can also be used to convert vehicle driving data. The data transceiver module is used to receive or forward data from the vehicle 100 (such as vehicle surrounding environment information and vehicle driving data). The data transceiver module can also be used to send rendered surrounding environment information, and also to send converted vehicle driving data. Optionally, the data transceiver module can also be used to transmit audio data between the terminal device 300 and the vehicle 100.

[0071] For example, take the terminal device 300 including a head-mounted device and a simulated seat as an example. A user (or remote user, such as family or friends) can experience the actual driving scene of the vehicle 100 in an immersive way by wearing the head-mounted device included in the terminal device 300 and sitting in the simulated seat included in the terminal device 300, so that the user can share the entire journey with the driver of the vehicle 100, and realize the user's synchronous experience of the external video screen of the vehicle 100 and the vehicle driving status (or vehicle driving information). Among them, the rendered surrounding environment information is used to provide to the head-mounted device so that the head-mounted device can present the video screen outside the vehicle in a timely and effective manner; the converted vehicle driving data is used to provide to the simulated seat so that the simulated seat can restore the vehicle driving status of the vehicle 100 in synchronization with the video screen outside the vehicle in a timely and effective manner.

[0072] Optionally, the vehicle driving accompaniment method provided in the embodiment of the present application can be applied to driving accompaniment communication scenarios, and can also be applied to other scenarios (such as remote assisted reminder driving scenarios).

[0073] It should be noted that Figure 1Only one possible application scenario is provided schematically, which is intended to more clearly illustrate the technical solution of the embodiment of the present application and does not limit the application scenario of the vehicle driving accompanying method provided by the present application. Figure 1 The form and quantity of each device in the application scenario shown are for example only and do not constitute a limitation to this application. Figure 1 The names of the various devices in the application scenario shown are only examples. In a specific implementation, the names of the various devices may also be other names, and this application does not specifically limit this. In addition, it is known to those skilled in the art that as new application scenarios emerge, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0074] based on Figure 1 The application scenario shown in the figure, this application also provides a functional module structure of a vehicle. Figure 2 , divided according to logical functions, the vehicle can be divided into the following functional modules: image acquisition device, telematics box (T-Box), central gateway (gateway), vehicle computing unit, power transmission unit, chassis management system, chassis sensor equipment, body control module (BCM), vehicle computer (or called vehicle terminal, center console, in-car audio and video entertainment device) and other electronic control units (ECU). Among them, each ECU transmits information based on the corresponding automotive bus (including body bus, power bus, chassis bus, etc.) to form a local area network inside the vehicle. Each ECU can also be called a network element of the local area network inside the vehicle. Among them, Figure 2 The dashed box shown indicates that the radar device is optional. For the relevant description of the radar device, please refer to the relevant description of the radar device above, which will not be repeated here.

[0075] It should be noted that Figure 2 The connection relationship between the functional modules shown is only an example and does not constitute a limitation of the present application. The functions of each functional module are described below.

[0076] For the description of the image acquisition device, please refer to the above description of the image acquisition device, which will not be repeated here.

[0077] T-Box can communicate with the cloud, RSU, or other vehicles.

[0078] The central gateway can obtain the real-time status information of the vehicle, such as speed, acceleration, position, attitude angle, etc.

[0079] The onboard computing unit can integrate the sensory information obtained by the image acquisition device (or the image acquisition device and radar equipment) to determine the vehicle's external environmental conditions, including the vehicle's position within the current lane, the positions of other vehicles around the vehicle, and the relative position of the vehicle to other vehicles. In addition, the onboard computing unit can also determine the vehicle's driving status information (such as speed, acceleration, attitude angle, etc.) through the sensory information obtained by the chassis sensing equipment.

[0080] The power transmission unit can transmit the calculated control information (such as vehicle speed, vehicle acceleration, vehicle steering, etc.) to the chassis management system through the motor.

[0081] The chassis management system converts the control information into the drive data required by the electric motor.

[0082] The chassis sensing device can be used to collect vehicle driving data (such as vehicle speed, vehicle acceleration, or vehicle attitude angle). For example, in an embodiment of the present application, the chassis sensing device can be an IMU. For example, the chassis sensing device can be a gyroscope sensor or an accelerometer sensor included in the IMU.

[0083] The body control module will achieve intelligent and manual driving through the coordination and cooperation of the vehicle bus. For example, vehicle buses may include: the controller area network (CAN) bus, the local interconnect network (LIN) bus, the high-speed fault-tolerant network protocol (FlexRay) bus, the Media Oriented System Transport (MOST) bus for vehicle multimedia and navigation, and computer network-compatible Bluetooth and wireless LAN. It is understood that the body bus and power bus can be CAN buses, and the chassis bus can be CAN buses or FlexRay buses.

[0084] The car computer can selectively display some information or prompt information on the central control display screen of the car computer for the user to view or remind the user. Optionally, in the embodiment of the present application, the car computer can also be used to play the audio data transmitted by the terminal device 300.

[0085] The following is based on Figure 1 The application scenario shown is a detailed introduction to the specific implementation of the vehicle driving accompaniment method in the embodiment of the present application.

[0086] Figure 3 The following is a flow chart showing a method for accompanying a vehicle in driving provided by an embodiment of the present application. Figure 1The application scenario shown is as follows. It is understandable that Figure 3 The illustrated vehicle travel companionship method is illustrated using multiple vehicle travel companionship devices (e.g., a first vehicle travel companionship device and a second vehicle travel companionship device) as the execution entities of the interactive illustration, but this application does not limit the execution entities of the interactive illustration. For example, the first vehicle travel companionship device can be the cloud or a module in the cloud (e.g., a processor, processing unit, chip system, circuit, or chip, etc.); the second vehicle travel companionship device can be the first vehicle or a module of the first vehicle (e.g., a processor, processing unit, chip system, circuit, or chip, etc.). It should be understood that the method executed by the first vehicle travel companionship device in this application can also be implemented by a logical node, logical module, or software that can implement all or part of the functions of the first vehicle travel companionship device; the method executed by the second vehicle travel companionship device in this application can also be implemented by a logical node, logical module, or software that can implement all or part of the functions of the second vehicle travel companionship device. In addition, with respect to the terminal devices described below, the methods described in this application that are executed by the terminal devices (e.g., the first terminal device or the second terminal device) may also be executed by a module (e.g., a processor, a processing unit, a chip system, a circuit, or a chip, etc.) applied to the terminal device, or may be implemented by a logical node, a logical module, or software that implements all or part of the functions of the terminal device. For example, the first terminal device may be a smartphone, a tablet computer, or a laptop computer; the second terminal device may include a device capable of presenting a video image (e.g., a head-mounted device) and a device capable of simulating the motion state of a vehicle (e.g., a simulated seat).

[0087] like Figure 3 As shown, the method includes:

[0088] Step 301: The second vehicle driving companion device sends a first data stream. Correspondingly, the first vehicle driving companion device receives the first data stream.

[0089] Optionally, in an embodiment of the present application, if the first vehicle driving companion device is a functional module such as a chip system, the functional module may not be aware of which device the received data stream (or information) comes from; if the second vehicle driving companion device is a functional module such as a chip system, the functional module may not be aware of which device the sent data stream (or information) is sent to.

[0090] For example, the first data stream (or combined data) can be used to carry the video data of the first vehicle at the first time and the driving data of the first vehicle at the first time. Exemplarily, the video data of the first vehicle at the first time may include vehicle external video data (referred to as vehicle external video data for short). For example, the data format of the video data may be a YUV format or other equations. The driving data of the first vehicle at the first time may include vehicle position, vehicle speed, vehicle acceleration and deceleration, or vehicle attitude angle (such as heading angle), etc. Optionally, the first data stream can also be used to carry at least one of the following: the offset of the driving data of the first time or the first vehicle at the first time in the first data stream, etc. It should be understood that the first vehicle can be any electric vehicle (or new energy vehicle) or any other connected vehicle (such as an intelligent manufacturing vehicle, etc.). Furthermore, the first vehicle may refer to a vehicle that interacts with the cloud.

[0091] Furthermore, the driving data of the first vehicle at the first time may include at least one of the following: motion data of the first vehicle in a first direction (such as vehicle speed or vehicle acceleration), motion data of the first vehicle in a second direction, or motion data of the first vehicle in a third direction. The first direction, the second direction, and the third direction are different. For example, the first direction may be the horizontal axis direction (such as Figure 4 The second direction may be the longitudinal direction (e.g., the x-axis direction shown in FIG. Figure 4 The third direction may be a vertical direction (e.g., Figure 4 For example, the motion data is the vehicle acceleration. Figure 4 As shown, the motion data of the first vehicle in the first direction may refer to the acceleration of the first vehicle in the x-axis direction, the motion data of the first vehicle in the second direction may refer to the acceleration of the first vehicle in the y-axis direction, and the motion data of the first vehicle in the third direction may refer to the acceleration of the first vehicle in the z-axis direction.

[0092] It is understandable that the video data of the first vehicle at the first time is collected at the first time by the image acquisition device located on the first vehicle (such as a camera deployed outside the vehicle), and the driving data of the first vehicle at the first time is collected at the first time by the chassis sensing device (such as IMU) located on the first vehicle. For example, the image acquisition device and the chassis sensing device can collect data periodically according to a set time interval (such as 40ms, etc.). For example, taking the set time interval as 40ms as an example, the image acquisition device and the chassis sensing device can first collect data at time t1, and then after 40ms, collect data at time t2, and then after 40ms, collect data at time t3, and so on. It should be understood that the time interval between time t2 and time t1 is 40ms, the time interval between time t3 and time t2 is also 40ms, and so on.

[0093] In one possible implementation, after obtaining the video data and driving data of the first vehicle at a first time, the second vehicle's driving companion device (e.g., the first vehicle) can splice the video data and driving data of the first vehicle at the first time to obtain a first data stream. The second vehicle's driving companion device can then send the first data stream. Correspondingly, the first vehicle's driving companion device (e.g., the cloud) can receive the first data stream from the second vehicle's driving companion device. It should be understood that in this implementation, the first data stream includes the video data and driving data of the first vehicle at the first time.

[0094] For example, take the first vehicle's driving companion device as the cloud, the second vehicle's driving companion device as the first vehicle, and the video data including vehicle-external video data as an example. After the first vehicle obtains the vehicle-external video data collected by the image acquisition device on the first vehicle at the first time and the driving data collected by the chassis sensor device on the first vehicle at the first time, the driving data can be spliced ​​behind the vehicle-external video data to form combined data (i.e., the first data stream). Afterwards, the first vehicle can send the combined data. Correspondingly, the cloud can receive the combined data from the first vehicle. Optionally, the first vehicle can also splice the driving data in front of the vehicle-external video data to form combined data, and the embodiment of the present application does not limit this. It can be understood that the vehicle-external video data of the first vehicle at the first time and the driving data of the first vehicle at the first time are combined and sent together, which can be convenient for subsequent synchronous use.

[0095] In another possible implementation, after obtaining the video data of the first vehicle at the first time and the driving data of the first vehicle at the first time, the second vehicle driving companion device can encode the video data of the first vehicle at the first time according to a set video encoding standard or reuse (or adopt) an existing encoder and decoder (CODEC) standard to obtain video encoding data. Afterwards, the second vehicle driving companion device can splice the video encoding data and the driving data of the first vehicle at the first time to obtain a first data stream. Afterwards, the second vehicle driving companion device can send the first data stream. Correspondingly, the first vehicle driving companion device (such as the cloud) can receive the first data stream from the second vehicle driving companion device. It should be understood that in this implementation, the first data stream includes the driving data of the first vehicle at the first time and the encoded video data (i.e., video encoding data).

[0096] For example, the set video coding standard or existing codec standard may include, but is not limited to, H264, H265, H266, AV1, or AVS3. These standards are selected to achieve better compression performance. Furthermore, reusing existing codec standards offers openness and flexibility, allowing selection of appropriate codec standards based on different application scenarios.

[0097] For example, it is assumed that the video coding standard is H265, the first vehicle driving companion device is the cloud, the second vehicle driving companion device is the first vehicle, and the video data includes video data outside the vehicle. Figure 5 This is a schematic diagram of forming a combined data (i.e., first data stream) provided in an embodiment of the present application. Figure 5 As shown, after the first vehicle obtains the video data outside the vehicle collected by the image acquisition device on the first vehicle at the first time, it can use H265 to encode the video data outside the vehicle to obtain video encoding data. Afterwards, the first vehicle can splice the driving data collected by the chassis sensing device on the first vehicle at the first time behind the video encoding data to form combined data. Afterwards, the first vehicle can send the combined data. Correspondingly, the cloud can receive the combined data from the first vehicle. It should be understood that the header information of the combined data is in H265 format, and the entire data content cannot be decoded and played, and is only for the purpose of achieving synchronous use of video data and driving data. Optionally, the first vehicle can also splice the driving data collected by the chassis sensing device on the first vehicle at the first time in front of the video encoding data to form combined data, and this embodiment of the application does not impose any restrictions on this.

[0098] Step 302: The first vehicle driving companion device sends a second data stream. Correspondingly, a terminal device (such as the first terminal device or the second terminal device) receives the second data stream.

[0099] Optionally, in an embodiment of the present application, if the first vehicle driving accompaniment device is a functional module such as a chip system, the functional module may not be aware of which device the sent data stream (or information) is sent to; if the terminal device is replaced with a functional module such as a chip system, the functional module may not be aware of which device the received data stream (or information) comes from.

[0100] The second data stream is related to the first data stream. For example, the second data stream can be used to carry the video data of the first vehicle at the first time and the driving data of the first vehicle at the first time. The video data of the first vehicle at the first time can be used to restore the video screen of the first vehicle at the first time (such as the video screen outside the vehicle), and the driving data of the first vehicle at the first time can be used to restore the driving state of the first vehicle at the first time. For example, the driving state of the first vehicle at the first time may include at least one of the following: the position information of the first vehicle at the first time, the driving speed of the first vehicle at the first time, the relative position of the first vehicle with other vehicles at the first time, the acceleration of the first vehicle at the first time, the attitude angle of the first vehicle at the first time (such as the heading angle), or whether the first vehicle is in an automatic driving state, etc.

[0101] It is understood that in some scenarios, the second data stream being related to the first data stream can be understood as the second data stream being the first data stream. In some scenarios, the second data stream being related to the first data stream can be understood as the second data stream being obtained by processing the first data stream.

[0102] The following describes the implementation process of the first vehicle driving companion device sending the second data stream through the following possible implementation methods.

[0103] Implementation method 1: The first vehicle driving companion device can send the second data stream to the first terminal device.

[0104] It is understood that the first terminal device has installed a target application for assisting in vehicle driving companionship. A remote user can use this target application to interact with the driver or passengers of the first vehicle. For example, the target application can be a client, a web application, or a mini-program embedded in another application. Optionally, the first terminal device may include a display for presenting a vehicle driving companionship interface to facilitate human-computer interaction. The vehicle driving companionship interface displays information edited by the user or information provided to the user.

[0105] In the first implementation method described above, the first vehicle driving companion device may also send a delay parameter (or may be referred to as a delay parameter) to the first terminal device. The delay parameter may be used to synchronize the video image of the first vehicle at a first time (e.g., a video image outside the vehicle) with the driving status of the first vehicle at a first time. For example, the delay parameter may include the time delay required for the video image and the driving status to be restored synchronously. In an embodiment of the present application, the delay parameter may be determined by the first vehicle driving companion device (e.g., a cloud-based device) or other device with model training capabilities by training an artificial intelligence (AI) model based on video data samples and driving data samples (e.g., forward training). It should be understood that the delay parameter may be updated periodically. For example, the first vehicle driving companion device (e.g., a cloud-based device) or other device with model training capabilities may periodically obtain video data samples and driving data samples, and train the AI ​​model based on the periodically obtained video data samples and driving data samples to determine the corresponding delay parameter. Optionally, the delay parameter may also be updated accordingly with version upgrades (e.g., vehicle-side version upgrades).

[0106] In one example, when the terminal device is a first terminal device, after receiving a first data stream from a second vehicle driving companion device, the first vehicle driving companion device may transmit the first data stream as a second data stream to the first terminal device. For example, the first data stream may include video data and driving data of the first vehicle at a first time, or may include encoded video data (i.e., encoded data) and driving data of the first vehicle at a first time. Optionally, the first vehicle driving companion device may also transmit a delay parameter to the first terminal device.

[0107] For example, consider a case where the first data stream also carries the offset of the driving data of the first vehicle at a first time in the first data stream, the first data stream includes the video encoding data and driving data of the first vehicle at a first time, and the video encoding standard is set to H.265. When the first terminal device receives the second data stream from the first vehicle driving companion device, the first terminal device can parse the second data stream based on the offset to obtain the video encoding data and driving data of the first vehicle at a first time. Thereafter, the first terminal device can use H.265 to decode the video encoding data of the first vehicle at a first time to obtain the video data of the first vehicle at a first time. The first terminal device can then display the video image of the first vehicle at a first time (e.g., the video image outside the vehicle) in the first area of ​​the vehicle driving companion interface based on the video data of the first vehicle at a first time, and can also display the driving status of the first vehicle at a first time in the second area of ​​the vehicle driving companion interface based on the driving data of the first vehicle at a first time, or can also restore the driving process of the first vehicle at a first time in the second area of ​​the vehicle driving companion interface. Optionally, when the first terminal device receives the delay parameter, the first terminal device can use the delay parameter, so as to ensure that the video image presented at the first time and the vehicle driving state (or vehicle driving process) restored at the first time can be synchronized.

[0108] In another example, when the terminal device is a first terminal device, if the first data stream also carries the offset of the driving data of the first vehicle at the first time in the first data stream, the first vehicle driving companion device can parse the first data stream according to the offset to obtain the video data of the first vehicle at the first time and the driving data of the first vehicle at the first time. Afterwards, the first vehicle driving companion device can generate a second data stream based on the video data of the first vehicle at the first time and the driving data of the first vehicle at the first time. Then, the first vehicle driving companion device can send the second data stream to the first terminal device. It should be understood that in this implementation mode, the second data stream includes the video data and driving data of the first vehicle at the first time. Optionally, the first vehicle driving companion device can also send a delay parameter to the first terminal device.

[0109] For example, when the first terminal device receives the second data stream from the first vehicle driving companion device, the first terminal device can obtain the video data of the first vehicle at the first time and the driving data of the first vehicle at the first time from the second data stream. Afterwards, the first terminal device can display the video picture of the first vehicle at the first time (such as the video picture outside the vehicle) in the first area of ​​the vehicle driving companion interface based on the video data of the first vehicle at the first time, and can display the driving status of the first vehicle at the first time in the second area of ​​the vehicle driving companion interface based on the driving data of the first vehicle at the first time, or can restore the driving process of the first vehicle at the first time in the second area of ​​the vehicle driving companion interface. Optionally, when the first terminal device receives the delay parameter, the first terminal device can use the delay parameter to ensure that the video picture presented at the first time and the vehicle driving status (or vehicle driving process) restored at the first time can be synchronized.

[0110] In another example, when the terminal device is a first terminal device, if the first data stream also carries the offset of the driving data of the first vehicle at the first time in the first data stream, the first vehicle driving companion device can parse the first data stream according to the offset to obtain the video coding data of the first vehicle at the first time and the driving data of the first vehicle at the first time. Afterwards, the first vehicle driving companion device can generate a second data stream based on the video coding data of the first vehicle at the first time and the driving data of the first vehicle at the first time. Then, the first vehicle driving companion device can send the second data stream to the first terminal device. It should be understood that in this implementation mode, the second data stream includes the video coding data and driving data of the first vehicle at the first time. Optionally, the first vehicle driving companion device can also send a delay parameter to the first terminal device.

[0111] For example, let's assume the video encoding standard is set to H265. When the first terminal device receives the second data stream from the first vehicle driving companion device, the first terminal device can obtain the video encoding data of the first vehicle at the first time and the driving data of the first vehicle at the first time from the second data stream. The first terminal device can then use H265 to decode the video encoding data of the first vehicle at the first time to obtain the video data of the first vehicle at the first time. The first terminal device can then display the video image of the first vehicle at the first time (e.g., the video image outside the vehicle) in the first area of ​​the vehicle driving companion interface based on the video data of the first vehicle at the first time, and can also display the driving status of the first vehicle at the first time in the second area of ​​the vehicle driving companion interface based on the driving data of the first vehicle at the first time, or can also restore the driving process of the first vehicle at the first time in the second area of ​​the vehicle driving companion interface. Optionally, when the first terminal device receives a delay parameter, the first terminal device can use the delay parameter to ensure that the video image presented at the first time and the vehicle driving status (or vehicle driving process) restored at the first time are synchronized.

[0112] Implementation method two: The first vehicle driving companion device can render the video data carried by the first data stream to obtain video rendering data, and can convert the driving data carried by the first data stream to obtain motion simulation data. Afterwards, the first vehicle driving companion device can send a second data stream to the second terminal device. Among them, the second data stream may include video rendering data and motion simulation data. For example, take the second terminal device as an example including a head-mounted device (such as VR glasses) and a simulated seat. The video rendering data can be used by the head-mounted device to restore the video screen of the first vehicle at the first time (such as the video screen outside the vehicle), and the motion simulation data is used to simulate the seat to restore the driving state (or can be understood as the motion state) of the first vehicle at the first time.

[0113] It is understandable that in the above implementation mode 2, the first vehicle driving companion device can also send a delay parameter to the second terminal device. The relevant description of the delay parameter can refer to the relevant description of the delay parameter in the above implementation mode 1, which will not be repeated here.

[0114] For example, the following describes the implementation process of the first vehicle driving companion device sending the second data stream to the second terminal device through the following possible examples.

[0115] Example 1: When the first data stream carries the video data of the first vehicle at the first time and the driving data of the first vehicle at the first time, if the first data stream also carries the offset of the driving data of the first vehicle at the first time in the first data stream, the first vehicle driving companion device can parse the first data stream according to the offset to obtain the video data of the first vehicle at the first time and the driving data of the first vehicle at the first time. Afterwards, the first vehicle driving companion device can render the video data of the first vehicle at the first time (such as AR / VR rendering) to obtain video rendering data, and can convert the driving data of the first vehicle at the first time to obtain motion simulation data. Then, the first vehicle driving companion device can generate a second data stream based on the video rendering data and the motion simulation data, and can send the second data stream to the second terminal device.

[0116] It should be understood that in this example, the second data stream includes the video rendering data of the first vehicle at the first time and the motion simulation data of the first vehicle at the first time. Optionally, the first vehicle driving companion device may also send a delay parameter to the second terminal device. It is understandable that in the case where the second terminal device includes a head-mounted device and a simulated seat, the first vehicle driving companion device may send the second data stream to the second terminal device, and the second terminal device sends the video rendering data to the head-mounted device and the motion simulation data to the simulated seat, or the first vehicle driving companion device may directly send the video rendering data to the head-mounted device and directly send the motion simulation data to the simulated seat. Optionally, the first vehicle driving companion device may send the delay parameter to the second terminal device, and the second terminal device may send the delay parameter to the head-mounted device, or the first vehicle driving companion device may directly send the delay parameter to the head-mounted device.

[0117] Example 2: When a first data stream carries the video encoding data of a first vehicle at a first time and the driving data of the first vehicle at a first time, if the first data stream also carries the offset of the driving data of the first vehicle at the first time in the first data stream, the first vehicle driving companion device can parse the first data stream based on the offset to obtain the video encoding data of the first vehicle at the first time and the driving data of the first vehicle at the first time. Afterwards, the first vehicle driving companion device can use a video decoding standard to decode the video encoding data of the first vehicle at the first time to obtain the video data of the first vehicle at the first time. Then, the first vehicle driving companion device can render the video data of the first vehicle at the first time (such as AR / VR rendering) to obtain video rendering data, and can convert the driving data of the first vehicle at the first time to obtain motion simulation data. Finally, the first vehicle driving companion device can generate a second data stream based on the video rendering data and the motion simulation data, and can send the second data stream to the second terminal device.

[0118] It should be understood that in this example, the second data stream includes the video rendering data of the first vehicle at the first time and the motion simulation data of the first vehicle at the first time. Optionally, the first vehicle driving companion device may also send a delay parameter to the second terminal device. It is understandable that in the case where the second terminal device includes a head-mounted device and a simulated seat, the first vehicle driving companion device may send the second data stream to the second terminal device, and the second terminal device sends the video rendering data to the head-mounted device and the motion simulation data to the simulated seat, or the first vehicle driving companion device may directly send the video rendering data to the head-mounted device and directly send the motion simulation data to the simulated seat. Optionally, the first vehicle driving companion device may send the delay parameter to the second terminal device, and the second terminal device may send the delay parameter to the head-mounted device, or the first vehicle driving companion device may directly send the delay parameter to the head-mounted device.

[0119] For example, the second terminal device includes VR glasses and a simulated seat, and the first vehicle driving companion device is a cloud. Figure 6 As shown, the cloud sends the video rendering data of the first vehicle at the first time to the VR glasses, and the cloud sends the motion simulation data of the first vehicle at the first time to the simulation seat. In this example, after receiving the video rendering data of the first vehicle at the first time from the cloud, the VR glasses can use the video rendering data of the first vehicle at the first time to render and present the video picture of the first vehicle at the first time, so that the user can experience the driving scene of the first vehicle (or can be called a driving picture) in an immersive way, and the user and the driver of the first vehicle can share the field of view of the vehicle's driving route. After receiving the motion simulation data of the first vehicle at the first time from the cloud, the simulation seat can use the motion simulation data of the first vehicle at the first time to perform digital-to-analog conversion, so that the user can experience the driving status of the first vehicle (or can be understood as the actual driving information of the vehicle).

[0120] In another example, the cloud sends a second data stream (including video rendering data of the first vehicle at a first time and motion simulation data of the first vehicle at a first time) to a second terminal device. The second terminal device then sends the video rendering data included in the second data stream to the VR glasses and the motion simulation data included in the second data stream to the simulated seat. In this example, after receiving the second data stream from the cloud, the second terminal device can obtain the video rendering data and motion simulation data of the first vehicle at a first time from the second data stream. The second terminal device can then send the video rendering data of the first vehicle at a first time to the VR glasses and the motion simulation data of the first vehicle at a first time to the simulated seat. After receiving the video rendering data of the first vehicle at a first time, the VR glasses can use this video rendering data to render and present the video of the first vehicle at a first time, allowing the user to immersively experience the driving scene of the first vehicle and share a view of the vehicle's journey with the driver of the first vehicle. After receiving the motion simulation data of the first vehicle at a first time, the simulated seat can use this motion simulation data to perform digital-to-analog conversion, allowing the user to experience the driving state of the first vehicle.

[0121] The following takes the driving data of the first vehicle at the first time, which includes one or more of the movement data of the first vehicle in the first direction, the movement data of the first vehicle in the second direction, and the movement data of the first vehicle in the third direction, as an example to introduce the implementation process of the first vehicle driving companion device converting the driving data carried by the first data stream to obtain motion simulation data.

[0122] Method 1: When the driving data of the first vehicle at the first time includes the movement data of the first vehicle in the first direction, the first vehicle driving companion device can convert the movement data of the first vehicle in the first direction to obtain first motion simulation data. Afterwards, the first vehicle driving companion device can send the first motion simulation data to the terminal device (such as the second terminal device). For example, take the terminal device as the second terminal device. When the second terminal device includes a simulated seat, the first motion simulation data can be used for the simulated seat in the second terminal device to simulate (or restore) the movement state of the first vehicle at the first time and in the first direction.

[0123] In this embodiment of the present application, the first vehicle driving companion device may determine a first level value corresponding to the motion data of the first vehicle in a first direction, and may determine a first numerical value indicating (or representing or characterizing) the first direction. Subsequently, the first vehicle driving companion device may determine first motion simulation data based on the first level value and the first numerical value.

[0124] For example, assuming the first vehicle's driving companion device is the cloud, and the first vehicle's motion data in the first direction is lateral acceleration, the cloud can determine the level value corresponding to the acceleration range of the first vehicle's lateral acceleration based on a mapping relationship (or correspondence) between level values ​​and acceleration ranges, and can also determine the corresponding lateral value (or a value representing the lateral direction) based on the mapping relationship between direction and value. Subsequently, the cloud can determine first motion simulation data (or first motion instruction or first motion simulation instruction) based on the level value and the corresponding lateral value corresponding to the first vehicle's lateral acceleration.

[0125] For example, the mapping relationship between the level value and the acceleration range can be seen in Table 1. It should be understood that Table 1 is an example for the purpose of explaining the technical solutions in the embodiments of the present application, and does not constitute a limitation on the technical solutions in the embodiments of the present application.

[0126] Table 1

[0127] <![CDATA[Acceleration range (unit: m / s 2 )]]> Sports level (or level) Grade value (or grade value) 0～0.5 Level 1 0000 0001 0.5～1.5 Level 2 0000 0010 1.5～2.5 Level 3 0000 0011 2.5～3 Level 4 0000 0100 3～3.5 Level 5 0000 0101 3.5～4 Level 6 0000 0110 4～5 Level 7 0000 0111 5～6 Level 8 0000 1000 6～7 Level 9 0000 1001 +7 or above or -7 or below Level 10 0000 1010

[0128] It should be noted that acceleration is a vector value and can be positive or negative. Among them, the acceleration range is 0 to +0.5 or the acceleration range is -0.5 to 0, and the corresponding level is level 1; the acceleration range is +0.5 to +1.5 or the acceleration range is -1.5 to -0.5, and the corresponding level is level 2; the acceleration range is +1.5 to +2.5 or the acceleration range is -2.5 to -1.5, and the corresponding level is level 3; the acceleration range is +2.5 to +3 or the acceleration range is -3 to -2.5, and the corresponding level is level 4; the acceleration range is +3 to +3.5 or the acceleration range is -3.5 to -3, and the corresponding level is level 5. The corresponding levels are level five; the acceleration range is +3.5 to +4 or -4 to -3.5; the acceleration range is +4 to +5 or -5 to -4, and the corresponding levels are level seven; the acceleration range is +5 to +6 or -6 to -5, and the corresponding levels are level eight; the acceleration range is +6 to +7 or -7 to -6, and the corresponding levels are level nine; the acceleration range is above +7 (i.e. (+7, +∞)) or below -7 (i.e. (-7, -∞)), and the corresponding levels are level ten.

[0129] For example, the mapping relationship between directions and values ​​can be found in Table 2. It should be understood that Table 2 is an example for the purpose of illustrating the technical solutions in the embodiments of the present application, and does not constitute a limitation on the technical solutions in the embodiments of the present application.

[0130] Table 2

[0131] direction Numerical value (or direction value) First direction (such as horizontal) 00 Second direction (such as vertical) 01 The third direction (such as vertical direction) 10

[0132] It should be noted that for each direction, positive and negative values ​​can be used to distinguish between front and back, left and right, up and down. For example, for a positive direction, a prefix value of 10 can be added after (or before) the value corresponding to the positive direction, and for a negative direction, a prefix value of 11 can be added after (or before) the value corresponding to the negative direction. For example, taking the horizontal direction as an example, the vehicle's lateral motion data (such as acceleration) can be distinguished between front and back by positive and negative values. Taking the longitudinal direction as an example, the vehicle's longitudinal motion data (such as acceleration) can be distinguished between left and right by positive and negative values. Taking the vertical direction as an example, the vehicle's vertical motion data (such as acceleration) can be distinguished between up and down by positive and negative values.

[0133] Method 2: When the driving data of the first vehicle at the first time includes the movement data of the first vehicle in the second direction, the first vehicle driving companion device can convert the movement data of the first vehicle in the second direction to obtain second movement simulation data. Afterwards, the first vehicle driving companion device can send the second movement simulation data to the terminal device (such as the second terminal device). For example, take the terminal device as the second terminal device. When the second terminal device includes a simulated seat, the second movement simulation data can be used by the simulated seat in the second terminal device to simulate the movement state of the first vehicle at the first time and in the second direction.

[0134] In this embodiment of the present application, the first vehicle driving companion device can determine a second level value corresponding to the motion data of the first vehicle in the second direction, and can also determine a second numerical value indicating the second direction. The first vehicle driving companion device can then determine second motion simulation data based on the second level value and the second numerical value.

[0135] For example, take the first vehicle's driving accompaniment device as the cloud, and the motion data of the first vehicle in the second direction as the longitudinal acceleration as an example. The cloud can determine the level value corresponding to the acceleration range of the longitudinal acceleration of the first vehicle based on the mapping relationship between the level value and the acceleration range, and can determine the longitudinal corresponding value (or it can be understood as the value used to represent the longitudinal direction) based on the mapping relationship between the direction and the value. Afterwards, the cloud can determine the second motion simulation data (or it can be called the second motion instruction or the second motion simulation instruction) based on the level value corresponding to the longitudinal acceleration of the first vehicle and the longitudinal corresponding value. Optionally, the mapping relationship between the level value and the acceleration range and the mapping relationship between the direction and the value can refer to the relevant description in method one, which will not be repeated here.

[0136] Method 3: When the driving data of the first vehicle at the first time includes the motion data of the first vehicle in the third direction, the first vehicle driving companion device can convert the motion data of the first vehicle in the third direction to obtain third motion simulation data. Afterwards, the first vehicle driving companion device can send the third motion simulation data to the terminal device (such as the second terminal device). For example, take the terminal device as the second terminal device. When the second terminal device includes a simulated seat, the third motion simulation data can be used for the simulated seat in the second terminal device to simulate the motion state of the first vehicle at the first time and in the third direction.

[0137] In this embodiment of the present application, the first vehicle driving companion device can determine a third level value corresponding to the motion data of the first vehicle in a third direction and can also determine a third numerical value indicating the third direction. The first vehicle driving companion device can then determine third motion simulation data based on the third level value and the third numerical value.

[0138] For example, take the first vehicle's driving accompaniment device as the cloud, and the motion data of the first vehicle in the second direction as the longitudinal acceleration. The cloud can determine the level value corresponding to the acceleration range of the vertical acceleration of the first vehicle based on the mapping relationship between the level value and the acceleration range, and can determine the value corresponding to the vertical direction (or can be understood as a value used to represent the vertical direction) based on the mapping relationship between the direction and the value. Afterwards, the cloud can determine the third motion simulation data (or can be called a third motion instruction or a third motion simulation instruction) based on the level value corresponding to the vertical acceleration of the first vehicle and the value corresponding to the vertical direction. Optionally, the mapping relationship between the level value and the acceleration range and the mapping relationship between the direction and the value can refer to the relevant description in method one, which will not be repeated here.

[0139] Optionally, any multiple of the above-mentioned method 1, method 2 and method 3 can be used in combination.

[0140] For example, consider a scenario where a first vehicle suddenly brakes while driving. The first vehicle's driving companion device is the cloud, and the terminal device is a second terminal device, which includes a simulated seat. In this scenario, the driving data collected by the chassis sensor equipment on the first vehicle includes lateral acceleration, longitudinal acceleration, and vertical acceleration. The lateral acceleration of the first vehicle is 0, the longitudinal acceleration of the first vehicle is -6, and the vertical acceleration of the first vehicle is 0. After obtaining the first vehicle's lateral acceleration of 0, the first vehicle's longitudinal acceleration of -6, and the first vehicle's vertical acceleration of 0, the cloud can determine, based on the mapping relationship between level values ​​and acceleration ranges shown in Table 1 above, that the acceleration range for the first vehicle's lateral acceleration of 0 is 0 to 0.5, the acceleration range for the first vehicle's longitudinal acceleration of -6 is -6 to -5, and the acceleration range for the first vehicle's vertical acceleration of 0 is 0 to 0.5. The cloud can then determine that the level corresponding to lateral acceleration 0 is level 1 based on the acceleration range of 0 to 0.5, that the longitudinal acceleration -6 is level 8 based on the acceleration range of -6 to -5, and that the vertical acceleration 0 is level 1 based on the acceleration range of 0 to 0.5. The cloud can then determine that the level value corresponding to lateral acceleration 0 is 0000 0001 based on the level 1 corresponding to lateral acceleration 0, that the longitudinal acceleration -6 is level 0000 1000 based on the level 8 corresponding to longitudinal acceleration -6, and that the vertical acceleration 0 is level 1 based on the level 1 corresponding to vertical acceleration 0. Furthermore, the cloud can determine that the value corresponding to lateral direction is 00, the value corresponding to longitudinal direction is 01, and the value corresponding to vertical direction is 10 based on the mapping relationship between direction and value shown in Table 2 above.

[0141] After determining that the level value corresponding to the lateral acceleration of 0 is 0000 0001 and the numerical value corresponding to the lateral direction is 00, the cloud can generate a lateral motion instruction for the first vehicle, i.e., 0010 0000 0001, based on the level value corresponding to the lateral acceleration of 0, the numerical value corresponding to the lateral direction is 00, and the prefix numerical value 10 used to indicate a positive direction. After determining that the level value corresponding to the longitudinal acceleration of -6 is 0000 1000 and the numerical value corresponding to the longitudinal direction is 01, the cloud can generate a longitudinal motion instruction for the first vehicle, i.e., 0111 0000 1000, based on the level value corresponding to the longitudinal acceleration of -6, the numerical value corresponding to the longitudinal direction is 0000 1000, and the prefix numerical value 11 used to indicate a negative direction. After determining the level value 00000001 corresponding to the vertical acceleration of 0 and the numerical value 10 corresponding to the vertical direction, the cloud can generate a movement instruction for the first vehicle in the vertical direction, that is, 1010 0000 0001, based on the level value 00000001 corresponding to the vertical acceleration of 0, the numerical value 10 corresponding to the vertical direction, and the prefix numerical value 10 used to indicate the positive direction. Afterwards, the cloud can send the first vehicle's lateral motion instruction 0010 00000001, the first vehicle's longitudinal motion instruction 0111 0000 1000, and the first vehicle's vertical motion instruction 1010 0000 0001 to the second terminal device, and the second terminal device sends the motion instruction 0010 0000 0001, the motion instruction 0111 0000 1000, and the motion instruction 1010 0000 0001 to the simulated seat, or the cloud can directly send the motion instruction 0010 0000 0001, the motion instruction 0111 0000 1000, and the motion instruction 1010 0000 0001 to the simulated seat. Among them, the motion instruction 0010 0000 0001 can be used to simulate the lateral motion of the first vehicle when the seat is restored in an emergency braking scenario; the motion instruction 0111 0000 1000 can be used to simulate the longitudinal motion of the first vehicle when the seat is restored in an emergency braking scenario; and the motion instruction 1010 0000 0001 can be used to simulate the vertical motion of the first vehicle when the seat is restored in an emergency braking scenario. It should be understood that since the lateral and vertical directions are positive, the prefix value 10 corresponding to the positive direction can be used, and since the longitudinal direction is a negative direction, the prefix value 11 corresponding to the negative direction can be used.

[0142] It can be seen from the above steps 301 to 302 that the second vehicle driving companion device synchronously transmits the video data of the first vehicle at the first time and the driving data of the first vehicle at the first time to the first vehicle driving companion device. After that, the first vehicle driving companion device synchronously transmits the video data and driving data of the first vehicle at the first time to the terminal device (or the first vehicle driving companion device synchronously transmits the video rendering data and motion simulation data of the first vehicle at the first time to the terminal device), which can achieve synchronous restoration of the video image and driving status of the first vehicle at the first time, so that the remote user can experience the driving process of the first vehicle in an immersive way, so that the remote user can synchronously share the entire journey with the driver of the first vehicle, so that the remote user has a good companionship and communication experience, and the remote user can also communicate well with the driver of the first vehicle during the driving of the vehicle.

[0143] It should be noted that in the description of this application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of A, B and C" includes A, B, C, AB, AC, BC or ABC. And, unless otherwise specified, the ordinal numbers such as "first", "second", and "third" mentioned in the embodiments of this application are used to distinguish multiple objects and are not used to limit the order, timing, priority or importance of multiple objects. In addition, the terms "including", "comprising", "having" and their variations appearing in this application all mean "including but not limited to" unless otherwise specifically emphasized.

[0144] In addition, it should be noted that each step involved in the above embodiments can be performed by a corresponding device, or by a component such as a chip, processor, or chip system within the device, and the embodiments of the present application do not limit this. The above embodiments are described only as examples of execution by corresponding devices.

[0145] It should be noted that in each of the above embodiments, some steps may be selected for implementation, and the order of the steps in the diagrams may be adjusted for implementation, and this application does not limit this. It should be understood that executing some of the steps in the diagrams, adjusting the order of the steps, or combining them for specific implementation all fall within the scope of protection of this application.

[0146] It is understandable that in order to implement the functions in the above embodiments, the various devices involved in the above embodiments include hardware structures and / or software modules corresponding to the execution of each function. It should be readily apparent to those skilled in the art that, in combination with the units and method steps of the various examples described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0147] It should be noted that the "steps" in the embodiments of this application are merely illustrative and serve as a method of expression for a better understanding of the embodiments. They do not constitute a substantial limitation on the execution of the solutions of this application. For example, the "steps" can also be understood as "features." Furthermore, the steps do not constitute any limitation on the execution order of the solutions of this application. Any changes to the order of steps, or any operations such as step merging or splitting that do not affect the implementation of the overall solution, resulting in new technical solutions, are also within the scope of this application.

[0148] The following is a schematic diagram of the structure of a possible vehicle driving companion device provided in the embodiment of the present application. These vehicle driving companion devices can be used to implement the functions of the first vehicle or cloud in the above method embodiment, thereby also achieving the beneficial effects of the above method embodiment. For example, the second vehicle driving companion device can be as follows Figure 1 The vehicle 100 shown in the figure, or a functional element (such as a processor or chip, etc.) provided in the vehicle 100, has the function of implementing the vehicle driving companion method executed by the second vehicle driving companion device. The first vehicle driving companion device may be as follows Figure 1 The cloud 200 shown, or a functional element (such as a processor or chip, etc.) set in the cloud 200, has the function of implementing the vehicle driving companionship method executed by the first vehicle driving companionship device.

[0149] like Figure 7 As shown, the vehicle driving companion device 700 includes a transceiver module 701 (or may be called a communication module or a transceiver unit or a communication unit, for sending and receiving data) and a processing module 702 (or may be called a processing unit). The vehicle driving companion device 700 is used to implement the above Figure 3 The functions of the first vehicle travel companion device or the second vehicle travel companion device in the method embodiment shown in FIG.

[0150] Optionally, the transceiver module 701 may include a receiving module and / or a transmitting module. The receiving module may be used by the vehicle driving companion device 700 to receive signals (or information or data, etc.); the transmitting module may be used by the vehicle driving companion device 700 to transmit signals (or information or data, etc.). The transmitting module may transmit signals (or information or data, etc.) under the control of the processing module 702, and the receiving module may receive signals (or information or data, etc.) under the control of the processing module 702.

[0151] When the vehicle driving accompanying device 700 is used to achieve the above Figure 3 In the method embodiment shown, the function of the first vehicle driving companion device (such as the cloud) is as follows: the transceiver module 701 is used to receive the first data stream. The first data stream can be used to carry the video data of the first vehicle at the first time and the driving data of the first vehicle at the first time. The transceiver module 701 is also used to send the second data stream. The second data stream can be used to carry the video data of the first vehicle at the first time and the driving data of the first vehicle at the first time, and the second data stream is related to the first data stream. The video data of the first vehicle at the first time is used to restore the video image of the first vehicle at the first time, and the driving data of the first vehicle at the first time is used to restore the driving state of the first vehicle at the first time. The processing module 702 is used to perform corresponding processing operations, such as for rendering the video data of the first vehicle at the first time or for converting the driving data of the first vehicle at the first time.

[0152] When the vehicle driving accompanying device 700 is used to achieve the above Figure 3 In the method embodiment shown, the function of the second vehicle driving companion device (such as the first vehicle) is as follows: the processing module 702 is used to splice the video data of the first vehicle at the first time and the driving data of the first vehicle at the first time to obtain a first data stream. The video data of the first vehicle at the first time is collected by the image acquisition device located on the first vehicle at the first time, and the driving data of the first vehicle at the first time is collected by the chassis sensor device located on the first vehicle at the first time. The transceiver module 701 is used to send the first data stream. The first data stream can be used to carry the video data of the first vehicle at the first time and the driving data of the first vehicle at the first time.

[0153] For a more detailed description of the transceiver module 701 and the processing module 702, please refer to the above Figure 3 The relevant descriptions in the method embodiment shown are not repeated here.

[0154] It should be understood that the transceiver module 701 in the embodiment of the present application can be implemented by a communication interface or a communication interface-related circuit component, and the processing module 702 can be implemented by a processor or a processor-related circuit component.

[0155] It should be noted that the division of modules in the embodiments of the present application is illustrative and is merely a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0156] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, or a server, etc.) or a processor to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0157] Based on the same concept, the embodiment of the present application also provides a possible vehicle driving companion device, which is suitable for Figure 1 The vehicle driving companion device is used to achieve the above Figure 3 The technical solution of the first vehicle driving companion device or the second vehicle driving companion device in the method embodiment shown can also achieve the above Figure 3 The beneficial effects of the first vehicle driving companion device or the second vehicle driving companion device in the method embodiment shown are shown. Figure 8 The vehicle travel companion device 800 includes: a communication interface 801 and a processor 802. Optionally, the vehicle travel companion device 800 also includes a memory 803. The communication interface 801, the processor 802, and the memory 803 are interconnected. When the vehicle travel companion device 800 is used to implement the technical solutions of the first vehicle travel companion device or the second vehicle travel companion device provided in the above embodiments, the communication interface 801 can be used to implement the functions of the above-mentioned transceiver module 701, and the processor 802 is used to implement the functions of the above-mentioned processing module 702.

[0158] Optionally, the communication interface 801, the processor 802, and the memory 803 are interconnected via a bus 804. The bus 804 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0159] Communication interface 801 is used to receive and send data. For example, when the vehicle driving accompanying device 800 is as follows Figure 1 In the case of the vehicle 100 shown in FIG. 1 , the communication interface 801 can be implemented with Figure 1 Communicate with the cloud 200 shown, or it can also be achieved with Figure 1 The communication interface can communicate with other devices (such as other vehicles or vehicle servers) outside of the illustrated application scenario. In one example, the communication interface can be a transceiver device with integrated data transceiver functions. In another example, the communication interface can also be composed of a transmitter and a receiver, wherein the transmitter is used to send data and the receiver is used to receive data.

[0160] Optionally, the communication interface 801 may include a transmitter and / or a receiver. The transmitter is used to transmit signals, messages, information, or data. The receiver is used to receive signals, messages, information, or data. For example, the transmitter transmits signals, messages, information, or data under the control of the processor 802. The receiver receives signals, messages, information, or data under the control of the processor 802.

[0161] The functions of processor 802 can refer to the description of the corresponding functions involved in the first vehicle driving companion device or the second vehicle driving companion device in the above embodiments, and will not be repeated here. Among them, processor 802 can be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and an NP, etc. Processor 802 can further include a hardware chip. The above-mentioned hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The above-mentioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. When implementing the above-mentioned functions, processor 802 can be implemented through hardware, and of course, it can also execute the corresponding software implementation through hardware.

[0162] Memory 803 is used to store program instructions, etc. Specifically, program instructions may include program code, which includes computer operating instructions. Memory 803 may include random access memory (RAM) or non-volatile memory, such as at least one disk drive. Processor 802 executes the program instructions stored in memory 803 to implement the aforementioned functions, thereby performing the method steps required for the first or second vehicle travel companion device in the aforementioned embodiments.

[0163] Based on the same concept, embodiments of the present application further provide a vehicle comprising an image acquisition device, a chassis sensing device, and a second vehicle driving companion device. The image acquisition device is configured to acquire video data of the vehicle at a first time, the chassis sensing device is configured to acquire driving data of the vehicle at a first time, and the second vehicle driving companion device is configured to implement the technical solutions described above for the second vehicle driving companion device in the embodiments above.

[0164] Based on the same concept, the embodiment of the present application further provides a cloud comprising a first vehicle driving companion device, wherein the first vehicle driving companion device is used to implement the technical solution involved in the first vehicle driving companion device in the above embodiment.

[0165] Based on the same concept, an embodiment of the present application also provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are run on a computer, the computer executes the vehicle driving accompaniment method provided in the above embodiment.

[0166] Based on the same concept, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program or instruction. When the computer program or instruction is executed by a computer, the computer executes the vehicle driving accompaniment method provided in the above embodiment.

[0167] The storage medium may be any available medium that can be accessed by a computer. By way of example and not limitation, computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer.

[0168] Based on the same concept, an embodiment of the present application further provides a chip, which may include a processor and a memory (or the chip is coupled to the memory). The processor executes program instructions in the memory to enable the chip to perform the vehicle driving companionship method provided in the above embodiment. "Coupled" refers to the direct or indirect connection of two components to each other, such as electrical connection between two components.

[0169] Based on the same concept, embodiments of the present application also provide a chip system, which includes a processor for supporting a computer device in implementing the functions of the first vehicle travel companion device, the second vehicle travel companion device, or the terminal device described in the above embodiments. In one possible implementation, the chip system also includes a memory for storing the necessary programs and data for the computer device. The chip system can be composed of a single chip or can include a chip and other discrete components.

[0170] The methods provided in the embodiments of the present application can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiments of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state drive (SSD)).

[0171] The steps of the methods described in the embodiments of the present application can be directly embedded in hardware, software units executed by a processor, or a combination of the two. The software units can be stored in RAM, ROM, EEPROM, registers, hard disks, removable disks, CD-ROMs, or other storage media in any form known in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and write information to the storage medium. Alternatively, the storage medium can also be integrated into the processor. The processor and storage medium can be arranged in an ASIC.

[0172] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0173] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0174] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A vehicle driving accompaniment method, characterized in that: include: receiving a first data stream, the first data stream being used to carry video data of a first vehicle at a first time and driving data of the first vehicle at the first time; Sending a second data stream, where the second data stream is used to carry the video data and the driving data, and the second data stream is related to the first data stream; The video data is used to restore the video image of the first vehicle at the first time, and the driving data is used to restore the driving status of the first vehicle at the first time.

2. The method according to claim 1, wherein Sending a second data stream, including: Send the second data stream to the first terminal device.

3. The method according to claim 1, wherein Sending a second data stream, including: Rendering the video data to obtain video rendering data, and converting the driving data to obtain motion simulation data; Sending the second data stream to a second terminal device, where the second data stream includes the video rendering data and the motion simulation data; The video rendering data is used by the head mounted device included in the second terminal device to restore the video image, and the motion simulation data is used by the simulated seat included in the second terminal device to restore the driving state.

4. The method according to claim 3, wherein The driving data includes at least one of the following: movement data of the first vehicle in a first direction, movement data of the first vehicle in a second direction, or movement data of the first vehicle in a third direction; wherein the first direction, the second direction, and the third direction are different; Converting the driving data to obtain motion simulation data includes: Converting the motion data of the first vehicle in the first direction to obtain first motion simulation data, wherein the first motion simulation data is used to restore the motion state of the first vehicle at the first time and the first direction; or Converting the motion data of the first vehicle in the second direction to obtain second motion simulation data, wherein the second motion simulation data is used to restore the motion state of the first vehicle at the first time and the second direction; or The motion data of the first vehicle in the third direction is converted to obtain third motion simulation data, and the third motion simulation data is used to restore the motion state of the first vehicle at the first time and in the third direction.

5. The method according to claim 4, wherein Converting the motion data of the first vehicle in the first direction to obtain first motion simulation data includes: determining a first level value corresponding to the motion data of the first vehicle in the first direction, and determining a first numerical value indicating the first direction; The first motion simulation data is determined according to the first level value and the first numerical value.

6. The method according to claim 4 or 5, characterized in that Converting the motion data of the first vehicle in the second direction to obtain second motion simulation data includes: determining a second level value corresponding to the movement data of the first vehicle in the second direction, and determining a second numerical value indicating the second direction; The second motion simulation data is determined according to the second level value and the second numerical value.

7. The method according to any one of claims 4 to 6, characterized in that Converting the motion data of the first vehicle in the third direction to obtain third motion simulation data includes: determining a third level value corresponding to the motion data of the first vehicle in the third direction, and determining a third numerical value indicating the third direction; The third motion simulation data is determined according to the third level value and the third numerical value.

8. The method according to any one of claims 1 to 7, wherein: The method further comprises: Send a delay parameter, where the delay parameter is used to synchronize the video image of the first vehicle at the first time with the driving state of the first vehicle at the first time, and the delay parameter is determined by training an artificial intelligence AI model based on video data samples and driving data samples.

9. The method according to any one of claims 1 to 8, wherein The first data stream further carries at least one of the following: the first time or the offset of the driving data in the first data stream.

10. The method according to claim 9, wherein When the first data stream also carries the offset, the method further includes: The first data stream is parsed according to the offset to obtain the video data and the driving data.

11. The method according to any one of claims 1 to 10, wherein: The video data includes vehicle exterior video data.

12. The method according to any one of claims 1 to 11, wherein: The driving status includes at least one of the following: position information of the first vehicle, the driving speed of the first vehicle, the relative position of the first vehicle and other vehicles, the acceleration of the first vehicle, the attitude angle of the first vehicle, or whether the first vehicle is in an automatic driving state.

13. A vehicle driving accompaniment method, characterized in that: include: splicing video data of a first vehicle at a first time and driving data of the first vehicle at the first time to obtain a first data stream, wherein the video data is collected by an image acquisition device located on the first vehicle at the first time, and the driving data is collected by a chassis sensor device located on the first vehicle at the first time; The first data stream is sent.

14. The method according to claim 13, wherein Splicing video data of a first vehicle at a first time and driving data of the first vehicle at the first time to obtain a first data stream includes: Encoding the video data according to a set video encoding standard to obtain video encoding data; The video encoding data and the driving data are spliced ​​together to obtain the first data stream.

15. The method according to claim 13 or 14, characterized in that The first data stream further carries at least one of the following: the first time or the offset of the driving data in the first data stream.

16. The method according to any one of claims 13 to 15, wherein: The video data includes vehicle exterior video data.

17. A vehicle driving companion device, characterized in that: Including transceiver module; The transceiver module is configured to receive a first data stream, where the first data stream is configured to carry video data of a first vehicle at a first time and driving data of the first vehicle at the first time; The transceiver module is further configured to send a second data stream, the second data stream being configured to carry the video data and the driving data, the second data stream being related to the first data stream; The video data is used to restore the video image of the first vehicle at the first time, and the driving data is used to restore the driving status of the first vehicle at the first time.

18. A vehicle driving companion device, characterized in that: Includes a transceiver module and a processing module; the processing module is configured to combine video data of a first vehicle at a first time and driving data of the first vehicle at the first time to obtain a first data stream, wherein the video data is collected by an image acquisition device on the first vehicle at the first time, and the driving data is collected by a chassis sensor device on the first vehicle at the first time; The transceiver module is used to send the first data stream.

19. A vehicle driving companion device, characterized in that: including a communication interface, a processor, and a memory; The communication interface is used to receive and send data; The memory is used to store computer program instructions and data; The processor is configured to execute computer program instructions and data stored in the memory, so as to enable the vehicle driving companion device to perform the method according to any one of claims 1 to 12 or the method according to any one of claims 13 to 16.

20. A cloud, characterized in that: The invention comprises a vehicle driving accompanying device for executing the method according to any one of claims 1 to 12.

21. A vehicle, characterized in that: It comprises an image acquisition device, a chassis sensing device and a vehicle driving accompaniment device for executing the method as described in any one of claims 13 to 16; wherein, the image acquisition device is used to obtain video data of the vehicle at the first time, and the chassis sensing device is used to obtain driving data of the vehicle at the first time.

22. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program or instructions. When the computer program or instructions are executed by a computer, the computer is enabled to perform the method according to any one of claims 1 to 12 or the method according to any one of claims 13 to 16.

23. A computer program product, characterized in that The computer program product comprises a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is caused to perform the method according to any one of claims 1 to 12 or the method according to any one of claims 13 to 16.

24. A chip, characterized in that: The chip includes a processor coupled to a memory, and the processor is configured to execute a computer program or instruction stored in the memory, so that the chip performs the method according to any one of claims 1 to 12 or the method according to any one of claims 13 to 16.

Images