Control method and device, remote driving cabin and storage medium

Abstract

The application discloses a control method, comprising: receiving a service order of the remote driving service; determining vehicle information according to the service order, and starting the network flow service to establish a communication connection with the vehicle according to the vehicle information; in the case of successful network connection, receiving raw data of the vehicle in real time and processing the raw data to obtain image data and driving data; calling the display service to display the image data and / or the driving data; and sending the driving data to the control service, so that a safety officer of a remote driving cabin controls the service order, and returns first operation data of the control operation to the vehicle to remotely control the vehicle. The safety officer can monitor and control the whole process of the remote vehicle from receiving the order to remotely controlling the vehicle. The application also provides a control device, a remote driving cabin and a computer readable storage medium.

Description

Technical Field

[0001] This application relates to the field of transportation, and in particular to a control method, control device, remote cockpit, and computer-readable storage medium for a remote cockpit. Background Technology

[0002] With the development of intelligent vehicles, early vehicle driving only included manual driving. Today, vehicles can be driven through manual driving, autonomous driving, and other methods. However, autonomous driving is still in its early stages and the technology is not yet mature. Therefore, remote driving control can be implemented, allowing for both autonomous driving and manual remote driving control simultaneously, thus improving vehicle safety. Currently, some remote driving control technologies implement separate control systems for business and driving functions, while others have relatively simple functionalities that cannot meet the needs of various application scenarios. Therefore, a holistic control method for a remote driving cockpit is urgently needed. Summary of the Invention

[0003] In view of this, this application provides a method for controlling a remote cockpit, including:

[0004] Receive the service order for the remote driving service;

[0005] The vehicle information is determined based on the service order, and the network stream service is initiated to establish a communication connection with the vehicle based on the vehicle information;

[0006] Once a network connection is successfully established, the system receives raw data from the vehicle in real time and processes the raw data to obtain image data and driving data.

[0007] The display service is invoked to display the image data and / or the driving data; and

[0008] The driving data is sent to the control service so that the safety operator in the remote cockpit can perform control operations on the service order and send the first operation data of the control operation back to the vehicle to remotely control the vehicle.

[0009] On the one hand, in terms of data structure processing, this application involves a simple transformation of the basic tree structure, allowing all service modules of the remote driving cockpit to share an array with identical content, but only authorized to access the addresses of their parent and child nodes. This enables developers to easily obtain interfaces from each other. Compared to common multi-process systems like ROS, the data structure of this application is more flexible and convenient. Furthermore, applying this structure only to core layer services ensures clean and direct dependencies for the remote driving cockpit software, thereby improving software stability. On the other hand, the remote driving cockpit can directly establish a connection with the controlled remote vehicle and transmit data. After establishing the connection, each service module is activated, processing and outputting different vehicle data. Combined with a display screen, interactive screen, and vehicle simulation controller, it can adapt to various business scenarios, multiple drivers, and multiple operating methods, allowing safety operators to comprehensively monitor and control the entire process from order acceptance to remote vehicle control.

[0010] Before receiving the service order for the remote driving service, the method further includes:

[0011] The core services of the remote cockpit, including client services, configuration services, interaction services, and network flow services, are initialized as second-layer nodes in the head node.

[0012] The client service establishes a network connection and receives service orders for the remote driving service.

[0013] Thus, in terms of data structure processing, this application involves a simple transformation of the basic tree structure, allowing all remote cockpit service modules to share an array with identical content, but with access permissions only for their parent and child node addresses. This enables developers to easily obtain interfaces from each other. Compared to common multi-process systems like ROS, the data structure of this application is more flexible and convenient. Furthermore, applying this structure only to core layer services ensures clean and direct dependencies for the remote cockpit software, thereby improving software stability.

[0014] The step of calling the display service to display the image data and / or the driving data includes:

[0015] The image data is sent to the decoding service via the network streaming service for decoding processing.

[0016] The decoded image data after decoding is sent to the display service through the screen display interface for display.

[0017] In this way, the cockpit safety operator can obtain real-time information on the driving status of the controlled vehicle, such as real-time display of road information ahead and surrounding environment information, as well as driver information in the cockpit, enabling the safety operator to remotely monitor the vehicle's driving status from the driver's perspective.

[0018] The driving data includes raw vehicle condition data, and the control method further includes:

[0019] After successfully establishing the communication connection, the accessibility service is started;

[0020] The step of calling the display service to display the image data and / or the driving data includes:

[0021] The original vehicle condition data is sent to the auxiliary function service for visualization processing via the network streaming service.

[0022] The visualized vehicle condition data is sent to the display service for display via a text display interface.

[0023] In this way, the vehicle status data acquired can be visualized in the remote driver's cockpit through the auxiliary function service, so that the safety driver can obtain information about the interior and exterior environment of the remote vehicle in real time on the display screen and effectively monitor it. This provides a visually immersive experience and improves the effect of remote driving control to a certain extent.

[0024] The driving data includes driving control data, and sending the driving data to the control service so that the safety operator in the remote cockpit can perform control operations on the service order includes:

[0025] The driving control data is sent to the control service via the network streaming service;

[0026] The control service controls the vehicle simulation controller based on the driving control data to perform data synchronization.

[0027] In this way, by processing driving control data through the control service and synchronizing it to the vehicle simulation controller, the operation of the remote driving cockpit and the vehicle driving kit can be synchronized. The vehicle simulation controller synchronously displays the vehicle's operating status, allowing the safety operator to know and monitor the driving situation of the remote vehicle in real time. Furthermore, when the safety operator needs to take over the vehicle for manual driving, they can operate the vehicle simulation controller at any time. The current driving status of the vehicle and the safety operator's operation are seamlessly connected, allowing the safety operator to quickly and promptly take over the vehicle.

[0028] The control method further includes:

[0029] The original vehicle condition data is obtained through the interactive service and processed to obtain order data; and

[0030] The order data is sent to the display service for display via the screen display interface and / or the text display interface.

[0031] Thus, interactive services can process order status and process data for remote driving control, as well as control data for the vehicle's electronic and electrical systems. The safety driver can be provided with the order and vehicle electronic and electrical status via a display screen or interactive screen, and can control order status updates and remotely operate the vehicle's electronic and electrical systems through interactive operations with the interactive screen, enabling the safety driver to complete remote driving services.

[0032] The control method further includes:

[0033] The second operation data input by the safety officer from the interactive screen is obtained through the interactive service and sent to the client service.

[0034] The client service processes the second operation data and sends it back to the vehicle so that the safety operator can remotely control the vehicle.

[0035] In this way, the business that requires interaction with the safety driver is implemented through UIservice. UIservice can provide an interactive screen so that the safety driver can interact with the order process of remote driving service and control some of the vehicle's electronic and electrical functions.

[0036] The control method further includes:

[0037] After successfully establishing the communication connection, the network data statistics service is started;

[0038] The network data of the communication connection is monitored through the listening interface, and the network data is sent to the network data statistics service for broadcasting.

[0039] In this way, the vehicle's communication network can be monitored in real time through network data statistics services.

[0040] The raw data includes raw audio data, and the method further includes:

[0041] After successfully establishing the communication connection, the voice service is started;

[0042] The original audio data is sent to the audio service for processing via the network streaming service.

[0043] The processed audio data is sent to the speaker device for audio playback.

[0044] This allows safety officers to remotely monitor the vehicle and its external environment, providing an immersive auditory experience and improving the effectiveness of remote monitoring.

[0045] The control method further includes:

[0046] Acquire the voice data received by the microphone device and process the voice data;

[0047] The processed voice data is transmitted back to the network streaming service via the voice service and then sent to the vehicle.

[0048] In this way, the voice service can simultaneously acquire voice data from the remote cockpit and send it to the vehicle, enabling voice synchronization from the remote cockpit to the vehicle for more remote monitoring applications, such as remote dialogue with the outside of the vehicle. This expands the scope of remote monitoring applications to a certain extent.

[0049] The control method further includes:

[0050] After the client service receives the service order, it starts the configuration service;

[0051] The configuration service determines the configuration service information based on the service order and sends the configuration service information back to the client service, so that the client service sends the configuration service information to each service used to implement the remote driving service through the configuration interface for service configuration.

[0052] In this way, the vehicle type can be determined based on the service order, and each service module can be configured accordingly based on different vehicle types, so that each service module and vehicle can be better matched.

[0053] This application also provides a control device for a remote cockpit, including:

[0054] The first receiving module is used to receive the service order for the remote driving service;

[0055] The startup module is used to determine vehicle information based on the service order, and to start the network stream service to establish a communication connection with the vehicle based on the vehicle information;

[0056] The second receiving module is used to receive the vehicle's raw data in real time and process the raw data to obtain image data and driving data when a network connection is successfully established.

[0057] The display module is used to call the display service to display the image data and / or the driving data;

[0058] The control module is used to send the driving data to the control service so that the safety operator in the remote cockpit can perform control operations on the service order and send the first operation data of the control operation back to the vehicle to remotely control the vehicle.

[0059] This application also provides a remote driving cockpit, the vehicle including a memory and a processor, the memory storing a computer program, which, when executed by the processor, implements any of the control methods described above.

[0060] This application also provides a non-volatile computer-readable storage medium that, when the computer program is executed by one or more processors, implements the control method described in any of the above embodiments.

[0061] This application involves receiving service orders for remote driving services; determining vehicle information based on the service order; establishing a communication connection with the vehicle by initiating a network streaming service based on the vehicle information; receiving and processing the vehicle's raw data in real time to obtain image data and driving data once the network connection is successfully established; displaying the image data and / or driving data using a display service; and sending the driving data to a control service, enabling the safety operator in the remote driving cockpit to perform control operations on the service order and sending the first operation data of the control operation back to the vehicle for remote vehicle control. This allows the remote driving cockpit to directly establish a connection with the controlled remote vehicle and transmit data. After establishing the connection, various service modules are activated, each processing and outputting different vehicle data. Combined with a display screen, interactive screen, and vehicle simulation controller, this allows the safety operator to comprehensively monitor and control the remote vehicle from order acceptance to remote vehicle control.

[0062] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0063] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein:

[0064] Figure 1 This is a flowchart illustrating the control method of this application;

[0065] Figure 2 This is a schematic diagram of the control device module of this application;

[0066] Figure 3 This is a flowchart illustrating the control method of this application;

[0067] Figure 4This is a flowchart illustrating the control method of this application;

[0068] Figure 5 This is a schematic diagram of the control device module of this application;

[0069] Figure 6 This is a flowchart illustrating the control method of this application;

[0070] Figure 7 This is a flowchart illustrating the control method of this application;

[0071] Figure 8 This is a flowchart illustrating the control method of this application;

[0072] Figure 9 This is a flowchart illustrating the control method of this application;

[0073] Figure 10 This is a flowchart illustrating the control method of this application;

[0074] Figure 11 This is a flowchart illustrating the control method of this application. Detailed Implementation

[0075] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of this application, and should not be construed as limiting the embodiments of this application.

[0076] Please see Figure 1 This application provides a method for controlling a remote cockpit, including:

[0077] 01: Receive service orders for remote driving services;

[0078] 02: Determine vehicle information based on the service order, and initiate network streaming service to establish a communication connection with the vehicle based on the vehicle information;

[0079] 03: Once a network connection is successfully established, the system receives raw vehicle data in real time and processes the raw data to obtain image data and driving data.

[0080] 04: Call the display service to display image data and / or driving data; and

[0081] 05: Send driving data to the control service, enabling the safety operator in the remote cockpit to perform control operations on the service order and send the first operation data of the control operation back to the vehicle for remote vehicle control.

[0082] Please seeFigure 2 , this application also provides a control device 100, and the control method of this application can be implemented by the control device 100. The control device 100 includes a first receiving module 110, a starting module 120, a second receiving module 130, a display module 140, and a control module 150. Step 01 can be implemented by the first receiving module 110, step 02 can be implemented by the starting module 120, step 03 can be implemented by the second receiving module 130, step 04 can be implemented by the display module 140, and step 05 can be implemented by the control module 150. Or rather, the first receiving module 110 is used to receive the service order of the remote driving service. The starting module 120 is used to determine the vehicle information according to the service order, and start the network flow service to establish a communication connection with the vehicle according to the vehicle information. The second receiving module 130 is used to receive the original data of the vehicle in real time and process the original data to obtain image data and driving data when the network connection is successfully established. The display module 140 is used to call the display service to display the image data and / or driving data. The control module 150 is used to send the driving data to the control service, so that the safety officer in the remote driving cockpit can perform control operations on the service order, and send the first operation data of the control operation back to the vehicle to remotely control the vehicle.

[0083] This application also provides a remote driving cockpit. The remote driving cockpit includes a memory and a processor. A computer program is stored in the memory, and the processor is used to receive the service order of the remote driving service; determine the vehicle information according to the service order, and start the network flow service to establish a communication connection with the vehicle according to the vehicle information; receive the original data of the vehicle in real time and process the original data to obtain image data and driving data when the network connection is successfully established; call the display service to display the image data and / or driving data; and send the driving data to the control service, so that the safety officer in the remote driving cockpit can perform control operations on the service order, and send the first operation data of the control operation back to the vehicle to remotely control the vehicle.

[0084] Specifically, the safety officer can establish a connection with the remotely controlled vehicle through the service provided by the remote driving cockpit, perform data transmission, and then process the data. At the same time, the safety officer can operate the hardware facilities supporting the remote driving cockpit to remotely control the remotely controlled vehicle. For example, for a summon service order initiated by a user, the order requests the driverless vehicle to drive from the current location to the designated location summoned by the user. The safety officer can process the order through the remote driving cockpit to remotely monitor or remotely drive the vehicle to complete the summon service.

[0085] Before receiving the service order of the remote driving service, it further includes:

[0086] The core services of the remote cockpit, including client services, configuration services, interaction services, and network flow services, are initialized as second-layer nodes in the head node.

[0087] Establish a network connection through the client service and receive vehicle information corresponding to remote driving service orders.

[0088] The remote cockpit software framework employs a simplified variation of a basic tree structure. This includes a custom node, an array containing sub-services, and a parent node. Unlike existing bidirectional connection trees, each node in this tree has a pre-defined unique ID, and all nodes are positioned at the corresponding IDs within their parent node's array. This data structure allows all services in the core layer to share a single, identical array, but with access only to their parent and child nodes. Furthermore, it facilitates easy interface exchange between developers. Compared to common multi-process systems like ROS, this data structure is more flexible and convenient. Applying this structure only to core layer services ensures clean and direct dependencies, improving software stability.

[0089] The software framework of the remote cockpit can be initialized starting from a head node. All core services, including client services, configuration services, interaction services, and network flow services, are initialized in the head node as second-layer nodes, and then the developers of each core service collaborate on development in the second layer.

[0090] Thus, in terms of data structure processing, this application involves a simple transformation of the basic tree structure, allowing all remote cockpit service modules to share an array with identical content, but with access permissions only for their parent and child node addresses. This enables developers to easily obtain interfaces from each other. Compared to common multi-process systems like ROS, the data structure of this application is more flexible and convenient. Furthermore, applying this structure only to core layer services ensures clean and direct dependencies for the remote cockpit software, thereby improving software stability.

[0091] Furthermore, a network connection is established through the client service to receive service orders for remote driving services.

[0092] After the core services are initialized in the head node, the client service (ClientService) is started, and a network connection is established between ClientService and the server. The server receives service orders for the remote driving service.

[0093] In steps 01 and 02, a service order for remote driving service is received, vehicle information is determined based on the service order, and network streaming service is initiated to establish a communication connection with the vehicle based on the vehicle information.

[0094] When the server receives a remote driving service order, such as a call service order, initiated by a user, it processes the order to obtain the information of the remotely controlled vehicle and sends the vehicle information to the ClientService.

[0095] Specifically, the ClientService establishes a connection with the netstream service by handshaking. The ClientService sends vehicle information to the netstream service to enable the netstream service to establish a communication connection with the vehicle. The communication connection can be initiated by the vehicle, with the remote driver's cabin authenticating and establishing communication. Alternatively, the communication connection can be initiated by the remote driver's cabin through the netstream service, with the vehicle authenticating and establishing communication. This communication connection can be established via a wireless communication network such as a 5G network.

[0096] Once the communication connection with the vehicle is successfully established, all services are started, including the display service and the control service. The display service processes relevant data and invokes other services to control the relevant display screens or interactive screens for display. The control service sends driving control data to the vehicle simulation controller for synchronization and / or receives control signal data from the vehicle simulation controller and sends it back to the vehicle to implement control-related functions. The specific implementation of these services will be explained in the following steps.

[0097] In steps 03 and 04, if a network connection is successfully established, the system receives raw vehicle data in real time, processes the raw data to obtain image data and driving data, and calls the display service to display the image data and / or driving data.

[0098] Once the netstreamservice establishes a communication connection with the vehicle, it receives raw data from the vehicle in real time, such as 5G raw data packets. This raw data is then processed to generate image data and driving data, which are then displayed using display services. The processing of the raw data includes classification, filtering, and combination to divide the merged raw data into adaptive data for each service module.

[0099] Please seeFigure 3 Step 04 includes:

[0100] 041: The image data is sent to the decoding service via the network streaming service for decoding processing;

[0101] 042: The decoded image data after decoding is sent to the display service through the screen display interface for display.

[0102] Steps 041 and 042 can be implemented by the display module 140. Alternatively, the display module 140 is used to send image data to a decoding service for decoding processing via a network streaming service; and to send the decoded image data, after decoding processing, to the display service for display via a screen display interface.

[0103] The processor is used to send image data to the decoding service for decoding processing via a network streaming service; and to send the decoded image data to the display service for display via a screen display interface.

[0104] Specifically, the image data includes road information ahead of the vehicle collected by the vehicle's internal cameras, environmental road information surrounding the vehicle collected by the vehicle's external cameras, and / or information inside the cabin, such as the driver and cabin devices, collected by in-vehicle intelligent camera devices. Netstreamservice categorizes the image data from the raw data packets and sends it to the decodeservice to decode the image data. After decoding, the decoded data is sent to the displayservice via the display interface. The displayservice then sends the image data to the display screen for display. The cabin safety operator can obtain real-time information about the vehicle's driving status, such as real-time display of road information ahead, surrounding environmental information, and driver information within the cabin. This allows the safety operator to remotely monitor the vehicle's driving status from the driver's perspective.

[0105] Please see Figure 4 Driving data includes raw vehicle condition data, and control methods also include:

[0106] 06: After successfully establishing a communication connection, start the accessibility service;

[0107] Accordingly, step 04 also includes:

[0108] 043: Send the raw vehicle condition data to the auxiliary function service for visualization processing via network streaming service;

[0109] 044: The visualized vehicle condition data is sent to the display service for display via the text display interface.

[0110] Step 06 can be implemented by the control module 150. Steps 043 and 044 can be implemented by the display module 140. Alternatively, the control module 150 is used to activate the auxiliary function service after successfully establishing a communication connection. The display module 140 is used to send the raw vehicle condition data to the auxiliary function service for visualization processing via a network streaming service; and to send the visualized vehicle condition data to the display service for display via a text display interface.

[0111] After successfully establishing a communication connection, the processor is used to start the auxiliary function service; send the raw vehicle condition data to the auxiliary function service for visualization processing via the network streaming service; and send the visualized vehicle condition data to the display service for display via the text display interface.

[0112] Specifically, the raw data includes raw vehicle condition data, such as ultrasonic radar data, collision warning data, and blind spot warning data. After successfully establishing a communication connection, the assistance service is started. The assistance service performs visualization processing on the raw vehicle condition data, including processing it into image and text data. For example, when ultrasonic radar data is received, the radar data is visualized. The assistance service can call the display service to draw the vehicle and its surrounding boundaries. When an object approaches the boundary, the view can change, such as a blinking boundary effect.

[0113] In this way, the vehicle status data acquired can be visualized in the remote driver's cockpit through the auxiliary function service, so that the safety driver can obtain information about the interior and exterior environment of the remote vehicle in real time on the display screen and effectively monitor it. This provides a visually immersive experience and improves the effect of remote driving control to a certain extent.

[0114] Please see Figure 5 Driving data includes driving control data. Step 05 includes:

[0115] 051: Send driving control data to the control service via network streaming service;

[0116] 052: The control service controls the vehicle simulation controller based on the driving control data to synchronize the data.

[0117] Steps 051 and 052 can be implemented by control module 150. Alternatively, control module 150 is used to send driving control data to a control service via a network streaming service; the control service then controls the vehicle simulation controller based on the driving control data for data synchronization.

[0118] The processor is used to send driving control data to the control service via a network streaming service; the control service then controls the vehicle simulation controller to synchronize the data based on the driving control data.

[0119] Specifically, driving data includes driving control data. The control service sends the driving control data to the vehicle simulation controller and synchronizes the vehicle simulation controller based on the driving control data. Once the driving control data is synchronized on the vehicle simulation controller, the vehicle simulation controller in the remote cockpit synchronizes with the steering wheel-related operations, braking, gear shifting, etc., in the remote vehicle to the same position.

[0120] The driving control data is sent to the vehicle simulation controller for synchronous processing.

[0121] The vehicle simulation controller includes a steering wheel and its buttons, brake, accelerator, gear shift, and other driving components that simulate a driver's operation. Driving control data includes data on remote steering wheel operations and control signals from the steering wheel buttons, brake, accelerator, gear shift, etc.

[0122] Synchronization processing involves controlling the vehicle simulation controller to perform the same operations as the vehicle, or operations with the same proportions, based on driving control data. For example, if a remote vehicle turns its steering wheel 30 degrees to the right during autonomous driving, the vehicle simulation controller will also turn its steering wheel 30 degrees to the right, or turn it to the right at a predetermined proportion of 30 degrees, so that the vehicle simulation controller performs the same driving operation as the vehicle.

[0123] Meanwhile, during the remote driving service, when the safety driver needs to manually take over and drive the vehicle, the safety driver can operate the vehicle simulator controller at any time to remotely control the vehicle for driving. The data of the safety driver operating the vehicle simulator controller is the first operation data. The remote driving cockpit service transmits this first operation data back to the vehicle in real time, so that the vehicle can perform the same driving operation as the vehicle simulator controller according to the first operation data.

[0124] For example, if a remote vehicle turns its steering wheel 30 degrees to the right during autonomous driving, the vehicle simulation controller will also turn its steering wheel 30 degrees to the right after synchronization.

[0125] In this way, by processing driving control data through the control service and synchronizing it to the vehicle simulation controller, the operation of the remote driving cockpit and the vehicle driving kit can be synchronized. The vehicle simulation controller synchronously displays the vehicle's operating status, allowing the safety operator to know and monitor the driving situation of the remote vehicle in real time. Furthermore, when the safety operator needs to take over the vehicle for manual driving, they can operate the vehicle simulation controller at any time. The current driving status of the vehicle and the safety operator's operation are seamlessly connected, allowing the safety operator to quickly and promptly take over the vehicle.

[0126] Thus, this application receives service orders for remote driving services; determines vehicle information based on the service order, and initiates a network streaming service to establish a communication connection with the vehicle based on the vehicle information; upon successful network connection establishment, it receives and processes the vehicle's raw data in real time to obtain image data and driving data; it invokes a display service to display the image data and / or driving data; and it sends the driving data to a control service, enabling the safety operator in the remote driving cockpit to perform control operations on the service order and transmit the first operation data of the control operation back to the vehicle for remote vehicle control. This allows the remote driving cockpit to directly establish a connection with the controlled remote vehicle and transmit data. After establishing the connection, it activates various service modules, each of which processes and outputs different vehicle data. Combined with a display screen, interactive screen, and vehicle simulation controller, the safety operator can comprehensively monitor and control the remote vehicle from order acceptance to remote vehicle control.

[0127] Please see Figure 6 The control methods also include:

[0128] 07: Obtain raw vehicle condition data through interactive services and process it to obtain order data; and

[0129] 08: Send the order data to the display service for display via the screen display interface and / or text display interface.

[0130] Steps 07 and 08 can be implemented by the control module 150. Alternatively, the control module 150 is used to obtain raw vehicle condition data through an interactive service and process it to obtain order data; and to send the order data to the display service for display through a screen display interface and / or a text display interface.

[0131] The processor is used to obtain raw vehicle condition data through interactive services and process it to obtain order data; and to send the order data to the display service for display through the screen display interface and / or text display interface.

[0132] Specifically, an interactive screen can be provided through the UI service, allowing safety drivers to interactively operate the remote driving service order process and control certain vehicle electronic and electrical functions, such as turning on the high beams and activating the defroster. The UI service acquires and processes raw vehicle condition data to obtain order data, which includes the remote driving service order process and vehicle electronic and electrical function data.

[0133] Furthermore, the order data is sent to the display service for display via a screen display interface and / or a text display interface. Specifically, some graphical data within the order data, such as map data, can be displayed on the screen via the screen display interface, while the order status can be displayed on the screen or interactive screen via the text display interface, for example, displaying the order status as "Vehicle has arrived".

[0134] Thus, interactive services can process order status and process data for remote driving control, as well as control data for the vehicle's electronic and electrical systems. The safety driver can be provided with the order and vehicle electronic and electrical status via a display screen or interactive screen, and can control order status updates and remotely operate the vehicle's electronic and electrical systems through interactive operations with the interactive screen, enabling the safety driver to complete remote driving services.

[0135] Please see Figure 7 The control methods also include:

[0136] 09: Obtain the second operation data input by the safety officer from the interactive screen through the interactive service and send it to the client service;

[0137] 010: The second operation data is processed through the client service and sent back to the vehicle so that the safety operator can remotely control the vehicle.

[0138] Steps 09 and 010 can be implemented by control module 150. Alternatively, control module 150 is used to obtain the second operation data input by the safety operator from the interactive screen via an interactive service and send it to the client service; the client service processes the second operation data and sends it back to the vehicle so that the safety operator can remotely control the vehicle.

[0139] The processor is used to obtain the second operation data input by the safety driver from the interactive screen through the interactive service and send it to the client service; the client service processes the second operation data and sends it back to the vehicle so that the safety driver can remotely control the vehicle.

[0140] Specifically, the safety operator can input second operation data through the interactive screen. This second operation data includes corresponding operations performed by the safety operator based on the displayed order data. For example, if the order data shows that the corresponding vehicle self-check was successful, the safety operator can click "Confirm Departure" on the interactive screen. The second operation data may also include the safety operator operating the vehicle's electronic and electrical equipment through the interactive screen.

[0141] Furthermore, the UIservice transmits the second operation data to the ClientService, which processes the second operation data and sends it back to the vehicle so that the safety driver can remotely control the vehicle.

[0142] For example, if the order data shows that the lighting ahead is insufficient due to an abnormality, the safety officer clicks on the "High Beams" button displayed on the interactive screen to generate the second operation data for "Turn on High Beams". The ClientService processes and sends "Turn on High Beams" to the vehicle to generate the corresponding control signal, and the vehicle executes the operation of turning on the high beams.

[0143] In this way, the business that requires interaction with the safety driver is implemented through UIservice. UIservice can provide an interactive screen so that the safety driver can interact with the order process of remote driving service and control some of the vehicle's electronic and electrical functions.

[0144] Please see Figure 8 The control methods also include:

[0145] 011: After successfully establishing a communication connection, start the network data statistics service;

[0146] 012: Listens to network data of the communication connection through the listening interface and sends the network data to the network data statistics service for broadcasting.

[0147] Steps 011 and 012 can be implemented by control module 150. Alternatively, control module 150 is used to start the network data statistics service after a successful communication connection is established; it listens for network data of the communication connection through a listening interface and broadcasts the network data to the network data statistics service.

[0148] The processor is used to start the network data statistics service after a communication connection is successfully established; it listens for network data of the communication connection through the listening interface and sends the network data to the network data statistics service for broadcasting.

[0149] Specifically, once the netstreamservice establishes a communication connection with the vehicle, it starts the network data statistics service (statservice). Statservice monitors the network data of the communication connection in real time through a listening interface, including 5G network traffic, packet loss rate, video frame rate, frame loss rate, and other network data.

[0150] Furthermore, `statservice` broadcasts network data, for example, by sending data to `UIservice`. `UIservice` can then call `displayservice` to display network status information, such as latency, signal strength, network characteristics, and 5G status. When frame loss, packet loss, or latency exceeds a threshold, a warning message can pop up.

[0151] In this way, the vehicle's communication network can be monitored in real time through network data statistics services.

[0152] Please see Figure 9 The control methods also include:

[0153] 013: After successfully establishing a communication connection, start the voice service;

[0154] 014: The raw audio data is sent to the audio service for processing via the network streaming service;

[0155] 015: Send the processed audio data to the speaker device for audio playback.

[0156] Steps 013-015 can be implemented by the control module 150. Alternatively, the control module 150 is used to start the sound service after successfully establishing a communication connection; send the raw sound data to the sound service for processing via the network streaming service; and send the processed sound data to the speaker device for sound playback.

[0157] The processor is used to start the sound service after successfully establishing a communication connection; send the raw sound data to the sound service for processing via the network streaming service; and send the processed sound data to the speaker device for sound playback.

[0158] Specifically, the raw data includes raw audio data. Once the netstreamservice establishes a communication connection with the vehicle, it starts the soundservice. The netstreamservice sends the raw audio data to the soundservice for processing, and then sends the processed audio data to the speaker device for sound output. The raw audio data originates from either inside or outside the vehicle, and can include audio recordings of vehicle-related content or external road and environmental information. By outputting the audio collected by the vehicle through the soundservice, safety operators can remotely monitor the vehicle and its external environment, creating an immersive auditory experience and improving the effectiveness of remote monitoring.

[0159] Please see Figure 10 The control methods also include:

[0160] 016: Acquire the voice data received by the microphone device and process the voice data;

[0161] 017: The processed voice data is transmitted back to the network streaming service via the voice service and then sent to the vehicle.

[0162] Steps 016 and 017 can be implemented by the control module 150. In other words, the control module 150 is used to acquire the voice data received by the microphone device, process the voice data, and transmit the processed voice data back to the network streaming service through the voice service and send it to the vehicle.

[0163] The processor is used to acquire the voice data received by the microphone device, process the voice data, and then transmit the processed voice data back to the network streaming service and send it to the vehicle via the voice service.

[0164] Specifically, the microphone can collect voice data from the safety officer or remote cockpit, including the safety officer speaking and audio data. The soundservice processes the cloud data collected by the microphone and calls the sending interface to send it back to the netstreamservice, which then sends it to the vehicle. The vehicle can process the voice data, including recognizing the voice data or playing the voice data.

[0165] In this way, the voice service can simultaneously acquire voice data from the remote cockpit and send it to the vehicle, enabling voice synchronization from the remote cockpit to the vehicle for more remote monitoring applications, such as remote dialogue with the outside of the vehicle. This expands the scope of remote monitoring applications to a certain extent.

[0166] Please see Figure 11 The control methods also include:

[0167] 018: After the client service receives the service order, it starts the configuration service;

[0168] 019: The configuration service determines the configuration service information based on the service order and sends it back to the client service, so that the client service can send the configuration service information to the various services used to implement the remote driving service through the configuration interface for service configuration.

[0169] Steps 018 and 019 can be implemented by the control module 150. Alternatively, the control module 150 is used to start the configuration service after the client service receives a service order; the configuration service determines the configuration service information based on the service order and sends it back to the client service, so that the client service can send the configuration service information to the various services used to implement the remote driving service through the configuration interface for service configuration.

[0170] The processor is used to start the configuration service after the client service receives the service order; the configuration service determines the configuration service information according to the service order and sends it back to the client service, so that the client service can send the configuration service information to the various services used to implement the remote driving service through the configuration interface for service configuration.

[0171] Specifically, after receiving a service order, the ClientService starts the configuration service (configservice) and sends the service order to it. The configservice then configures the various service modules of the remote driving service based on the information provided by the ClientService, including the UserService, AssistService, DecodeService, SoundService, ControlService, DisplayService, and StatService. Each of these service modules configures its own service according to the configuration information to implement the various functions of the remote driving cockpit. For example, for different vehicle types, the display service needs to adjust the display size and scale; therefore, the configservice can obtain the vehicle type information from the service order and configure the display service accordingly.

[0172] After establishing a network connection through the client service, start the log service to record log data used to implement the remote driving service.

[0173] In this way, the vehicle type can be determined based on the service order, and each service module can be configured accordingly based on different vehicle types, so that each service module and vehicle can be better matched.

[0174] This application also provides a computer-readable storage medium. One or more non-volatile computer-readable storage media storing a computer program, which, when executed by one or more processors, implements the control method of any of the above embodiments. Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related software. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), etc.

[0175] In the description of this specification, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with an embodiment or example that are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction. Meanwhile, the terms "first," "second," etc., are intended to distinguish similar or analogous operations. There is a logical relationship between "first" and "second" in some embodiments, but not necessarily in others. This relationship needs to be determined based on the actual embodiments and should not be judged solely by the literal meaning of the terms.

[0176] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0177] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A control method for a remote driving service, characterized in that, include: Receive the service order for the remote driving service; The vehicle information is determined based on the service order, and a network streaming service is initiated to establish a communication connection with the vehicle based on the vehicle information; Once a network connection is successfully established, the system receives raw data from the vehicle in real time and processes the raw data to obtain image data and driving data. The display service is invoked to display the image data and / or the driving data; and The driving data is sent to the control service so that the safety operator in the remote cockpit can process the service order. The system performs control operations and sends the first operation data of the control operations back to the vehicle to remotely control the vehicle. Before receiving the service order for the remote driving service, the method further includes: The core services of the remote cockpit, including client services, configuration services, interaction services, and network flow services, are initialized as second-layer nodes in the head node. Establish a network connection through the client service and receive service orders for the remote driving service; The head node and the second-level nodes constitute a structure tree. Each node in the structure tree includes a corresponding array containing sub-services and a corresponding parent node. Each node has a unique and non-repeating id, and all nodes are located at the position defined by the id in the array of their corresponding parent nodes.

2. The control method according to claim 1, characterized in that, The step of calling the display service to display the image data and / or the driving data includes: The image data is sent to the decoding service via the network streaming service for decoding processing. The decoded image data after decoding is sent to the display service through the screen display interface for display.

3. The control method according to claim 2, characterized in that, The driving data includes raw vehicle condition data, and the control method further includes: After successfully establishing the communication connection, the accessibility service is started; The step of calling the display service to display the image data and / or the driving data includes: The original vehicle condition data is sent to the auxiliary function service for visualization processing via the network streaming service. The visualized vehicle condition data is sent to the display service for display via a text display interface.

4. The control method according to claim 3, characterized in that, The driving data includes driving control data, and sending the driving data to the control service so that the safety operator in the remote cockpit can perform control operations on the service order includes: The driving control data is sent to the control service via the network streaming service; The control service controls the vehicle simulation controller based on the driving control data to perform data synchronization.

5. The control method according to claim 3, characterized in that, The control method further includes: The original vehicle condition data is obtained through the interactive service and processed to obtain order data; and The order data is sent to the display service for display via the screen display interface and / or the text display interface.

6. The control method according to claim 5, characterized in that, The control method further includes: The second operation data input by the safety officer from the interactive screen is obtained through the interactive service and sent to the client service. The client service processes the second operation data and sends it back to the vehicle so that the safety operator can remotely control the vehicle.

7. The control method according to claim 1, characterized in that, The control method further includes: After successfully establishing the communication connection, the network data statistics service is started; The network data of the communication connection is monitored through the listening interface, and the network data is sent to the network data statistics service for broadcasting.

8. The control method according to claim 1, characterized in that, The raw data includes raw audio data, and the method further includes: After successfully establishing the communication connection, the voice service is started; The original audio data is sent to the audio service for processing via the network streaming service. The processed audio data is sent to the speaker device for audio playback.

9. The control method according to claim 8, characterized in that, The control method further includes: Acquire the voice data received by the microphone device and process the voice data; The processed voice data is transmitted back to the network streaming service via the voice service and then sent to the vehicle.

10. The control method according to any one of claims 1 to 9, characterized in that, The control method further includes: After the client service receives the service order, it starts the configuration service; The configuration service determines the configuration service information based on the service order and sends the configuration service information back to the client service, so that the client service sends the configuration service information to each service used to implement the remote driving service through the configuration interface for service configuration.

11. A control device for a remote cockpit, characterized in that, include: The first receiving module is used to receive service orders for remote driving services; The startup module is used to determine vehicle information based on the service order and to start a network stream service to establish a communication connection with the vehicle based on the vehicle information. The second receiving module is used to receive the vehicle's raw data in real time and process the raw data to obtain image data and driving data when a network connection is successfully established. The display module is used to call the display service to display the image data and / or the driving data; The control module is used to send the driving data to the control service so that the safety operator in the remote cockpit can perform control operations on the service order and send the first operation data of the control operation back to the vehicle to remotely control the vehicle. Before receiving the service order for the remote driving service through the first receiving module, the control device is further configured to: The core services of the remote cockpit, including client services, configuration services, interaction services, and network flow services, are initialized as second-layer nodes in the head node. Establish a network connection through the client service and receive service orders for the remote driving service; The head node and the second-level nodes constitute a structure tree. Each node in the structure tree includes a corresponding array containing sub-services and a corresponding parent node. Each node has a unique and non-repeating id, and all nodes are located at the position defined by the id in the array of their corresponding parent nodes.

12. A remote-controlled cockpit, characterized in that, The remote cockpit includes a memory and a processor, the memory storing a computer program that, when executed by the processor, implements the method according to any one of claims 1-10.

13. A non-volatile computer-readable storage medium for computer programs, characterized in that, When the computer program is executed by one or more processors, it implements the method according to any one of claims 1-10.

Images