Intelligent environmental sanitation fleet remote driving control system and method

Abstract

The invention discloses an intelligent sanitation vehicle team remote driving control system and method, and the system comprises a client, a cloud system, a plurality of vehicle end systems and intelligent sanitation vehicles, each intelligent sanitation vehicle is provided with a vehicle end system, a remote operator inputs a remote control instruction at the client and issues the remote control instruction through remote control topics. The cloud system receives and stores the remote control topic, a vehicle end system on the intelligent sanitation vehicle receives the subscribed remote control topic and realizes automatic driving of the intelligent sanitation vehicle, and the vehicle end system uploads real-time operation data and video data of the intelligent sanitation vehicle in the automatic driving process to the cloud system; the cloud system receives and stores the real-time operation data and the video data and forwards the real-time operation data and the video data, the client receives and displays the operation data and the video data in real time, and a remote control instruction is adjusted according to a display result to reissue a remote control topic. According to the system and the method, the system cost can be reduced, a data transmission chain is shortened, and the real-time performance of the system is improved.

Description

Technical Field

[0001] The present invention relates to the field of remote driving control technology, and in particular to a remote driving control system and method for an intelligent sanitation fleet. Background Art

[0002] Mechanized cleaning operations have basically replaced traditional manual cleaning operations. Mechanized operations have greatly improved cleaning efficiency and reduced the intensity of manual operations. With the continuous increase in the rate of mechanized operations, problems such as driver laziness, empty driving, and inadequate cleaning have been exposed. The continuous marketization of cleaning operations has also increased the incidence of problems and the difficulty of supervision. The development of intelligent network technology and collaborative intelligence has provided key support for the implementation of unmanned driving in the commercial field, especially in unmanned operation application scenarios such as smart sanitation vehicles, which have brought many advantages: intelligent network technology realizes real-time communication between smart sanitation vehicles and cloud systems through high-speed communication networks, promotes data sharing, and helps smart sanitation fleets to work together. Through mutual communication, they can make collaborative decisions and jointly optimize path planning, operation plans, etc., to improve overall efficiency. The combination of autonomous driving and remote driving technology can achieve a more flexible, intelligent and safe driving experience. This combination can solve some technical and practical challenges faced by autonomous driving alone. In addition, smart sanitation vehicles are equipped with sensors for environmental perception and autonomous decision-making to ensure autonomous driving capabilities. The technology that combines autonomous driving and remote driving can play an important role in improving safety, solving complex scenarios, reducing technical difficulty, etc., and provide a more comprehensive solution for the future commercialization of intelligent transportation and driverless driving. Summary of the invention

[0003] In response to the problems in existing remote driving control, such as long data links, the need for multiple communication protocols leading to signal delays, data loss, unstable control, etc., this invention application proposes a remote driving control system and method for an intelligent sanitation fleet, which realizes remote driving control of an intelligent sanitation fleet by building a collaborative work system.

[0004] On one hand, the present invention provides a remote driving control system for an intelligent sanitation fleet, which includes a client, a cloud system, multiple vehicle-side systems, and an intelligent sanitation fleet. The intelligent sanitation fleet includes the same number of intelligent sanitation vehicles as the multiple vehicle-side systems. Each intelligent sanitation vehicle is provided with a vehicle-side system. The client communicates with the multiple vehicle-side systems through the cloud system, wherein: The client is used to provide a control interface for the remote operator. The remote operator inputs remote control instructions through the control interface and publishes the remote control instructions through the remote control topic. It is also used to receive and display the real-time operation data and video data of the smart sanitation vehicle during the automatic driving process sent by the cloud system. The cloud system is used to receive and store remote control topics published by the client; it is also used to receive and store real-time operation data and video data of the smart sanitation vehicle during the autonomous driving process sent by the vehicle-side system; The vehicle-side system on the smart sanitation vehicle is used to subscribe to and receive remote control topics that match the corresponding smart sanitation vehicle IDs, and realize automatic driving of the corresponding smart sanitation vehicle based on the received remote control topics; it is also used to upload the real-time operation data and video data of the smart sanitation vehicle during the automatic driving process to the cloud system.

[0005] Preferably, the client includes a simulated cockpit and a front-end management module, the simulated cockpit and the front-end management module are respectively integrated with an MQTT proxy module and an MQTT communication module, the simulated cockpit communicates based on the MQTT protocol, and the front-end management module realizes real-time data exchange between the client and the cloud system through the MQTT protocol and the HTTP protocol.

[0006] Preferably, the cloud system includes a streaming media server, an application server, an EMQX message server, a database and a cache module. The EMQX message server exchanges information with the simulation cockpit and the front-end management module respectively through the MQTT protocol. The application server exchanges information with the EMQX message server through the MQTT protocol. The application server also realizes real-time data interaction with the front-end management module through the HTTP protocol. The streaming media server is used to receive real-time video data of the intelligent sanitation vehicle and transmit it to the client. The EMQX message server is used to process the remote control topic published or subscribed through the MQTT protocol. The application server is used to receive the operation data corresponding to the remote control topic and parse the operation data. The database is used to store the operation data of the intelligent sanitation fleet processed by the application server; The cache module is used to store other non-operational data that is frequently accessed within a short period of time.

[0007] Preferably, the vehicle-side system includes a vehicle controller and an automatic driving system. The automatic driving system is connected to the vehicle controller via a CAN bus. A data transceiver service module is provided on the automatic driving system. The data transceiver service module is connected to the cloud system. The data transceiver service module receives the subscribed remote control topic from the EMQX message server of the cloud system and forwards it to the automatic driving system. The automatic driving system converts the received remote control topic into a CAN protocol. The vehicle controller receives the CAN protocol and controls the intelligent sanitation vehicle to realize automatic driving. The data transceiver service module receives the operating data of the intelligent sanitation vehicle during the automatic driving process and sends it to the EMQX message server of the cloud system. The data transceiver service module also receives a video trigger instruction from the user end, thereby triggering the automatic driving system of the intelligent sanitation vehicle to send the video stream data to the streaming media server of the cloud system.

[0008] Preferably, the intelligent sanitation vehicle is provided with several sub-control systems and angle sensors, and the whole vehicle controller is also connected to the several sub-control systems and angle sensors on the intelligent sanitation vehicle through the CAN bus. The whole vehicle controller receives and processes the CAN protocol to generate corresponding control instructions for the several sub-control systems and the automatic driving system. The automatic driving system receives the corresponding control instructions and is responsible for sensing, making decisions and controlling the behavior of the intelligent sanitation vehicle. The several sub-control systems receive the corresponding control instructions and realize the control of the intelligent sanitation vehicle.

[0009] Another aspect of the present invention provides a remote driving control method for an intelligent sanitation fleet, which uses the intelligent sanitation fleet remote driving control system as described above to remotely drive and control the intelligent sanitation fleet, and the method comprises the following steps: S1. The remote operator inputs the remote control command on the client and publishes it through the remote control topic. The cloud system receives and stores the remote control topic. S2. The vehicle-side system on the smart sanitation vehicle subscribes to the remote control topic that matches the corresponding smart sanitation vehicle ID from the cloud system, and controls the smart sanitation vehicle to achieve automatic driving according to the received remote control topic; S3, obtaining real-time operation data and video data of the intelligent sanitation vehicle during the automatic driving process, and sending the real-time operation data and video data to the cloud system through the vehicle-side system on the intelligent sanitation vehicle; S4, the cloud system receives and stores the real-time operation data of the intelligent sanitation vehicle, and the cloud system also receives and forwards the real-time video data of the intelligent sanitation vehicle; S5. The client receives and displays the real-time operation data and video data of the smart sanitation vehicle, adjusts the remote control instructions according to the display results, and republishes the remote control topic to realize remote driving and control of the smart sanitation fleet.

[0010] Preferably, the remote control topic in S1 includes a remote control topic and an operation control topic, the remote control topic is used to implement driving control, and the operation control topic is used to implement control of sanitation actions.

[0011] Preferably, the cloud system includes a streaming media server, an application server, an EMQX message server, a database and a cache module, and S4 specifically includes the following steps: S41, the EMQX message server on the cloud system receives the real-time operation data of the smart sanitation vehicle and sends it to the application server; S42, the application server on the cloud system receives and analyzes the real-time operation data of the intelligent sanitation vehicle, and stores the analyzed data in a database or a cache module according to the call frequency; S43. The streaming media server on the cloud system receives the real-time video data of the smart sanitation vehicle and forwards it to the client.

[0012] The above-mentioned intelligent sanitation fleet remote driving control system and method include a client, a cloud system, multiple vehicle-side systems and an intelligent sanitation fleet. The intelligent sanitation fleet includes the same number of intelligent sanitation vehicles as the multiple vehicle-side systems. Each intelligent sanitation vehicle is provided with a vehicle-side system. The client communicates with the multiple vehicle-side systems through the cloud system. The client provides a control interface for the remote operator. The remote operator publishes a remote control topic through the control interface. The cloud system receives and stores the remote control topic published by the client. The vehicle-side system on the intelligent sanitation vehicle receives the subscribed remote control topic and realizes the automatic driving of the intelligent sanitation vehicle. The vehicle-side system uploads the real-time operation data and video data of the intelligent sanitation vehicle during the automatic driving process to the cloud system. The cloud system stores the real-time operation data on the intelligent sanitation vehicle and forwards the real-time operation data and video data on the intelligent sanitation vehicle to the client. The client receives and displays the received real-time operation data and video data on the intelligent sanitation vehicle in real time, and adjusts the remote control instructions according to the display results. The system and method can greatly shorten the data chain, not only greatly reduce the system cost, but also improve the real-time and robustness of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a structural schematic diagram of a remote driving control system for an intelligent sanitation fleet in one embodiment of the present invention; Figure 2 is a schematic diagram of the structure of a client in an embodiment of the present invention; Figure 3 is a schematic diagram of the structure of a cloud system in one embodiment of the present invention; Figure 4 is a schematic structural diagram of a vehicle-side system in one embodiment of the present invention; Figure 5 It is an overall structural diagram of a remote driving control system for an intelligent sanitation fleet in one embodiment of the present invention; Figure 6 It is a flow chart of a remote driving control method for an intelligent sanitation fleet in one embodiment of the present invention. DETAILED DESCRIPTION

[0014] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings.

[0015] A remote driving control system for an intelligent sanitation fleet, see Figure 1 and Figure 5, including a client, a cloud system, multiple vehicle-side systems and an intelligent sanitation fleet. The intelligent sanitation fleet includes the same number of intelligent sanitation vehicles as the multiple vehicle-side systems. Each intelligent sanitation vehicle is equipped with a vehicle-side system. The client communicates with the multiple vehicle-side systems through the cloud system, including: The client is used to provide a control interface for the remote operator. The remote operator inputs remote control instructions through the control interface and publishes the remote control instructions through the remote control topic. It is also used to receive and display the real-time operation data and video data of the smart sanitation vehicle during the automatic driving process sent by the cloud system. The cloud system is used to receive and store remote control topics published by the client; it is also used to receive and store real-time operation data and video data of the smart sanitation vehicle during the autonomous driving process sent by the vehicle-side system; The vehicle-side system on the smart sanitation vehicle is used to subscribe to and receive remote control topics that match the corresponding smart sanitation vehicle IDs, and realize automatic driving of the corresponding smart sanitation vehicle based on the received remote control topics; it is also used to upload the real-time operation data and video data of the smart sanitation vehicle during the automatic driving process to the cloud system.

[0016] The above-mentioned intelligent sanitation fleet remote driving control system forms a collaborative working system through the client, cloud system, vehicle-side system and an intelligent sanitation fleet composed of multiple intelligent sanitation vehicles to realize remote driving and control of intelligent sanitation vehicles.

[0017] In one embodiment, see Figure 2 and Figure 5 The client includes a simulation cockpit and a front-end management module. The simulation cockpit and the front-end management module are respectively integrated with the MQTT agent module and the MQTT communication module. The simulation cockpit communicates based on the MQTT protocol, and the front-end management module realizes real-time data exchange between the client and the cloud system through the MQTT protocol and HTTP protocol.

[0018] Specifically, the client includes a simulated cockpit and a front-end management module. The simulated cockpit provides an intuitive and friendly control interface for the remote operator, and provides control devices similar to real driving, such as a steering wheel, pedals, and gear levers. The remote operator uses these devices to remotely control the smart sanitation vehicle. Remote control includes driving control, operation control, task scheduling, etc.

[0019] The front-end management module includes a user interface that allows for visual management of the smart sanitation vehicle fleet, including visualization of operational data and video data. For example, the user interface can be used to conduct real-time video monitoring of the surroundings of the smart sanitation vehicle and display the status information of the smart sanitation vehicle, such as battery power, vehicle speed, sensor status, etc., so that remote operators can promptly understand the relevant information and situation of the smart sanitation vehicle.

[0020] In addition, in an emergency, the operator can perform emergency operations through the client to ensure the system safety of the smart sanitation vehicle; it also allows multiple remote operators to perform remote collaborative operations at the same time (one simulated cockpit can be bound to a smart sanitation vehicle for remote control), simulating the real collaborative driving and operation scenarios of a smart sanitation fleet.

[0021] The detailed design process of the client with the above functions is described as follows: The front-end management module integrates the MQTT proxy module and the MQTT communication module. The front-end management module can establish a connection with the cloud system through the HTTP protocol and the MQTT protocol to achieve data exchange between the client and the cloud system. For example, the MQTT protocol can be used to publish remote control topics, such as the status of the smart sanitation vehicle, sensor data, etc., to achieve real-time data push and update.

[0022] The front-end management module also provides an HTTP interface for intelligent sanitation fleet management, allowing remote operators to request registration, configuration and management of intelligent sanitation fleet information through the HTTP interface. The unified user interface enables remote operators to easily manage and configure intelligent sanitation fleets and monitor them in real time.

[0023] The simulated cockpit is also integrated with the MQTT proxy module and the MQTT communication module, which can realize real-time communication between the simulated cockpit and the cloud system through the MQTT protocol. In the simulated cockpit, remote control topics can be published to the cloud system through the MQTT protocol, or the status of the smart sanitation vehicle and sensor data can be obtained from the cloud system. The front-end management module also has built-in webRTC, which supports the playback of audio and video streaming data.

[0024] You can also remotely take over the target smart sanitation vehicle through the client. The specific method is as follows: a. The remote operator logs in to the user interface set on the client, enters the single vehicle supervision page, connects to the target smart sanitation vehicle through the user interface of the front-end management module, obtains the target smart sanitation vehicle ID, and binds the simulated cockpit to the target smart sanitation vehicle. The binding status of the simulated cockpit will be managed by the front-end management module.

[0025] b. After binding the simulated cockpit, the remote operator sends a "start video stream" command on the client. After receiving the "start video stream" command, the cloud system sends a streaming request to the vehicle-side system of the target intelligent sanitation vehicle. The automatic driving system of the target intelligent sanitation vehicle pushes the corresponding video stream data to the streaming media server in the cloud. The streaming media server returns a video stream address to the client. The client obtains the video stream address and plays the video stream data of the intelligent sanitation vehicle.

[0026] c. After binding the simulated cockpit, the remote operator selects the "remote driving" mode on the client and selects the gear mode. The simulated cockpit enters the remote control of the smart sanitation vehicle. The parameters related to the remote control (such as the steering and speed of the smart sanitation vehicle) are sent to the EMQX message server according to the preset cycle.

[0027] d. Each vehicle-side system subscribes to the remote control topic that matches its corresponding smart sanitation vehicle ID to achieve remote control of the smart sanitation vehicle.

[0028] There are two types of remote control topics: remote control topics and operation control topics. Remote control topics are used to implement driving control, and operation control topics are used to implement sanitation action control.

[0029] Remote control topics: province / city / park / sanitation vehicle / intelligent sanitation vehicle ID / remote control. The remote control topics include acceleration, deceleration, steering, gear position and other instructions. These instructions are issued periodically to achieve real-time remote control of the intelligent sanitation vehicle.

[0030] Operation control topics: province / city / park / sanitation vehicle / intelligent sanitation vehicle ID / operation control. Operation control topics include: lifting, opening and closing, speed adjustment of the sweeping disc on the intelligent sanitation vehicle, fan switch and speed adjustment, suction cup lifting, dumping and returning of the garbage bin, opening and closing of the garbage bin door, etc. At the same time, in order to improve the convenience of the operation of the intelligent sanitation vehicle, a one-button start method is designed, allowing the intelligent sanitation vehicle to perform cleaning operations according to the preset operation sequence (such as suction cup lowering, fan starting, sweeping disc starting).

[0031] The user operates by periodically publishing these two remote control topics and the vehicle side subscribes to related remote control topics, thereby realizing the control of the smart sanitation vehicle.

[0032] The simulation cockpit can be deployed according to actual factors such as the number of parks and the scale of the smart sanitation vehicle. Each park can have its own simulation cockpit, or several parks can share one simulation cockpit. It can be flexibly deployed according to the scale of the smart sanitation vehicle to ensure that even if the simulation of one park fails, the simulation cockpits of other parks can still operate normally.

[0033] In one embodiment, see Figure 3 and Figure 5The cloud system includes a streaming media server, an application server, an EMQX message server, a database and a cache module. The EMQX message server exchanges information with the simulation cockpit and the front-end management module through the MQTT protocol. The application server exchanges information with the EMQX message server through the MQTT protocol. The application server also realizes real-time data interaction with the front-end management module through the HTTP protocol. The streaming media server is used to receive the real-time video data of the intelligent sanitation vehicle and transmit it to the client. The EMQX message server is used to process the remote control topics published or subscribed through the MQTT protocol. The application server is used to receive the operation data corresponding to the remote control topic and parse the operation data; The database is used to store the operation data of the intelligent sanitation fleet processed by the application server; The cache module is used to store other non-operational data that is frequently accessed within a short period of time.

[0034] Specifically, as the core component of the remote driving control system of the intelligent sanitation fleet, the cloud system is responsible for processing and coordinating the data flow and services between the client and the vehicle-side system installed on each intelligent sanitation vehicle.

[0035] The cloud system includes streaming media server, application server, EMQX message server, database and cache module. The functions of each part are as follows: 1. EMQX message server EMQX message server is an MQTT message proxy server, which is used to process the message publishing and subscription of MQTT protocol, and realize the transmission of subscription topics or publishing topics between the client and various terminal systems. EMQX message server supports large-scale distributed architecture to meet the real-time communication requirements of large-scale vehicle-side systems and clients.

[0036] The EMQX message server is responsible for receiving and checking the consistency and integrity between the topics published by the client and the topics subscribed by each vehicle-side system. The details are as follows: Data parsing based on the MQTT protocol parses and checks the original data sent from the client or each terminal system to see if it meets the agreed message format, message length, QoS level, identity authentication, security check, etc. For example, if the protocol says that the data length is 100 bytes, the EMQX message service will check whether the data length in the remote control topic is 100 bytes. Among them, the original data of the terminal system includes the status of the smart sanitation vehicle, sensor data, positioning information, etc., and the original data of the client includes the remote control instructions of the client.

[0037] At the same time, high-speed, low-latency communication networks (such as 5G) are used for data transmission to ensure real-time and reliability.

[0038] 2. Application Server The application server is a component that processes business logic, responsible for user management and authority control, processing remote control commands corresponding to remote control topics from the client, and processing data from smart sanitation vehicles corresponding to each vehicle-side system (data comes from controllers and sensors installed on each smart sanitation vehicle, etc.), and monitoring and managing these data. The details are as follows: 1) User management and authority control: Manage the accounts of remote operators and set account permissions. Only authorized users can access and control the smart sanitation vehicles in the corresponding park to ensure system security.

[0039] 2) Data storage and analysis: The operation data of each smart sanitation vehicle is stored to support subsequent data mining and statistical analysis; the operation data of each smart sanitation vehicle can also be sent to the client for data display, and the remote operator can understand the status changes of the smart sanitation vehicle through the operation data.

[0040] The application server receives the relevant topics subscribed by the client and executes the corresponding business logic. The relevant topics include: province / city / park / sanitation vehicle / intelligent sanitation vehicle ID / intelligent sanitation vehicle status, province / city / park / sanitation vehicle / intelligent sanitation vehicle ID / battery status, province / city / park / sanitation vehicle / intelligent sanitation vehicle ID / network status, province / city / park / sanitation vehicle / intelligent sanitation vehicle ID / intelligent sanitation vehicle perception results, etc. The perception results may include vehicles, pedestrians, bicycles, etc.

[0041] The application server also receives the video stream address uploaded by each vehicle-side system and the operating data of the smart sanitation vehicle (including position, speed, steering wheel angle, gear position, throttle opening, power, battery status, etc.), stores these operating data in the database, and stores non-operating data such as video stream address, password, data dictionary in the cache module. At the same time, the operating data is sent to the client for display, and the perception results of objects around the smart sanitation vehicle are reported in real time, displayed in real time on the client, and related fault status is reported, so that the client can understand the operating status and surrounding conditions of the smart sanitation vehicle in real time, and make analysis and judgment.

[0042] In order to improve the reading performance, other data that are frequently accessed in a short period of time are stored in the cache module. These data include user login authentication data, intelligent sanitation vehicle control mode, the relationship between the intelligent sanitation vehicle and the remote operator, current limiting measures, etc.

[0043] 3) Service status monitoring: Real-time monitoring of the operating status of each key service in the platform, including message service, video service, data service, etc. Set alarm rules to promptly notify of service anomalies or downtime.

[0044] 4) Server performance monitoring: monitor the server's CPU usage, memory usage, disk space and other performance indicators. By monitoring the server's operation and resource utilization, users can promptly discover and resolve potential problems to ensure that the platform always maintains high availability and stability.

[0045] 3. Streaming Media Server: The streaming media server is used to receive and transmit real-time video data of the smart sanitation vehicle. The streaming media server accesses each vehicle-side system through Rtmp to obtain the real-time video stream of the smart sanitation vehicle. The streaming media server is also connected to the client through webrtc to provide real-time video stream data to the remote operator.

[0046] It should be noted that there can be multiple streaming media servers, application servers, and EMQX message servers mentioned above. Multiple streaming media servers, multiple application servers, and multiple EMQX message servers can respectively form a streaming media server cluster, an application server cluster, and an EMQX message server cluster to better meet the needs of large-scale applications.

[0047] In one embodiment, see Figure 4 and Figure 5 The vehicle-side system includes a vehicle controller BCM and an automatic driving system. The automatic driving system is connected to the vehicle controller BCM through a CAN bus. A data transceiver service module is set on the automatic driving system. The data transceiver service module is connected to the cloud system. The data transceiver service module receives the subscribed remote control topic from the EMQX message server of the cloud system and forwards it to the automatic driving system, which converts it into a CAN protocol. The vehicle controller BCM receives the CAN protocol and controls the intelligent sanitation vehicle to achieve automatic driving. The data transceiver service module receives the operating data of the intelligent sanitation vehicle during the automatic driving process and sends it to the EMQX message server of the cloud system. The data transceiver service module also receives a video trigger command from the user end, thereby triggering the automatic driving system of the intelligent sanitation vehicle to send the video stream data to the streaming media server of the cloud system.

[0048] Furthermore, the intelligent sanitation vehicle is provided with several sub-control systems and angle sensors. The vehicle controller BCM is also connected to the several sub-control systems and angle sensors on the intelligent sanitation vehicle through the CAN bus. The vehicle controller BCM receives and processes the CAN protocol to generate corresponding control instructions for the automatic driving system and several sub-control systems. The automatic driving system receives the corresponding control instructions and is responsible for sensing, making decisions and controlling the behavior of the intelligent sanitation vehicle (for example, sensing the surrounding dynamic targets and running speed, thereby adjusting its own running speed and trajectory, and then controlling the longitudinal and lateral movement of the vehicle). Several sub-control systems receive the corresponding control instructions and realize the control of the intelligent sanitation vehicle.

[0049] Furthermore, several sub-control systems on the intelligent sanitation vehicle include: motor control system, brake control system, steering control system, battery management system, DC-DC converter, power distribution unit, human-computer interaction system, etc.

[0050] Specifically, the intelligent sanitation vehicle is equipped with an angle sensor, a braking control system, a steering control system, a motor control system, a battery management system, a human-computer interaction system, a DC-DC converter, and a power distribution unit. The vehicle-side system is a system installed on the intelligent sanitation vehicle, including a vehicle controller BCM and an automatic driving system. The vehicle controller BCM is connected to the above-mentioned sub-control systems through the CAN bus. A data transceiver service module is set on the automatic driving system, and the data transceiver service module is connected to the cloud system. The data transceiver service module receives the subscribed remote control topic from the EMQX message server of the cloud system and forwards it to the automatic driving system, which converts it into a CAN protocol. The vehicle controller BCM receives the CAN protocol and controls the intelligent sanitation vehicle to realize remote automatic driving. The data transceiver service module receives the operation data of the intelligent sanitation vehicle and sends it to the EMQX message server of the cloud system. The data transceiver service module also receives the video trigger command from the user end, thereby triggering the automatic driving system of the intelligent sanitation vehicle to send the video stream data to the streaming media server of the cloud system.

[0051] The main functions of the vehicle-side system include: 1) Video data collection and uploading: Through the angle sensor, camera and laser radar configured on the smart sanitation vehicle ( Figure 4 The smart sanitation vehicle is sensed by a sensor (not shown) and the like, and the video stream data is collected and sent to the streaming media server of the cloud system through a data transceiver service module provided on the automatic driving system, wherein the angle sensor is provided on the chassis of the smart sanitation vehicle, and the camera and the lidar are provided on the automatic driving system.

[0052] 2) Operation data collection and upload: The real-time operation data of the intelligent sanitation vehicle is collected through the electronic controller of each sub-control system, and the real-time operation data is uploaded to the EMQX message server of the cloud system through the data transceiver service module set on the automatic driving system. The application server obtains relevant data from the EMQX message server and monitors and analyzes it.

[0053] 3) Autonomous driving and control: The vehicle controller receives and processes the CAN protocol, generates corresponding control instructions for several sub-control systems and autonomous driving systems, and the autonomous driving system receives the corresponding control instructions and is responsible for sensing, making decisions, and controlling the behavior of the intelligent sanitation vehicle. Several sub-control systems receive the corresponding control instructions and realize the control of the intelligent sanitation vehicle. This architecture helps to realize the integrated control of the intelligent sanitation vehicle, so that each subsystem can work together to realize the operation and control of the intelligent sanitation vehicle.

[0054] For example, for the remote control topic published by the client, the data transceiver service module receives the subscribed remote control topic from the EMQX message server of the cloud system and forwards it to the automatic driving system, which converts it into the CAN protocol. The vehicle controller receives the CAN protocol and controls the intelligent sanitation vehicle to achieve automatic driving. For example, the steering, braking, acceleration and deceleration instructions in the subscribed remote control topic are sent to the steering control system, braking control system, and travel motor control system to achieve remote driving of the intelligent sanitation vehicle; the sweeping disc rotation, suction nozzle lifting, and fan remote control instructions in the subscribed remote control topic are sent to the motor controller, and the motor controller drives the intelligent sanitation vehicle to work and complete the corresponding operation instructions.

[0055] In one embodiment, see Figure 6 , a remote driving control method for an intelligent sanitation fleet, the method comprising the following steps: S1. The remote operator inputs the remote control command on the client and publishes it through the remote control topic. The cloud system receives and stores the remote control topic. S2. The vehicle-side system on the smart sanitation vehicle subscribes to the remote control topic that matches the corresponding smart sanitation vehicle ID from the cloud system, and controls the smart sanitation vehicle to achieve automatic driving according to the received remote control topic; S3, obtaining real-time operation data and video data of the intelligent sanitation vehicle during the automatic driving process, and sending the real-time operation data and video data to the cloud system through the vehicle-side system on the intelligent sanitation vehicle; S4, the cloud system receives and stores the real-time operation data of the intelligent sanitation vehicle, and the cloud system also receives and forwards the real-time video data of the intelligent sanitation vehicle; S5. The client receives and displays the real-time operation data and video data of the smart sanitation vehicle, adjusts the remote control instructions according to the display results, and republishes the remote control topic to realize remote driving and control of the smart sanitation fleet.

[0056] Furthermore, the remote control topic in S1 includes a remote control topic and an operation control topic. The remote control topic is used to implement driving control, and the operation control topic is used to implement control of sanitation actions.

[0057] Furthermore, the cloud system includes a streaming media server, an application server, an EMQX message server, a database and a cache module. S4 specifically includes the following steps: S41, the EMQX message server on the cloud system receives the real-time operation data of the smart sanitation vehicle and sends it to the application server; S42, the application server on the cloud system receives and analyzes the real-time operation data of the intelligent sanitation vehicle, and stores the analyzed data in a database or a cache module according to the call frequency; S43. The streaming media server on the cloud system receives the real-time video data of the smart sanitation vehicle and forwards it to the client.

[0058] For the specific limitations of a remote driving control method for an intelligent sanitation fleet, please refer to the limitations of a remote driving control system for an intelligent sanitation fleet mentioned above, which will not be repeated here.

[0059] The above-mentioned intelligent sanitation fleet remote driving control system and method include a client, a cloud system, multiple vehicle-side systems and an intelligent sanitation fleet. The intelligent sanitation fleet includes the same number of intelligent sanitation vehicles as the multiple vehicle-side systems. Each intelligent sanitation vehicle is equipped with a vehicle-side system. The client communicates with the multiple vehicle-side systems through the cloud system respectively. The client provides a control interface for the remote operator. The remote operator publishes remote control topics through the control interface. The cloud system receives and stores the remote control topics published by the client. The vehicle-side system on the intelligent sanitation vehicle receives the subscribed remote control topics and realizes automatic driving of the intelligent sanitation vehicle. The vehicle-side system uploads the real-time operation data and video data of the intelligent sanitation vehicle during the automatic driving process to the cloud system. The cloud system stores the real-time operation data on the intelligent sanitation vehicle and forwards the real-time operation data and video data on the intelligent sanitation vehicle to the client. The client receives and displays the received real-time operation data and video data on the intelligent sanitation vehicle in real time, and adjusts the remote control instructions according to the display results. The system and method integrate the vehicle-side data receiving and sending services into the autonomous driving system, greatly reducing the system cost. The cloud system adopts a unified MQTT protocol and has topic parsing, data storage and receiving and sending functions, which greatly shortens the data chain and improves the real-time and robustness of the system. The cloud system adopts a distributed architecture and can be expanded according to the scale of the networked smart sanitation vehicles.

[0060] The above is a detailed introduction to a remote driving control system and method for an intelligent sanitation fleet provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the core idea of ​​the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A remote driving control system for an intelligent sanitation fleet, characterized in that: The system includes a client, a cloud system, multiple vehicle-side systems and an intelligent sanitation fleet. The intelligent sanitation fleet includes the same number of intelligent sanitation vehicles as the multiple vehicle-side systems. Each intelligent sanitation vehicle is equipped with a vehicle-side system. The client communicates with the multiple vehicle-side systems through the cloud system, wherein: The client is used to provide a control interface for the remote operator. The remote operator inputs remote control instructions through the control interface and publishes the remote control instructions through the remote control topic. It is also used to receive and display the real-time operation data and video data of the smart sanitation vehicle during the automatic driving process sent by the cloud system. The cloud system is used to receive and store remote control topics published by the client; it is also used to receive and store real-time operation data and video data of the smart sanitation vehicle during the autonomous driving process sent by the vehicle-side system; The vehicle-side system on the smart sanitation vehicle is used to subscribe to and receive remote control topics that match the corresponding smart sanitation vehicle IDs, and realize automatic driving of the corresponding smart sanitation vehicle based on the received remote control topics; it is also used to upload the real-time operation data and video data of the smart sanitation vehicle during the automatic driving process to the cloud system.

2. The intelligent sanitation fleet remote driving control system according to claim 1, characterized in that: The client includes a simulated cockpit and a front-end management module. The simulated cockpit and the front-end management module are respectively integrated with the MQTT proxy module and the MQTT communication module. The simulated cockpit communicates based on the MQTT protocol, and the front-end management module realizes real-time data exchange between the client and the cloud system through the MQTT protocol and HTTP protocol.

3. The intelligent sanitation fleet remote driving control system as claimed in claim 2, characterized in that: The cloud system includes a streaming media server, an application server, an EMQX message server, a database and a cache module. The EMQX message server exchanges information with the simulated cockpit and the front-end management module through the MQTT protocol. The application server exchanges information with the EMQX message server through the MQTT protocol. The application server also exchanges real-time data with the front-end management module through the HTTP protocol. The streaming media server is used to receive real-time video data of the intelligent sanitation vehicle and transmit it to the client. The EMQX message server is used to process remote control topics published or subscribed through the MQTT protocol. The application server is used to receive the operating data corresponding to the remote control topic and parse the operating data. The database is used to store the operation data of the intelligent sanitation fleet processed by the application server; The cache module is used to store other non-operational data that is frequently accessed within a short period of time.

4. The remote driving control system for the intelligent sanitation fleet as claimed in claim 3, characterized in that: The vehicle-side system includes a vehicle controller and an automatic driving system. The automatic driving system is connected to the vehicle controller through a CAN bus. A data transceiver service module is set on the automatic driving system. The data transceiver service module is connected to the cloud system. The data transceiver service module receives the subscribed remote control topic from the EMQX message server of the cloud system and forwards it to the automatic driving system. The automatic driving system converts the received remote control topic into a CAN protocol. The vehicle controller receives the CAN protocol and controls the intelligent sanitation vehicle to achieve automatic driving. The data transceiver service module receives the operating data of the intelligent sanitation vehicle during the automatic driving process and sends it to the EMQX message server of the cloud system. The data transceiver service module also receives video trigger instructions from the user end, thereby triggering the automatic driving system of the intelligent sanitation vehicle to send video stream data to the streaming media server of the cloud system.

5. The intelligent sanitation fleet remote driving control system as claimed in claim 4, characterized in that: The intelligent sanitation vehicle is equipped with several sub-control systems and angle sensors. The vehicle controller is also connected to several sub-control systems and angle sensors on the intelligent sanitation vehicle through the CAN bus. The vehicle controller receives and processes the CAN protocol to generate corresponding control instructions for several sub-control systems and the automatic driving system. The automatic driving system receives the corresponding control instructions and is responsible for sensing, making decisions and controlling the behavior of the intelligent sanitation vehicle. Several sub-control systems receive the corresponding control instructions and realize the control of the intelligent sanitation vehicle.

6. A remote driving control method for an intelligent sanitation fleet, the method using the remote driving control system for an intelligent sanitation fleet as claimed in any one of claims 1 to 5 to remotely drive and control the intelligent sanitation fleet, characterized in that: The method comprises the following steps: S1. The remote operator inputs the remote control command on the client and publishes it through the remote control topic. The cloud system receives and stores the remote control topic. S2. The vehicle-side system on the smart sanitation vehicle subscribes to the remote control topic that matches the corresponding smart sanitation vehicle ID from the cloud system, and controls the smart sanitation vehicle to achieve automatic driving according to the received remote control topic; S3, obtaining real-time operation data and video data of the intelligent sanitation vehicle during the automatic driving process, and sending the real-time operation data and video data to the cloud system through the vehicle-side system on the intelligent sanitation vehicle; S4, the cloud system receives and stores the real-time operation data of the intelligent sanitation vehicle, and the cloud system also receives and forwards the real-time video data of the intelligent sanitation vehicle; S5. The client receives and displays the real-time operation data and video data of the smart sanitation vehicle, adjusts the remote control instructions according to the display results, and republishes the remote control topic to realize remote driving and control of the smart sanitation fleet.

7. The remote driving control method of the intelligent sanitation fleet according to claim 6, characterized in that: The remote control topics in S1 include remote control topics and operation control topics. The remote control topic is used to realize driving control, and the operation control topic is used to realize the control of sanitation actions.

8. The remote driving control method of the intelligent sanitation fleet according to claim 7, characterized in that: The cloud system includes streaming media server, application server, EMQX message server, database and cache module. S4 specifically includes the following steps: S41, the EMQX message server on the cloud system receives the real-time operation data of the smart sanitation vehicle and sends it to the application server; S42, the application server on the cloud system receives and analyzes the real-time operation data of the intelligent sanitation vehicle, and stores the analyzed data in a database or a cache module according to the call frequency; S43. The streaming media server on the cloud system receives the real-time video data of the smart sanitation vehicle and forwards it to the client.

Images