Unmanned driving system and method for vehicle testing

Abstract

The invention relates to the technical field of vehicle inspection and detection, and discloses an unmanned driving system and method for vehicle testing. And when the test vehicle has an unmanned driving function fault or enters an extreme working condition scene, the test vehicle sends prompt information to a remote driving cabin. And sending video information shot by an out-of-vehicle view shooting device installed on the test vehicle to a remote cockpit. And the remote cockpit receives the prompt information and the video information sent by the test vehicle, and displays the prompt information and the video information on a display device. And responding to the operation of the operation user on the driving simulator to obtain an operation instruction, and sending the operation instruction to the test vehicle. And the test vehicle executes vehicle control operation of the test vehicle according to the operation instruction. Through a mode of combining the remote cockpit and the test vehicle, remote takeover of the test vehicle can be realized at any time, safe operation of the test vehicle is ensured, and potential safety hazards are reduced.

Description

Technical Field

[0001] This specification relates to the field of vehicle inspection and testing technology, and in particular to an unmanned driving system and method for vehicle testing. Background Art

[0002] Currently, with the iterative upgrades in vehicle inspection technology, the industry is placing higher demands on the integrity and accuracy of vehicle testing. Reliability road tests (such as rough road tests) are an indispensable component of vehicle testing. Traditional manual driving, where drivers conduct tests on test vehicles, faces numerous challenges. Firstly, different drivers have varying driving habits and skill levels, which can lead to inconsistent testing, impacting the accuracy and repeatability of test data. Secondly, manual driving poses safety risks under extreme operating conditions.

[0003] Therefore, the current mainstream solution abroad is to use driving robots to replace human operators in vehicle testing. For example, Autonomous Solutions, Inc. (ASI) in the United States has proposed an unmanned reliability road test solution based on driving robots. Its core approach is to replace the human driver's hand and foot movements with actuators installed on the steering wheel, accelerator, and brake pedals, precisely controlling the vehicle's direction, speed, and braking, thereby achieving autonomous driving. However, this solution is limited by the vehicle's lack of a chassis domain controller, autonomous driving master controller, or perception sensors. It cannot establish a complete perception-decision-execution closed loop and can only perform tests following a preset path. For example, the Tianjin Automobile Testing Center of the China Automotive Technology and Research Center's Testing and Certification Division, drawing on ASI's solution, built a vehicle testing system based on the integration of steering, brake, and accelerator robots from AB Dynamics (ABD) in the United Kingdom. However, this testing system also fails to establish a complete perception-decision-execution closed loop and cannot effectively prevent unexpected and dangerous situations encountered during testing.

[0004] To this end, this specification provides an unmanned driving system and method for vehicle testing. Summary of the Invention

[0005] This specification provides an unmanned driving system and method for vehicle testing to partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions: This specification provides an unmanned driving system for vehicle testing, the system comprising a test vehicle and a remote cockpit, wherein: The remote cockpit is communicatively connected to the test vehicle; The test vehicle is configured to: during a vehicle unmanned driving test, when the test vehicle experiences an unmanned driving function failure or enters an extreme operating condition, the test vehicle sends a prompt message to the remote cockpit, the prompt message being used to prompt an operator of the remote cockpit to remotely take over the test vehicle; send video information captured by an external view camera installed on the test vehicle to the remote cockpit; receive operating instructions sent by the remote cockpit, and execute vehicle control operations of the test vehicle according to the operating instructions; The remote cockpit is configured to: receive the prompt information sent by the test vehicle and display the prompt information on the display device; receive the video information and display the video information on the display device; in response to the vehicle control operation of the operating user on the driving simulator in the remote cockpit, determine the operation instruction corresponding to the vehicle control operation, and send the operation instruction to the test vehicle.

[0007] The aforementioned technical approach, combined with a remote cockpit and test vehicle, allows remote control of the test vehicle at any time during autonomous driving testing, ensuring safe operation even in the event of autonomous driving malfunctions or extreme operating conditions, thereby mitigating potential safety hazards. Cameras installed on the test vehicle provide the remote cockpit with data analysis and external field-of-view video data for decision-making, establishing a complete perception-decision-execution closed loop. This effectively prevents unexpected and dangerous situations encountered by the test vehicle during testing.

[0008] Furthermore, the test vehicle includes an on-board driving system and an on-board remote control system, wherein: The on-board driving system is configured to: detect whether the unmanned driving function of the test vehicle fails or whether the test vehicle enters an extreme operating condition; if so, send a control request to the on-board remote control system; if not, the on-board driving system continues to perform the unmanned driving task of the test vehicle; The on-board remote control system is configured to: receive the control request sent by the on-board driving system; acquire in real time the video information captured by the outside view camera installed on the test vehicle, wherein the video information is used to display the real-time situation of the test vehicle in all directions of the front, back, left and right; and send the prompt information to the remote cockpit according to the control request, and send the video information to the remote cockpit.

[0009] According to the above technical means, the on-board driving system can detect in real time whether the unmanned driving function has failed or whether the vehicle has entered an extreme operating condition. Once a problem is found, it will immediately request to switch to remote control mode. This method ensures that even in cases where the unmanned driving function cannot guarantee safety, safety hazards can be effectively reduced. The video information provided by the external field of view camera covers the real-time situation in all directions of the test vehicle, front, back, left, and right, providing the operator of the remote cockpit with a full range of perspectives, helping to more accurately judge the road conditions and environment, and thus make more reasonable driving decisions. The unmanned driving system allows for flexible switching of control modes during the unmanned driving test process. Not only can tasks be automatically performed when the unmanned driving system is working normally, but manual remote control can also be quickly switched to in complex or dangerous situations, ensuring the continuity and efficiency of the test process.

[0010] Furthermore, the test vehicle further includes a wire-controlled chassis controller, and the on-board remote control system is communicatively connected to the wire-controlled chassis controller and the remote cockpit, respectively, wherein: The vehicle-mounted remote control system is configured to: receive the operation instruction sent by the remote cockpit; and send the operation instruction to the wire-controlled chassis controller; The wire-controlled chassis controller is used to receive the operation instruction sent by the vehicle-mounted remote controller and perform the vehicle control operation of the test vehicle according to the operation instruction.

[0011] Using the aforementioned technical approach, the by-wire chassis controller directly receives remote cockpit control commands forwarded from the onboard remote control system and executes specific vehicle control operations, such as acceleration, deceleration, and steering, based on these commands. This ensures accurate and flawless execution of control commands, improving the precision and responsiveness of test vehicle control. By transmitting control commands via electronic signals, by-wire technology replaces traditional mechanical or hydraulic connections, reducing physical wear and failure rates, thereby improving system reliability and safety. Especially in emergency situations, a fast and accurate response is crucial to avoiding accidents. The presence of a by-wire chassis controller makes the system more adaptable to different vehicle models and application scenarios. By simply adjusting the interface between the by-wire chassis controller and the specific vehicle model, it can be quickly deployed on a variety of test vehicles, enhancing the system's versatility and flexibility.

[0012] Furthermore, the vehicle control operation includes at least a throttle control operation on the test vehicle, a brake control operation on the test vehicle, a steering wheel control operation on the test vehicle, a gear control operation on the test vehicle, and a light control operation on the test vehicle.

[0013] Furthermore, the remote cockpit includes a cockpit industrial computer, a driving simulator, and a display device, wherein the cockpit industrial computer is communicatively connected to the test vehicle, the driving simulator, and the display device, respectively, wherein: The cockpit industrial computer is configured to: receive the prompt information sent by the test vehicle and send the prompt information to the display device; receive the video information and send the video information to the display device; receive the operation instruction sent by the driving simulator and send the operation instruction to the test vehicle; The display device is configured to: display the prompt information sent by the cockpit industrial computer after receiving the prompt information, and display the video information after receiving the video information; The driving simulator is configured to: respond to the vehicle control operation of the operating user and convert the vehicle control operation into the operation instruction; and send the operation instruction to the cockpit industrial computer.

[0014] Based on the above technical approach, the cockpit industrial computer, as a core component, is responsible for receiving prompts and video information from the test vehicle and forwarding this information to the corresponding output devices (such as displays). This design enables centralized information processing, ensuring efficient and timely data transmission, allowing operators to quickly obtain the information they need. The display device displays prompts and video information, providing an intuitive user interface. By displaying a real-time view outside the vehicle, operators can accurately judge the vehicle's surroundings and make more precise control decisions. Furthermore, the display of prompts helps operators quickly understand the test vehicle's status, improving their ability to respond to emergencies. The driving simulator responds to the operator's vehicle control operations, converting them into operational commands and sending them to the cockpit industrial computer. This design not only provides a near-realistic driving experience but also allows operators to perform complex driving tasks in a remote environment. This approach ensures effective vehicle control even in the event of unmanned driving system failure or extreme situations.

[0015] Furthermore, the display device is further configured to: in response to the user's selection operation of selecting a test vehicle, display video information corresponding to the test vehicle selected by the selection operation to the user.

[0016] The aforementioned technical measures support simultaneous monitoring and management of multiple test vehicles. Operators can select specific test vehicles and view real-time video information around them, enabling concurrent monitoring and intervention in multiple autonomous driving test missions. A graphical interface or interactive method allows users to quickly select target test vehicles, helping operators quickly locate problem vehicles and take over control promptly, especially in the event of a failure or extreme scenario. The remote cockpit can be centrally deployed and managed, with a single industrial computer and display device serving multiple test vehicles, reducing hardware costs and space requirements. Video information for each test vehicle can be displayed independently or in split-screen mode. Combined with a user-selectable mechanism, this allows operators to gain a more comprehensive understanding of the operating status and surrounding environment of each vehicle. This flexible video switching mechanism is particularly important in complex scenarios such as multi-vehicle platooning testing, vehicle-road collaboration testing, and swarm intelligence testing, enabling the remote cockpit to serve as a unified dispatch center.

[0017] Furthermore, the test vehicle is further configured to: send vehicle status information to the remote cockpit in real time, the vehicle status information at least including execution feedback information of the vehicle control operation and instrument panel information of the test vehicle; The remote cockpit is further configured to receive the vehicle status information and display the vehicle status information to the operating user via the display device.

[0018] Using these technologies, the execution feedback from vehicle control operations can inform remote operators in real time whether their commands have been correctly executed, helping to immediately identify and correct potential operational errors or system failures. Furthermore, by displaying the test vehicle's instrument panel information (such as speed, fuel consumption, and battery life), operators can fully understand the vehicle's operating status and make more accurate judgments and decisions.

[0019] Furthermore, the test vehicle is also configured to: when the test vehicle eliminates the unmanned driving function failure or leaves the extreme working condition scenario, the test vehicle continues to perform the preset unmanned driving task.

[0020] This specification provides an unmanned driving method for vehicle testing, which uses the above-mentioned unmanned driving system for vehicle testing and includes the following steps: S1: During the unmanned driving test of a vehicle, if the unmanned driving function of the test vehicle fails or enters an extreme operating condition, the test vehicle sends a prompt message to the remote cockpit, whereby the operator of the remote cockpit is prompted to remotely take over the test vehicle and to send video information captured by an external view camera installed on the test vehicle to the remote cockpit; S2: the remote cockpit receives the prompt information sent by the test vehicle and displays the prompt information on a display device, and receives video information captured by an external view camera on the test vehicle and displays the video information on the display device; S3: The remote cockpit determines an operation instruction corresponding to the vehicle control operation in response to the vehicle control operation of the operating user on the driving simulator, and sends the operation instruction to the test vehicle; S4: The test vehicle receives the operation instruction sent by the remote cockpit and performs vehicle control operations according to the operation instruction.

[0021] Further, the test vehicle includes an on-board driving system, an on-board remote control system, and a wire-controlled chassis controller, wherein the on-board remote control system is communicatively connected to the on-board driving system and the wire-controlled chassis controller respectively; the remote cockpit includes a cockpit industrial computer, a driving simulator, and a display device, wherein the cockpit industrial computer is communicatively connected to the on-board remote control system, the driving simulator, and the display device in the test vehicle respectively; The S1 step specifically includes: The vehicle-mounted driving system sends a control request to the vehicle-mounted remote control system; The vehicle remote control system receives the control request sent by the vehicle driving system; The vehicle-mounted remote control system sends the prompt information to the remote cockpit according to the control request; The on-board remote control system acquires in real time video information captured by an external field of view camera installed on the test vehicle, the video information being used to display the real-time situation of the test vehicle in all directions, front, back, left, and right, and transmits the video information to the remote cockpit; The S2 step specifically includes: The cockpit industrial computer receives the prompt information sent by the vehicle-mounted remote control system in the test vehicle, and sends the prompt information to the display device; The display device receives the prompt information and displays the prompt information; The cockpit industrial computer receives the video information and sends the video information to the display device; The display device receives the video information and displays the video information; The S3 step specifically includes: The driving simulator responds to the vehicle control operation of the operating user and converts the vehicle control operation into the operating instruction; The driving simulator sends the operation instruction to the cockpit industrial computer; The cockpit industrial computer receives the operation instruction sent by the driving simulator, and sends the operation instruction to the on-board remote driving control system in the test vehicle; The S4 step specifically includes: The vehicle-mounted remote control system receives the operation instruction sent by the remote cockpit and sends the operation instruction to the wire-controlled chassis controller; The drive-by-wire chassis controller receives the operation instruction sent by the on-vehicle remote controller, and executes the vehicle control operation of the test vehicle according to the operation instruction.

[0022] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects: The unmanned driving system for vehicle testing provided in this specification includes a test vehicle and a remote cockpit. During the unmanned driving test of the vehicle, when the test vehicle has an unmanned driving function failure or enters an extreme operating scenario, the test vehicle sends a prompt message to the remote cockpit, and the prompt message is used to prompt the operating user of the remote cockpit to remotely take over the test vehicle. The video information captured by the external field of view shooting device installed on the test vehicle is sent to the remote cockpit. The remote cockpit is used to receive the prompt message sent by the test vehicle and display the prompt message on the display device. Receive video information and display the video information on the display device. In response to the vehicle control operation of the operating user on the driving simulator in the remote cockpit, an operation instruction is determined and the operation instruction is sent to the test vehicle. The test vehicle receives the operation instruction sent by the remote cockpit. According to the operation instruction, the vehicle control operation of the test vehicle is executed.

[0023] By integrating a remote cockpit with the test vehicle, remote control of the test vehicle can be achieved at any time during autonomous driving testing, ensuring safe operation even if the autonomous driving function fails or enters extreme operating conditions, thereby reducing safety hazards. Cameras installed on the test vehicle provide the remote cockpit with data analysis and external field of view video data for decision-making, establishing a complete perception-decision-execution closed loop, effectively preventing unexpected dangerous situations encountered by the test vehicle during testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The exemplary embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation of this specification. In the drawings: Figure 1A schematic diagram of the structure of an unmanned driving system for vehicle testing provided in an embodiment of this specification; Figure 2 This is a schematic diagram of the structure of an unmanned driving system provided in this manual. Figure 2 As shown; Figure 3 This is a flow chart of an unmanned driving method for vehicle testing provided in this specification being applied to a test vehicle in an unmanned driving system. DETAILED DESCRIPTION

[0025] To make the purpose, technical solutions, and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in conjunction with the specific embodiments of this specification and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0026] The technical solutions provided by the embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0027] Currently, autonomous or unmanned driving functions in vehicles can become unreliable when the vehicle encounters an emergency. If the autonomous driving function is implemented by integrating actuators such as steering robots, brake robots, and throttle robots to create an autonomous driving system, this system lacks awareness of sudden and dangerous situations. Furthermore, the fixtures used to mount the actuators can inevitably loosen due to vehicle vibrations, resulting in reduced robot accuracy and decreased vehicle tracking performance, potentially posing a risk. If the autonomous driving system is an assisted driving system installed in the vehicle to provide autonomous driving assistance to the driver, when the vehicle is driving on reliable roads (such as bad roads), the assisted driving system will gradually exit autonomous mode as road conditions deteriorate or an emergency occurs, requiring the driver to take over. If a vehicle encounters a bad road or an emergency during autonomous driving testing while unmanned, the autonomous driving test will be interrupted. Even if the emergency is avoided, the autonomous driving test will remain interrupted.

[0028] It can be seen that the current unmanned driving process of vehicles has the above-mentioned problems. Based on this, this specification provides an unmanned driving system for vehicle testing to partially solve the existing problems.

[0029] Figure 1This is a schematic diagram of the structure of an unmanned driving system for vehicle testing, provided in an embodiment of this specification. The unmanned driving system includes a test vehicle in a test site and a remote cockpit. The test vehicle and the remote cockpit can communicate via wireless communication. During the unmanned driving test, if the test vehicle experiences an unmanned driving function failure on a bad road or enters an extreme operating scenario, the test vehicle will send a prompt to the remote cockpit, prompting the operator in the remote cockpit to remotely take over the test vehicle. The unmanned driving function failure in the test vehicle may be caused by a loose or malfunctioning actuator that performs a vehicle function, making it unable to perform the corresponding vehicle function. The extreme operating scenario encountered by the test vehicle may be a steep, bad road or a road with multiple obstacles that need to be avoided. In such cases, neither an actuator without the ability to perceive the external environment nor an autonomous driving system that automatically exits assisted driving mode can effectively cope with it. Of course, this specification does not limit the unmanned driving function failures and extreme operating scenarios encountered by the test vehicle; the above description is merely an example.

[0030] After the test vehicle sends the prompt information to the remote cockpit, the remote cockpit receives the prompt information through wireless communication with the test vehicle, and displays the prompt information on the display device connected to the remote cockpit for the operator who operates the remote cockpit to watch.

[0031] Of course, in order to facilitate the operating user to take over the test vehicle through the remote cockpit, the test vehicle will use the external field of view shooting equipment installed on the test vehicle to shoot video information outside the test vehicle and send the video information to the remote cockpit.

[0032] The remote cockpit also receives video information through wireless communication with the test vehicle and displays it on a display device. The operator can then control the vehicle through the driving simulator in the remote cockpit. The remote cockpit responds to the operator's vehicle control operations on the driving simulator, determines the corresponding operational instructions for the vehicle control operations, and transmits these instructions to the test vehicle.

[0033] The test vehicle receives the operation instructions sent by the remote cockpit, and then performs vehicle control operations of the test vehicle according to the operation instructions.

[0034] based on Figure 1The unmanned driving system for vehicle testing shown here, consisting of a remote cockpit and a test vehicle, enables remote control of the test vehicle at any time during the test. This ensures safe operation even if the unmanned driving function fails or enters extreme operating scenarios, mitigating safety hazards. Cameras installed on the test vehicle provide the remote cockpit with data analysis and external field of view video data for decision-making, establishing a complete perception-decision-execution closed loop and effectively preventing unexpected dangerous situations encountered by the test vehicle during testing.

[0035] In addition, in one or more embodiments of this specification, the test vehicle includes an onboard driving system and an onboard remote control system, and the onboard driving system and the onboard remote control system are communicatively connected. The onboard driving system is used to detect whether the test vehicle's unmanned driving function has failed or whether the test vehicle has entered an extreme operating condition. If so, it indicates that the remote cockpit needs to take over the test vehicle and sends a control request to the onboard remote control system. This control request may indicate a request for the remote cockpit to take over control. If not, the onboard driving system continues to perform the test vehicle's unmanned driving task. The onboard remote control system is used to receive the control request sent by the onboard driving system and obtain real-time video information captured by an external field of view camera installed on the test vehicle. This video information is used to display the real-time situation of the test vehicle in all directions, front, back, left, and right. The external field of view camera may be a device such as an 8-channel GMSL camera. Based on the control request, the onboard remote control system sends a prompt message to the remote cockpit and transmits the video information to the remote cockpit. To meet the computing performance requirements of the test vehicle, a high-computing domain control device may also be installed on the test vehicle to provide computing power support for the onboard remote control system.

[0036] Among them, the on-board driving system can be a control system that controls actuators such as steering robots, braking robots, and throttle robots. For example, if a robot fails to perform the corresponding car function, the control system provides normal execution feedback, and it can be determined that the robot has a fault, that is, the unmanned driving system of the test vehicle has a fault. Alternatively, an assisted driving system can provide autonomous driving assistance to the driver. Test vehicles can be equipped with a combined inertial positioning and navigation module, such as a real-time kinematic (RTK) and inertial measurement unit (IMU) system. (RTK uses carrier phase differential technology to achieve centimeter-level positioning in open areas. When integrated with the IMU, the system can still achieve sub-meter-level positioning even with poor RTK signals.) Hardware such as cameras, lidar, millimeter-wave radar, a UPS (uninterruptible power supply), and a main controller are also included. This system uses combined inertial positioning for tracking and employs multi-sensor fusion to detect obstacles (pedestrians, cyclists, motor vehicles, tires, cones, etc.) to reduce false and missed detections. Control commands from the main controller are received via a bus to implement throttle, braking, and steering functions, enabling the test vehicle to achieve autonomous (assisted driving) capabilities. When the onboard driving system utilizes assisted driving capabilities, if sensors such as radar or cameras detect a bad road or obstacles that need to be avoided, this represents an extreme operating condition. In this case, assisted driving automatically disengages, typically requiring a human operator to take over.

[0037] The on-board driving system can detect in real time whether the unmanned driving function has malfunctioned or whether the vehicle has entered an extreme operating condition. Once a problem is detected, it will immediately request to switch to remote control mode. This method ensures that safety hazards can be effectively reduced even when the unmanned driving function cannot guarantee safety. The video information provided by the external field of view camera covers the real-time situation in all directions of the test vehicle, front, back, left, and right, providing the operator of the remote cockpit with a full range of perspectives, helping to more accurately judge the road conditions and environment, and thus make more reasonable driving decisions. The unmanned driving system allows for flexible switching of control modes during unmanned driving tests. Not only can tasks be automatically performed when the unmanned driving system is operating normally, but manual remote control can also be quickly switched to in complex or dangerous situations, ensuring the continuity and efficiency of the testing process.

[0038] In one or more embodiments of the present specification, the test vehicle further includes a wire-controlled chassis controller, and the on-board remote control system is communicatively connected to the wire-controlled chassis controller and the remote cockpit, respectively. Through the wire-controlled chassis controller, a variety of wire-controlled functions of the test vehicle can be realized, and each wire-controlled function corresponds to a vehicle control operation, and the vehicle control operations include at least throttle control operations on the test vehicle, brake control operations on the test vehicle, steering wheel control operations on the test vehicle, gear control operations on the test vehicle, and light control operations on the test vehicle.

[0039] At this time, the vehicle remote control system is used to receive the operation instructions sent by the remote cockpit, and then send the operation instructions to the wire-controlled chassis controller.

[0040] The wire-controlled chassis controller is used to receive the operation instruction sent by the vehicle-mounted remote controller and perform vehicle control operations of the test vehicle according to the operation instruction.

[0041] The by-wire chassis controller directly receives remote cockpit control commands forwarded from the onboard remote control system and executes specific vehicle control operations such as acceleration, deceleration, and steering based on these commands. This ensures accurate execution of control commands and improves the precision and responsiveness of test vehicle control. By transmitting control commands via electronic signals, by-wire technology replaces traditional mechanical or hydraulic connections, reducing physical wear and failure rates, thereby improving system reliability and safety. Especially in emergency situations, a fast and accurate response is crucial to avoiding accidents. The by-wire chassis controller makes the system more adaptable to different vehicle models and application scenarios. By simply adjusting the interface between the by-wire chassis controller and the specific vehicle model, it can be quickly deployed on a variety of test vehicles, enhancing the system's versatility and flexibility.

[0042] In one or more embodiments of the present specification, the remote cockpit includes a cockpit industrial computer, a driving simulator, and a display device, and the cockpit industrial computer is communicatively connected to the test vehicle, the driving simulator, and the display device, respectively.

[0043] The cockpit industrial computer receives prompts from the test vehicle and sends them to a display device. It also receives and sends video information to a display device. It also receives and sends operating instructions from the driving simulator to the test vehicle.

[0044] The display device is used to display the prompt information sent by the cockpit industrial computer after receiving the prompt information, and to display the video information after receiving the video information.

[0045] The driving simulator responds to the user's vehicle control operations and converts them into operational commands. These commands are then sent to the cockpit industrial computer. Of course, the driving simulator can include a steering wheel simulator, accelerator pedal, brake pedal (or joystick and walking pedals).

[0046] The cockpit industrial computer, a core component, receives prompts and video information from the test vehicle and forwards this information to the appropriate output devices (such as displays). This design enables centralized information processing, ensuring efficient and timely data transmission, allowing operators to quickly obtain the information they need. The display device displays prompts and video information, providing an intuitive user interface. By displaying a real-time view outside the vehicle, operators can accurately assess the vehicle's surroundings and make more precise control decisions. Furthermore, the display of prompts helps operators quickly understand the test vehicle's status, improving their ability to respond to emergencies. The driving simulator responds to the operator's vehicle control operations, converting them into operational commands and sending them to the cockpit industrial computer. This design not only provides a near-realistic driving experience but also allows operators to perform complex driving tasks remotely. This approach ensures effective vehicle control even in the event of unmanned driving system failure or extreme circumstances.

[0047] In one or more embodiments of this specification, the display device can also be used to respond to the user's selection of a test vehicle by displaying video information corresponding to the selected test vehicle to the user. This supports simultaneous monitoring and management of multiple test vehicles. The user can select a specific test vehicle as needed and view real-time video information around it, thereby enabling parallel monitoring and intervention in multiple unmanned driving test tasks. Allowing users to quickly select a target test vehicle through a graphical interface or interactive method helps operators quickly locate the problem vehicle and take over control in a timely manner, especially in the event of a failure or extreme scenario. The remote cockpit can be centrally deployed and managed. A single industrial computer and a set of display devices can serve multiple test vehicles, reducing hardware costs and space usage. The video information of each test vehicle can be displayed independently or in split-screen. Combined with a user-initiated selection mechanism, this allows operators to more comprehensively understand the operating status and surrounding environment of different vehicles. This flexible video switching mechanism is particularly important in complex scenarios such as multi-vehicle platooning testing, vehicle-road collaboration testing, and swarm intelligence testing, making the remote cockpit a unified dispatch center.

[0048] In one or more embodiments of the present specification, the test vehicle is used to send vehicle status information to the remote cockpit in real time, and the vehicle status information at least includes execution feedback information of vehicle control operations and instrument panel information of the test vehicle.

[0049] The remote cockpit is used to receive vehicle status information and display the vehicle status information to the operating user through a display device.

[0050] Execution feedback from vehicle control operations provides real-time information to remote operators regarding whether their commands have been correctly executed, helping to immediately identify and correct potential operational errors or system failures. Furthermore, by displaying the test vehicle's instrument panel information (such as speed, fuel consumption, and battery life), operators can fully understand the vehicle's operating status and make more accurate judgments and decisions.

[0051] In one or more embodiments of the present specification, when the test vehicle resolves the unmanned driving function failure or leaves the extreme working condition, the test vehicle continues to perform the preset unmanned driving task.

[0052] Figure 2 This is a schematic diagram of the structure of an unmanned driving system provided in this manual. Figure 2As shown, the test vehicle includes an onboard driving system, an onboard remote control system, a drive-by-wire chassis controller, and an external field of view camera. The remote cockpit includes a cockpit industrial computer, a display device, and a driving simulator. Upon recognizing a failure in the unmanned driving system or an extreme operating scenario, the onboard driving system sends a control request to the onboard remote control system. The onboard remote control system then receives the video information captured by the external field of view camera and the control request, and sends instructions and video information to the cockpit industrial computer. The display device displays the instructions and video information. The driving simulator generates operating instructions based on the user's operations, sends them to the onboard remote control system via the cockpit industrial computer, and then forwards them to the drive-by-wire chassis controller for vehicle control operations. The vehicle's exterior field of view capture equipment uses GMSL cameras and other devices. These are 720p / 1080p high-definition digital cameras with IP68 dust and water resistance, a horizontal viewing angle of 180°, a vertical viewing angle of 140°, and support distortion correction. Test vehicles can be equipped with a binocular camera (primary view camera), a left view camera, a right view camera, and a rear view camera, providing video from four perspectives: front, left, and rear. The onboard remote control system captures the video captured by the GMSL cameras, encodes it, and transmits it to the remote cockpit. This video can then be transmitted back via a dedicated 5G network and displayed on a display device. The entire process has a latency of less than 150ms. The latency for transmitting the video from the other viewpoints to the ground-based large screen via the 5G network is less than 200ms. In poor network conditions, the video bitrate can be adjusted to ensure smooth video display. Bandwidth can be automatically adjusted according to the following policies: a) For good network conditions, the bandwidth for the primary viewpoint camera is set to 4M, and the bandwidth for the other viewpoint cameras is set to 2M. b) For average network conditions, the bandwidth for the primary viewpoint camera is set to 2M, and the bandwidth for the other viewpoint cameras is set to 1M. c) Poor network conditions: The bandwidth for the primary view camera is set to 1M, and the bandwidth for other view cameras is set to 512K. The remote cockpit supports multiple test vehicles. The operator can switch between the test vehicles at any time by selecting the vehicle number on the display device, but can only control one test vehicle at a time. The remote cockpit also supports permission management, requiring only users with designated user accounts to log in to the remote cockpit and take over remote operations. Test vehicle status information, video information, driving simulator operating instructions, network status, and other information are logged and stored on a server connected to the remote cockpit. Audio from the test vehicle's terminal is collected and played back in the remote cockpit via a remote communication protocol, simulating a real-world driving experience. Simultaneously, audio from the remote cockpit can be transmitted back to the test vehicle, enabling two-way real-time voice communication between the remote cockpit and the test vehicle. Furthermore, the display device can be multi-terminal, allowing simultaneous viewing of video information on tablets, monitors connected to the remote cockpit, and mobile phones. Figure 3This specification provides a flowchart of an unmanned driving method for vehicle testing, which uses the above-mentioned unmanned driving system for vehicle testing and includes the following steps: S1: During the vehicle unmanned driving test, when the test vehicle has an unmanned driving function failure or enters an extreme operating scenario, the test vehicle sends a prompt message to the remote cockpit, and the prompt message is used to prompt the operating user of the remote cockpit to remotely take over the test vehicle, and send the video information captured by the external view shooting device installed on the test vehicle to the remote cockpit.

[0053] In one or more embodiments of the present specification, during the unmanned driving test of a vehicle, when an unmanned driving function failure occurs or an extreme operating scenario occurs, the test vehicle can send a prompt message to the remote cockpit. The prompt message is used to prompt the operating user of the remote cockpit to remotely take over the test vehicle, and the test vehicle sends the video information captured by the outdoor view shooting equipment installed on the test vehicle to the remote cockpit, so that the operating user can determine the operating instructions of the test vehicle based on the video information.

[0054] S2: The remote cockpit receives the prompt information sent by the test vehicle and displays the prompt information on a display device, and receives video information captured by an external field of view camera on the test vehicle and displays the video information on the display device.

[0055] In one or more embodiments of the present specification, the remote cockpit receives prompt information sent by the test vehicle and displays the prompt information on a display device, and receives video information captured by an external field of view camera on the test vehicle and displays the video information on a display device.

[0056] S3: The remote cockpit determines an operation instruction corresponding to the vehicle control operation in response to the vehicle control operation of the operating user on the driving simulator, and sends the operation instruction to the test vehicle.

[0057] In one or more embodiments of the present specification, the remote cockpit determines an operation instruction corresponding to the vehicle control operation in response to the vehicle control operation of the operating user on the driving simulator, and sends the operation instruction to the test vehicle.

[0058] S4: The test vehicle receives the operation instruction sent by the remote cockpit and performs vehicle control operations according to the operation instruction.

[0059] In one or more embodiments of the present specification, the test vehicle receives an operation instruction sent by the remote cockpit and performs a vehicle control operation according to the operation instruction.

[0060] For the specific details of steps S1 to S4 in the above method, please refer to the previous description of the system and will not be described in detail here.

[0061] based on Figure 3 This manual describes an unmanned driving method for vehicle testing. By combining a remote cockpit with a test vehicle, the test vehicle can be remotely controlled at any time during the unmanned driving test, ensuring that the test vehicle can operate safely even if the unmanned driving function fails or enters extreme operating scenarios, thereby reducing safety hazards. By installing camera equipment on the test vehicle, the remote cockpit is provided with data analysis and video data of the vehicle's outside field of view for decision-making, thus establishing a complete perception-decision-execution closed loop. This can effectively avoid sudden dangerous situations encountered by the test vehicle during testing.

[0062] In one or more embodiments of this specification, reference is made to Figure 2 The test vehicle includes an onboard driving system, an onboard remote control system, and a drive-by-wire chassis controller. The onboard remote control system is connected to the onboard driving system and the drive-by-wire chassis controller. The remote cockpit includes a cockpit industrial computer, a driving simulator, and a display device. The cockpit industrial computer is connected to the onboard remote control system, driving simulator, and display device in the test vehicle.

[0063] The onboard driving system sends a control request to the onboard remote control system. The onboard remote control system receives the control request from the onboard driving system. Based on the control request, the onboard remote control system sends prompt information to the remote cockpit. The onboard remote control system captures real-time video information captured by the external field of view camera installed on the test vehicle. The video information is used to display the real-time situation in all directions of the test vehicle, including the front, back, left, and right directions. The video information is then sent to the remote cockpit. The cockpit industrial computer receives the prompt information sent by the onboard remote control system in the test vehicle and sends it to the display device. The display device receives the prompt information and displays it. The cockpit industrial computer receives the video information and sends it to the display device. The display device receives the video information and displays it. The driving simulator responds to the user's vehicle control operations and converts them into operational commands. The driving simulator sends the operational commands to the cockpit industrial computer. The cockpit industrial computer receives the operational commands from the driving simulator and sends them to the onboard remote driving control system in the test vehicle. The onboard remote control system receives the operational commands from the remote cockpit and sends them to the drive-by-wire chassis controller. The wire-controlled chassis controller receives operation instructions sent by the on-board remote controller and performs vehicle control operations of the test vehicle according to the operation instructions.

[0064] This specification also provides a computer-readable storage medium, which stores a computer program that can be used to execute the above Figure 3 An unmanned driving method for vehicle testing is provided.

[0065] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0066] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using physical hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly performed using software called a "logic compiler." This is similar to the software compilers used during program development. Before compilation, the original code must be written in a specific programming language, called a Hardware Description Language (HDL). There are many types of HDL, including ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that simply by programming a method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0067] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the memory control logic. Those skilled in the art will also appreciate that, in addition to implementing the controller purely in computer-readable program code, the controller can also be implemented in the form of logic gates, switches, an application-specific integrated circuit, a programmable logic controller, an embedded microcontroller, etc. by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the means for implementing the various functions included therein can also be considered as structures within the hardware component. Alternatively, the means for implementing the various functions can be considered both a software module implementing the method and a structure within the hardware component.

[0068] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0069] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0070] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Thus, this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0071] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0072] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0073] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0074] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0075] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0076] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0077] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0078] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0079] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0080] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0081] The foregoing is merely an example of the present invention and is not intended to limit the present invention. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. An unmanned driving system for vehicle testing, characterized in that: The system includes a test vehicle and a remote cockpit, wherein: The remote cockpit is communicatively connected to the test vehicle; The test vehicle is configured to: during a vehicle unmanned driving test, when the test vehicle experiences an unmanned driving function failure or enters an extreme operating condition, the test vehicle sends a prompt message to the remote cockpit, the prompt message being used to prompt an operator of the remote cockpit to remotely take over the test vehicle; send video information captured by an external view camera installed on the test vehicle to the remote cockpit; receive operating instructions sent by the remote cockpit, and execute vehicle control operations of the test vehicle according to the operating instructions; The remote cockpit is configured to: receive the prompt information sent by the test vehicle and display the prompt information on the display device; receive the video information and display the video information on the display device; in response to the vehicle control operation of the operating user on the driving simulator in the remote cockpit, determine the operation instruction corresponding to the vehicle control operation, and send the operation instruction to the test vehicle.

2. The unmanned driving system for vehicle testing according to claim 1, characterized in that: The test vehicle includes an on-board driving system and an on-board remote control system, wherein: The on-board driving system is configured to: detect whether the unmanned driving function of the test vehicle fails or whether the test vehicle enters an extreme operating condition; if so, send a control request to the on-board remote control system; if not, the on-board driving system continues to perform the unmanned driving task of the test vehicle; The on-board remote control system is configured to: receive the control request sent by the on-board driving system; acquire in real time the video information captured by the outside view camera installed on the test vehicle, wherein the video information is used to display the real-time situation of the test vehicle in all directions of the front, back, left and right; and send the prompt information to the remote cockpit according to the control request, and send the video information to the remote cockpit.

3. The unmanned driving system for vehicle testing according to claim 2, characterized in that: The test vehicle further includes a drive-by-wire chassis controller, and the onboard remote control system is communicatively connected to the drive-by-wire chassis controller and the remote cockpit, respectively, wherein: The vehicle-mounted remote control system is configured to: receive the operation instruction sent by the remote cockpit; and send the operation instruction to the wire-controlled chassis controller; The wire-controlled chassis controller is used to receive the operation instruction sent by the vehicle-mounted remote controller and perform the vehicle control operation of the test vehicle according to the operation instruction.

4. An unmanned driving system for vehicle testing according to claim 1 or 3, characterized in that: The vehicle control operation includes at least an accelerator control operation on the test vehicle, a brake control operation on the test vehicle, a steering wheel control operation on the test vehicle, a gear control operation on the test vehicle, and a light control operation on the test vehicle.

5. The unmanned driving system for vehicle testing according to claim 1, characterized in that: The remote cockpit includes a cockpit industrial computer, a driving simulator, and a display device. The cockpit industrial computer is communicatively connected to the test vehicle, the driving simulator, and the display device, respectively. The cockpit industrial computer is configured to: receive the prompt information sent by the test vehicle and send the prompt information to the display device; receive the video information and send the video information to the display device; receive the operation instruction sent by the driving simulator and send the operation instruction to the test vehicle; The display device is configured to: display the prompt information sent by the cockpit industrial computer after receiving the prompt information, and display the video information after receiving the video information; The driving simulator is configured to: respond to the vehicle control operation of the operating user and convert the vehicle control operation into the operation instruction; and send the operation instruction to the cockpit industrial computer.

6. The unmanned driving system for vehicle testing according to claim 5, characterized in that: The display device is further configured to: in response to the user's selection operation of selecting a test vehicle, display video information corresponding to the test vehicle selected by the selection operation to the user.

7. The unmanned driving system for vehicle testing according to claim 1, characterized in that: The test vehicle is further configured to: send vehicle status information to the remote cockpit in real time, the vehicle status information including at least execution feedback information of the vehicle control operation and instrument panel information of the test vehicle; The remote cockpit is further configured to receive the vehicle status information and display the vehicle status information to the operating user via the display device.

8. The unmanned driving system for vehicle testing according to claim 1, characterized in that: The test vehicle is further configured to continue to perform a preset unmanned driving task when the test vehicle resolves an unmanned driving function failure or leaves an extreme operating scenario.

9. An unmanned driving method for vehicle testing, characterized in that: The unmanned driving system for vehicle testing according to claim 1 comprises the following steps: S1: During the unmanned driving test of a vehicle, if the unmanned driving function of the test vehicle fails or enters an extreme operating condition, the test vehicle sends a prompt message to the remote cockpit, whereby the operator of the remote cockpit is prompted to remotely take over the test vehicle and to send video information captured by an external view camera installed on the test vehicle to the remote cockpit; S2: the remote cockpit receives the prompt information sent by the test vehicle and displays the prompt information on a display device, and receives video information captured by an external view camera on the test vehicle and displays the video information on the display device; S3: The remote cockpit determines an operation instruction corresponding to the vehicle control operation in response to the vehicle control operation of the operating user on the driving simulator, and sends the operation instruction to the test vehicle; S4: The test vehicle receives the operation instruction sent by the remote cockpit and performs vehicle control operations according to the operation instruction.

10. The unmanned driving method for vehicle testing according to claim 9, characterized in that: The test vehicle includes an on-board driving system, an on-board remote control system, and a wire-controlled chassis controller, wherein the on-board remote control system is communicatively connected to the on-board driving system and the wire-controlled chassis controller respectively; the remote cockpit includes a cockpit industrial computer, a driving simulator, and a display device, wherein the cockpit industrial computer is communicatively connected to the on-board remote control system, the driving simulator, and the display device in the test vehicle respectively; The S1 step specifically includes: The vehicle-mounted driving system sends a control request to the vehicle-mounted remote control system; The vehicle remote control system receives the control request sent by the vehicle driving system; The vehicle-mounted remote control system sends the prompt information to the remote cockpit according to the control request; The on-board remote control system acquires in real time video information captured by an external field of view camera installed on the test vehicle, the video information being used to display the real-time situation of the test vehicle in all directions, front, back, left, and right, and transmits the video information to the remote cockpit; The S2 step specifically includes: The cockpit industrial computer receives the prompt information sent by the vehicle-mounted remote control system in the test vehicle, and sends the prompt information to the display device; The display device receives the prompt information and displays the prompt information; The cockpit industrial computer receives the video information and sends the video information to the display device; The display device receives the video information and displays the video information; The S3 step specifically includes: The driving simulator responds to the vehicle control operation of the operating user and converts the vehicle control operation into the operating instruction; The driving simulator sends the operation instruction to the cockpit industrial computer; The cockpit industrial computer receives the operation instruction sent by the driving simulator, and sends the operation instruction to the on-board remote driving control system in the test vehicle; The S4 step specifically includes: The vehicle-mounted remote control system receives the operation instruction sent by the remote cockpit and sends the operation instruction to the wire-controlled chassis controller; The drive-by-wire chassis controller receives the operation instruction sent by the on-vehicle remote controller, and executes the vehicle control operation of the test vehicle according to the operation instruction.

Images