Remote assistance method, device and equipment for autonomous vehicle and storage medium

Abstract

The application discloses a remote assistance method, device and equipment of an automatic driving vehicle and a storage medium, relates to the technical field of automatic driving, and discloses a remote assistance method of an automatic driving vehicle, which comprises the following steps: acquiring a remote assistance request generated by an automatic driving vehicle in an abnormal running scene, collecting vehicle running assistance data according to the remote assistance request, and obtaining assistance scene data; generating a remote scene according to the assistance scene data, and generating a structured remote assistance instruction according to the remote scene; performing vehicle-end safety verification according to the structured remote assistance instruction and the vehicle running assistance data, and obtaining an instruction verification result; and when the instruction verification result is verified, performing remote assistance of the automatic driving vehicle according to the structured remote assistance instruction. The technical means of the application can improve the safety, compliance and traffic continuity of remote assistance of the automatic driving vehicle in the abnormal running scene.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to remote assistance methods, devices, equipment and storage media for autonomous vehicles. Background Technology

[0002] With the continuous development of autonomous driving technology in the commercial vehicle sector, Level 4 autonomous light trucks are gradually being applied to closed or semi-closed operating scenarios such as ports, industrial parks, and factories. In these scenarios, vehicles typically need to complete tasks such as logistics transportation, material transfer, and park delivery according to preset routes, relying on onboard perception, positioning, decision-making, and control capabilities to achieve a high degree of autonomous operation. However, the actual operating environment is not always stable, regular, and predictable. When vehicles encounter temporary obstacles, road construction, extreme road conditions, changes in road boundaries, inconsistencies between maps and the actual situation, or situations exceeding the designed operating range, the autonomous driving system may struggle to directly generate continuous driving decisions that meet safety requirements. The vehicle may then easily enter a state of deceleration, stopping, waiting, or requesting remote assistance, thereby affecting the operational continuity and efficiency of the autonomous vehicle.

[0003] Based on the aforementioned technological background, traditional remote assistance methods for autonomous driving mainly fall into two categories: remote takeover control and complete reliance on vehicle-side autonomous decision-making. Remote takeover control typically involves a remote operator directly gaining control of the vehicle and manipulating its direction, acceleration, and braking in real time. This method places high demands on communication link stability, latency control, and the handover process; network delays, command lags, or handover anomalies can compromise vehicle safety. Complete reliance on vehicle-side autonomous decision-making relies primarily on the onboard autonomous driving system to independently assess and handle abnormal scenarios. However, for unstructured, unconventional, or complex scenarios with significant temporary changes, the onboard system's adaptability is limited, easily leading to prolonged processing times, failed traffic decisions, or operational interruptions. Summary of the Invention

[0004] The main purpose of this application is to provide a method, device, equipment and storage medium for remote assistance of autonomous vehicles, which aims to solve the technical problem that existing remote assistance methods for autonomous vehicles are unable to simultaneously meet the requirements of compliant management of vehicle control and handling of complex scenarios under abnormal operating conditions, resulting in insufficient adaptability of remote assistance for autonomous light trucks.

[0005] To achieve the above objectives, this application proposes a remote assistance method for autonomous vehicles, the method comprising: The system acquires remote assistance requests generated by autonomous vehicles in abnormal operating scenarios, and collects vehicle operation assistance data based on the remote assistance requests to obtain assistance scenario data. A remote scenario is generated based on the assistance scenario data, and a structured remote assistance instruction is generated based on the remote scenario; Vehicle-side security verification is performed based on the structured remote assistance command and the vehicle operation assistance data to obtain the command verification result; When the instruction verification result is successful, remote assistance for the autonomous vehicle is performed according to the structured remote assistance instruction.

[0006] In one embodiment, the step of obtaining remote assistance requests generated by the autonomous vehicle under abnormal operating scenarios, and collecting vehicle operation assistance data based on the remote assistance requests to obtain assistance scenario data includes: Obtain the vehicle traffic status data generated by the autonomous vehicle in the autonomous driving activated state; Based on the vehicle traffic status data, determine whether the autonomous vehicle is in an abnormal operating scenario, and obtain the abnormal operating scenario determination result; When the abnormal operation scenario determination result indicates that the system is in an abnormal operation scenario, a remote assistance request is generated. According to the remote assistance request, vehicle pose data, surrounding environment data, obstacle data, road topology data, vehicle status data and map data are collected, and vehicle operation assistance data is generated based on the vehicle pose data, surrounding environment data, obstacle data, road topology data, vehicle status data and map data. Based on the vehicle operation assistance data, data synchronization and scene association are performed to obtain assistance scene data.

[0007] In one embodiment, the steps of generating a remote scene based on the assistance scene data and generating structured remote assistance instructions based on the remote scene include: Based on the vehicle pose data, surrounding environment data, obstacle data, road topology data, vehicle status data, and map data in the assisted scene data, scene fusion is performed to obtain remote scene basic data; The remote scene is obtained by reconstructing the on-site environment based on the aforementioned remote scene basic data; The remote assistance operation data is received according to the remote scenario, and the assistance action type, assistance action distance, assistance action angle, assistance action speed and assistance action priority are determined according to the remote assistance operation data. The instruction fields are encapsulated according to the type of assistance action, the distance of the assistance action, the angle of the assistance action, the speed of the assistance action, and the priority of the assistance action to obtain a structured remote assistance instruction.

[0008] In one embodiment, the step of performing vehicle-side security verification based on the structured remote assistance command and the vehicle operation assistance data to obtain the command verification result includes: The structured remote assistance command is parsed to obtain the action type parameter, action distance parameter, action angle parameter, action speed parameter, and command priority parameter; The legality of the command is verified by the action type parameter, the action distance parameter, the action angle parameter, and the action speed parameter according to the preset action constraint range, and the result of the command legality verification is obtained. Environmental risk verification is performed based on the vehicle operation assistance data and the structured remote assistance instructions to obtain the environmental risk verification result. Based on the executing entity corresponding to the structured remote assistance instruction, a dynamic driving task control right verification is performed to obtain the control right verification result; Based on the legality verification result of the instruction, the environmental risk verification result, and the control verification result, an instruction release information or an instruction rejection information is generated, and the instruction release information or the instruction rejection information is determined as the instruction verification result.

[0009] In one embodiment, the step of performing environmental risk verification based on the vehicle operation assistance data and the structured remote assistance command to obtain the environmental risk verification result includes: The current vehicle pose, current obstacle distribution, current road boundary, and current road topology of the autonomous vehicle are determined based on the vehicle operation assistance data. Predict the instruction execution path of the autonomous vehicle based on the current vehicle pose and the structured remote assistance instructions; Based on the current obstacle distribution, the command execution path is subjected to collision risk verification to obtain the collision risk verification result; Based on the current road boundary and the current road topology, the instruction execution path is checked for road boundary to obtain the road boundary check result. Based on the collision risk verification result and the road boundary verification result, environmental clearance information or environmental rejection information is generated, and the environmental clearance information or the environmental rejection information is determined as the environmental risk verification result.

[0010] In one embodiment, the step of remotely assisting the autonomous vehicle according to the structured remote assistance instruction when the instruction verification result is a pass includes: When the instruction verification result is successful, local path planning is performed based on the structured remote assistance instruction and the assistance scenario data to obtain the remote assistance execution path; Drive control parameters, braking control parameters, and steering control parameters are generated based on the remote assistance execution path; The autonomous vehicle is controlled to perform remote assistance actions based on the drive control parameters, the braking control parameters, and the steering control parameters, and the real-time status of the vehicle performing remote assistance actions and the environmental change status are collected. Based on the real-time status of the vehicle and the changing environmental conditions, the remote assistance action is monitored for safety, and the safety monitoring results are obtained. When the safety monitoring results are passed, remote assistance for the autonomous vehicle is provided according to the structured remote assistance instructions.

[0011] In one embodiment, after the step of remotely assisting the autonomous vehicle according to the structured remote assistance instruction when the instruction verification result is successful, the method further includes: The system acquires the real-time status of the autonomous vehicle, changes in the environment, and the progress of the assistance process during the remote assistance process. Based on the real-time status of the vehicle, the changing status of the environment, and the progress of the assisted execution, determine whether the autonomous vehicle meets the conditions for autonomous driving recovery, and obtain a recovery determination result; When the recovery determination result is that the autonomous driving recovery conditions are met, the autonomous vehicle is repositioned and the map is matched to obtain vehicle recovery positioning data; The autonomous driving operation mode of the autonomous vehicle is restored based on the vehicle's restored positioning data, thus obtaining the autonomous driving recovery status; Remote assistance feedback data is generated based on the vehicle's real-time status, the progress of the assistance execution, and the autonomous driving recovery status to provide remote assistance feedback.

[0012] In addition, to achieve the above objectives, this application also proposes a remote assistance device for autonomous vehicles, which includes: a data acquisition module, used to acquire remote assistance requests generated by the autonomous vehicle in abnormal operating scenarios, and to acquire vehicle operation assistance data according to the remote assistance requests to obtain assistance scenario data. The instruction generation module is used to generate a remote scenario based on the assistance scenario data, and to generate structured remote assistance instructions based on the remote scenario. The security verification module is used to perform vehicle-side security verification based on the structured remote assistance command and the vehicle operation assistance data, and obtain the command verification result. The remote assistance module is used to provide remote assistance to the autonomous vehicle according to the structured remote assistance instruction when the instruction verification result is successful.

[0013] In addition, to achieve the above objectives, this application also proposes a remote assistance device for an autonomous vehicle, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the remote assistance method for an autonomous vehicle as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps of the remote assistance method for autonomous vehicles as described above.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the remote assistance method for autonomous vehicles as described above.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: By acquiring remote assistance requests when autonomous vehicles enter abnormal operating scenarios and collecting vehicle operation assistance data based on these requests to form assistance scenario data, the remote end can generate remote scenarios based on the actual vehicle operating status and surrounding environment. Structured remote assistance instructions are then generated based on these scenarios. The vehicle end then performs security verification by combining the structured remote assistance instructions and vehicle operation assistance data. Remote assistance from the autonomous vehicle is only implemented based on the structured remote assistance instructions if the verification result is successful. This chain connects abnormal scenario identification, scenario data acquisition, remote instruction generation, vehicle-side verification, and assistance execution into a complete processing chain. This solution addresses the problems of control handover risks, insufficient command execution security, and inadequate adaptability to complex scenarios when existing autonomous vehicles rely solely on remote takeover or on-board autonomous judgment in abnormal operating scenarios. Compared to existing technologies, this solution does not directly have the remote end take over the vehicle to execute real-time control. Instead, after generating structured assistance commands at the remote end, the on-board end performs safety verification based on vehicle operation assistance data and executes remote assistance. This allows the remote assistance process to take into account both remote human judgment capabilities and on-board safe execution capabilities, thereby improving the safety, compliance, and operational continuity of remote assistance for autonomous vehicles in abnormal operating scenarios. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating an embodiment of the remote assistance method for autonomous vehicles in this application; Figure 2 A flowchart illustrating a second embodiment of the remote assistance method for autonomous vehicles in this application; Figure 3 This is a schematic diagram of the module structure of the remote assistance device for an autonomous vehicle according to an embodiment of this application; Figure 4 This is a schematic diagram of the device structure of the hardware operating environment involved in the remote assistance method for autonomous vehicles in the embodiments of this application.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] The main solution of this application embodiment is as follows: obtain remote assistance requests generated by autonomous vehicles in abnormal operating scenarios, and collect vehicle operation assistance data according to the remote assistance requests to obtain assistance scenario data; generate remote scenarios according to the assistance scenario data, and generate structured remote assistance instructions according to the remote scenarios; perform vehicle-side security verification according to the structured remote assistance instructions and the vehicle operation assistance data to obtain instruction verification results; when the instruction verification result is a successful verification, perform remote assistance of autonomous vehicles according to the structured remote assistance instructions.

[0024] In this embodiment, for ease of description, the following description will focus on a remote assistance device that identifies autonomous vehicles.

[0025] Because existing technologies and methods for remote assistance in autonomous driving struggle to balance compliance management of vehicle control with the handling of complex scenarios in abnormal operating conditions, the adaptability of remote assistance for autonomous light trucks is insufficient. This application provides a solution that obtains remote assistance requests when the autonomous vehicle enters an abnormal operating scenario, and collects vehicle operation assistance data based on these requests to form assistance scenario data. This allows the remote end to generate a remote scenario based on the actual operating state of the vehicle and its surrounding environment, and then generates structured remote assistance instructions based on the remote scenario. The vehicle end then performs a security verification by combining the structured remote assistance instructions and the vehicle operation assistance data, and only implements remote assistance for the autonomous vehicle based on the structured remote assistance instructions if the verification result is successful. This solution integrates abnormal scene identification, scene data acquisition, remote command generation, vehicle-side verification, and assistance execution into a complete processing chain. It addresses the issues that existing autonomous vehicles, when relying solely on remote takeover or on-board autonomous judgment in abnormal operating scenarios, are prone to risks in control transfer, insufficient command execution security, and inadequate adaptability to complex scenarios. Compared to existing technologies, this solution does not directly have the remote end take over the vehicle to execute real-time control. Instead, after generating structured assistance commands on the remote end, the vehicle-side performs safety verification based on vehicle operation assistance data and executes remote assistance. This allows the remote assistance process to balance remote human judgment capabilities with on-board safe execution capabilities, thereby improving the safety, compliance, and operational continuity of remote assistance for autonomous vehicles in abnormal operating scenarios.

[0026] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions, a remote assistance device for autonomous vehicles, etc. The following description uses a remote assistance device for autonomous vehicles as an example to illustrate this embodiment and the subsequent embodiments.

[0027] Based on this, embodiments of this application provide a remote assistance method for autonomous vehicles, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the remote assistance method for autonomous vehicles according to this application.

[0028] In this embodiment, the remote assistance method for the autonomous vehicle includes steps S10 to S40: Step S10: Obtain remote assistance requests generated by autonomous vehicles in abnormal operating scenarios, and collect vehicle operation assistance data based on the remote assistance requests to obtain assistance scenario data.

[0029] It should be noted that the data collection, tag management, rule setting, and push decision-making processes involved in this application are designed to work with other technical features to solve technical problems. They do not involve or support any illegal activities. Any data processing that may violate laws and regulations (such as unauthorized collection of privacy data, generation of discriminatory tags, setting unfair rules, or pushing illegal information) is not within the scope of protection of this application's technical solution. Of course, the user data in this application will be encrypted, anonymized, or de-identified before storage to ensure user data security.

[0030] It should be understood that an autonomous vehicle is a vehicle with autonomous driving capabilities. In this embodiment, it is an L4-level autonomous driving light truck, capable of autonomously completing all dynamic driving tasks in closed or semi-closed environments. L4-level autonomous driving refers to the fourth level in the classification of driving automation, namely highly automated driving. Within its designed operating range, the vehicle can autonomously perform all driving operations without human driver intervention.

[0031] Another type of abnormal operating scenario refers to situations encountered by autonomous vehicles during operation that exceed their autonomous decision-making and control capabilities, making it impossible for them to continue driving normally. These scenarios include exceeding the designed operating range, encountering temporary obstacles, road construction, and extreme road conditions, which can lead to the interruption of the vehicle's autonomous passage.

[0032] Additionally, a remote assistance request is a signal sent by an autonomous vehicle to a remote monitoring platform when it detects an abnormal operating scenario and is unable to complete its passage task autonomously. The remote assistance request notifies the remote monitoring platform that remote assistance support is needed, triggering subsequent remote assistance procedures.

[0033] Additionally, vehicle operation assistance data refers to various data related to the vehicle's operating status and surrounding environment that are collected simultaneously when an autonomous vehicle generates a remote assistance request. This data forms the basis for the remote monitoring platform to understand the on-site situation and generate effective assistance commands.

[0034] Additionally, the assistance scenario data is a dataset obtained by organizing and standardizing the collected vehicle operation assistance data, which can completely reconstruct the on-site situation of the vehicle. The assistance scenario data includes vehicle pose information, surrounding environment information, obstacle information, road topology information, and vehicle status information.

[0035] Understandably, when an autonomous vehicle detects that it is in an abnormal operating scenario and cannot complete its passage task autonomously, it will automatically generate a remote assistance request. The autonomous vehicle sends the remote assistance request to the remote monitoring platform and simultaneously initiates the vehicle operation assistance data collection process. The autonomous vehicle collects vehicle operation assistance data through various onboard sensors, performs preliminary processing of the collected data, and sends it along with the remote assistance request to the remote monitoring platform to obtain the assistance scenario data.

[0036] In one feasible implementation, step S10 may include steps S11 to S15: Step S11: Obtain vehicle traffic status data generated by the autonomous vehicle in the autonomous driving activated state.

[0037] It should be noted that the autonomous driving activation state refers to the state in which the autonomous driving function of the autonomous vehicle is activated and operating normally. At this time, the autonomous driving decision-making unit executes all dynamic driving tasks. The vehicle does not require any driving operation from a human driver and completes the driving task entirely by its own perception and decision-making system.

[0038] In addition, vehicle traffic status data is generated in real time during the operation of autonomous vehicles, reflecting the vehicle's current driving status and traffic capability. Vehicle traffic status data includes information such as vehicle speed, driving path, obstacle distance, and road access permissions, serving as the basis for determining whether a vehicle can pass normally.

[0039] Understandably, autonomous vehicles continuously generate traffic status data when in autonomous driving mode. This data is then transmitted in real-time to the autonomous driving decision-making unit for monitoring vehicle operating status and assessing traffic capability.

[0040] Step S12: Determine whether the autonomous vehicle is in an abnormal operating scenario based on the vehicle traffic status data, and obtain the abnormal operating scenario determination result.

[0041] It should be noted that the abnormal operating scenario determination result is obtained by the autonomous driving decision-making unit based on the analysis and judgment of vehicle traffic status data. It is used to indicate whether the vehicle is in an abnormal operating scenario that prevents it from autonomously passing through. The abnormal operating scenario determination result is divided into two cases: being in an abnormal operating scenario and not being in an abnormal operating scenario. Different results correspond to different subsequent processing procedures.

[0042] Understandably, the autonomous driving decision unit receives real-time vehicle traffic status data. The unit compares this data with preset normal traffic thresholds to determine whether the vehicle can continue autonomously. If the vehicle cannot continue autonomously, it is considered to be in an abnormal operating scenario; if it can continue autonomously, it is considered not to be in an abnormal operating scenario, and a corresponding abnormal operating scenario determination result is generated.

[0043] Step S13: When the abnormal operation scenario determination result is that the abnormal operation scenario is in an abnormal operation scenario, a remote assistance request is generated.

[0044] Understandably, when the autonomous driving decision-making unit determines that an abnormal operating scenario has occurred, it will automatically generate a remote assistance request. This request includes basic information such as the vehicle's unique identifier, the time of the anomaly, and the type of anomaly, and is used to notify the remote monitoring platform that the vehicle requires remote assistance.

[0045] Step S14: Collect vehicle pose data, surrounding environment data, obstacle data, road topology data, vehicle status data, and map data according to the remote assistance request, and generate vehicle operation assistance data based on the vehicle pose data, surrounding environment data, obstacle data, road topology data, vehicle status data, and map data.

[0046] It should be noted that vehicle pose data describes the position and attitude of an autonomous vehicle in space, including information such as the vehicle's longitude, latitude, altitude, heading angle, and pitch angle. Vehicle pose data can accurately determine the vehicle's specific location and direction of travel on a map, providing a foundation for subsequent scene reconstruction and path planning.

[0047] Additionally, ambient environmental data is collected by autonomous vehicles through onboard sensors, reflecting the environmental conditions within a certain range around the vehicle. This data includes road conditions, lighting conditions, weather conditions, and the distribution of static objects in the vicinity, helping remote operators to fully understand the external environmental conditions surrounding the vehicle.

[0048] Additionally, obstacle data refers to the data detected by autonomous vehicles using multi-source sensors, which includes information about objects around the vehicle that may affect its passage. Obstacle data includes the obstacle's location, size, shape, motion status, and distance from the vehicle, and is crucial for assessing vehicle safety and planning escape routes.

[0049] Additionally, road topology data describes the structure and connectivity of road networks, including road orientation, width, number of lanes, intersection locations, and traffic signs and markings. Road topology data clearly presents the structural characteristics of the road a vehicle is on, supporting remote operators in determining feasible travel routes.

[0050] Additionally, vehicle status data reflects the operational status of the autonomous vehicle itself, including information such as battery level, speed, steering angle, braking status, and engine speed. Vehicle status data helps remote operators understand the vehicle's operational capabilities and ensures that generated commands comply with the vehicle's performance limitations.

[0051] In addition, the map data is pre-stored on the vehicle and consists of high-precision maps containing all road and geographic information within the vehicle's operating area. This map data, combined with real-time sensor data, enables precise vehicle positioning and environmental awareness.

[0052] Understandably, when the autonomous driving decision-making unit generates a remote assistance request, it triggers a multi-source data acquisition process. The autonomous driving decision-making unit controls LiDAR, millimeter-wave radar, cameras, GPS, inertial measurement unit, and vehicle sensors to collect various types of data. Simultaneously, the autonomous driving decision-making unit calls upon pre-stored map data, integrating all the collected data and the called map data to generate vehicle operation assistance data.

[0053] Step S15: Perform data synchronization and scene association based on the vehicle operation assistance data to obtain assistance scene data.

[0054] It should be noted that data synchronization refers to the process of unifying various types of data collected by different sensors at different times under the same time reference. Data synchronization can ensure the time consistency of all data and avoid scene reconstruction errors and command generation mistakes caused by time deviations.

[0055] Additionally, scene association refers to the process of matching and associating different types of data according to spatial location and logical relationships. Scene association can bind vehicle pose data, surrounding environment data, obstacle data, etc., with corresponding map locations and road structures to form complete and unified scene information.

[0056] Understandably, after acquiring the generated vehicle operation assistance data, the autonomous driving decision-making unit will perform data synchronization processing. The unit calibrates data collected by different sensors according to a unified timestamp, ensuring that all data corresponds to the vehicle and environmental states at the same moment. After data synchronization is complete, the unit performs scene association processing on all data, matching and associating various data types with corresponding locations and road structures in the map data to obtain assistance scene data.

[0057] Step S20: Generate a remote scenario based on the assistance scenario data, and generate a structured remote assistance instruction based on the remote scenario.

[0058] It should be noted that the remote scene is a virtual scene that matches the actual location of the vehicle, obtained by the remote monitoring platform after visually reconstructing the received assistance scene data at the remote end. The remote scene can intuitively present information such as the vehicle's position and status, the distribution of surrounding obstacles, road structure, and environmental changes, allowing remote operators to observe and make judgments.

[0059] Additionally, structured remote assistance commands are generated by the remote operator based on observations and judgments of the remote scenario, following a pre-defined standardized format, and contain explicit action parameters. Structured remote assistance commands only include parameters such as action type, distance, angle, speed, and priority; they do not contain direct vehicle control signals.

[0060] Understandably, after receiving the assistance scenario data, the remote monitoring platform parses and renders it. The platform transforms information such as vehicle position, surrounding environment, obstacles, and road topology from the assistance scenario data into a visualized remote scene for remote operators to view. By observing the remote scene, the remote operator assesses the vehicle's actual situation and its need for assistance, generating structured remote assistance commands according to a pre-defined standardized format. The remote operator then sends these structured remote assistance commands to the vehicle via an encrypted channel.

[0061] In one feasible implementation, step S20 may include steps S21 to S24: Step S21: Based on the vehicle pose data, surrounding environment data, obstacle data, road topology data, vehicle status data and map data in the assisted scene data, scene fusion is performed to obtain remote scene basic data.

[0062] It should be noted that scene fusion is a process of integrating and processing multi-source, heterogeneous scene data. Its aim is to eliminate redundancy and conflicts between different data sources, forming a unified and complete scene description. Scene fusion can complement information from different sensors and data sources, improving the accuracy and comprehensiveness of scene descriptions.

[0063] In addition, the remote scene base data is a standardized and unified set of scene data obtained after scene fusion processing. The remote scene base data eliminates the time and space deviations between different data sources and can be directly used for subsequent on-site environment reconstruction operations.

[0064] Understandably, after acquiring the data from the assisted scenario, the remote monitoring platform extracts vehicle pose data, surrounding environment data, obstacle data, road topology data, vehicle status data, and map data. The platform then performs consistency checks on the extracted data, eliminating redundant information and conflicting content. Finally, using unified spatial coordinates and a unified time reference, the platform integrates and stitches all the data together to obtain the basic data for the remote scenario.

[0065] Step S22: Reconstruct the on-site environment based on the remote scene basic data to obtain the remote scene.

[0066] It should be noted that on-site environment reconstruction is the process of transforming digitized remote scene data into a visualized virtual scene. On-site environment reconstruction uses graphics rendering technology to present elements such as vehicles, obstacles, roads, and the surrounding environment in an intuitive three-dimensional form.

[0067] Understandably, after acquiring basic data of the remote scene, the remote monitoring platform calls its built-in graphics rendering engine. The platform inputs various element parameters from the basic remote scene data into the rendering engine to generate corresponding 3D models. Based on the vehicle's actual location and the actual layout of the environment, the platform combines and arranges all the 3D models to obtain a remote scene consistent with the vehicle's actual location.

[0068] Step S23: Receive remote assistance operation data according to the remote scenario, and determine the assistance action type, assistance action distance, assistance action angle, assistance action speed, and assistance action priority according to the remote assistance operation data.

[0069] It should be noted that remote assistance operation data is input by the remote operator through the input device of the remote monitoring platform, and is used to express the operator's intention to extricate the vehicle from trouble. Remote assistance operation data contains various requirements from the operator for the vehicle's extrication actions and is the direct basis for generating structured remote assistance instructions.

[0070] Additionally, the type of assisted action refers to the specific category of actions the vehicle needs to perform to get out of trouble, including basic actions such as moving forward, backward, turning left, turning right, and stopping. The type of assisted action clarifies the basic direction and method of movement for the vehicle to get out of trouble and is one of the core fields of structured remote assistance commands.

[0071] Additionally, the assisted action distance refers to the straight-line distance a vehicle needs to move to perform a corresponding assisted action, usually measured in meters. The assisted action distance limits the range of movement of a single vehicle action, preventing excessive vehicle movement that could lead to new safety risks.

[0072] Furthermore, the assist angle refers to the angle that a vehicle needs to turn when performing steering-related assist actions, usually measured in degrees. The assist angle clarifies the range of vehicle steering, ensuring that the vehicle can accurately enter the target traffic area.

[0073] Additionally, the assisted action speed refers to the maximum permissible speed of the vehicle during the assisted action, usually measured in kilometers per hour. The assisted action speed limits the vehicle's speed to ensure safe vehicle operation during the extrication process.

[0074] Additionally, the priority of assistance actions refers to the order in which multiple assistance actions are executed when there are multiple actions to be performed. Assistance action priority is used to standardize the execution flow of multiple actions, ensuring that the escape process proceeds in a reasonable sequence.

[0075] Understandably, the remote operator observes the generated remote scenario through the remote monitoring platform to determine the vehicle's need for extrication and feasible escape routes. The remote operator inputs corresponding remote assistance operation data through the input device of the remote monitoring platform, expressing specific extrication action requirements. The remote monitoring platform parses the input remote assistance operation data, extracting the assistance action type, assistance action distance, assistance action angle, assistance action speed, and assistance action priority.

[0076] Step S24: Encapsulate the instruction field according to the assistance action type, assistance action distance, assistance action angle, assistance action speed, and assistance action priority to obtain a structured remote assistance instruction.

[0077] It should be noted that instruction field encapsulation is the process of combining and packaging discrete action parameters according to a preset standardized format. Instruction field encapsulation can integrate multiple independent parameters into a complete instruction data packet, facilitating instruction transmission and parsing.

[0078] Understandably, after obtaining and parsing the assistance action type, assistance action distance, assistance action angle, assistance action speed, and assistance action priority, the remote monitoring platform arranges them according to the preset instruction field order. The remote monitoring platform assigns a corresponding instruction field position to each parameter and fills in the corresponding parameter value. The remote monitoring platform encrypts the combined instruction data packet and sends it to the vehicle end through an encrypted channel, obtaining a structured remote assistance instruction.

[0079] Step S30: Perform vehicle-side security verification based on the structured remote assistance command and the vehicle operation assistance data to obtain the command verification result.

[0080] It should be noted that vehicle-side security verification refers to a multi-dimensional security verification process performed on the received structured remote assistance commands at the vehicle end. The purpose of vehicle-side security verification is to ensure that structured remote assistance commands meet security and compliance requirements, and to prevent the execution of unsafe structured remote assistance commands from causing vehicle accidents.

[0081] Additionally, the command verification result is obtained after the vehicle-side completes all security verification processes. It serves as the final conclusion indicating whether the structured remote assistance command meets security requirements. The command verification result is divided into two cases: verification passed and verification failed. Different command verification results correspond to different subsequent processing procedures.

[0082] Understandably, upon receiving a structured remote assistance command, the vehicle initiates a multi-layered safety verification process. First, it verifies the command's legality, checking if the action type, distance, angle, and speed parameters within the structured remote assistance command are within preset allowable ranges. Second, it performs environmental collision verification, based on real-time collected vehicle operation assistance data, determining whether the execution path conflicts with obstacles or road boundaries. Finally, it performs compliance verification, ensuring that all dynamic driving tasks are executed by the autonomous driving decision-making unit throughout the entire process without any transfer of control. After completing all verifications, the corresponding command verification result is generated.

[0083] In one feasible implementation, step S30 may include steps S31 to S35: Step S31: Parse the structured remote assistance command to obtain the action type parameter, action distance parameter, action angle parameter, action speed parameter, and command priority parameter.

[0084] It should be noted that command parsing is the process of disassembling and reconstructing the received structured remote assistance command data packet. Command parsing extracts individual parameter content from the encrypted data packet according to a preset command field format, providing the foundational data for subsequent security verification.

[0085] Additionally, the action type parameter is extracted from the structured remote assistance command and is used to identify the specific type of extrication action the vehicle needs to perform. The value range of the action type parameter is consistent with the preset list of allowed actions, and is used to clarify the basic movement method for the vehicle to extricate itself from trouble.

[0086] Additionally, the action distance parameter is extracted from the structured remote assistance instructions and specifies the straight-line distance the vehicle needs to move when performing the corresponding action. The value of the action distance parameter is in meters, limiting the range of movement of the vehicle in a single extrication maneuver.

[0087] Additionally, the motion angle parameter is extracted from the structured remote assistance command and specifies the angle of rotation required when the vehicle performs steering maneuvers. The value of the motion angle parameter is in degrees, clearly defining the magnitude of the vehicle's steering.

[0088] Additionally, the action speed parameter, extracted from structured remote assistance commands, specifies the maximum permissible speed for the vehicle during the extrication maneuver. The action speed parameter is expressed in kilometers per hour, ensuring safe low-speed operation during the extrication process.

[0089] Additionally, the instruction priority parameter is extracted from structured remote assistance instructions and is used to specify the execution order of multiple instructions to be executed. The higher the value of the instruction priority parameter, the earlier the corresponding instruction is executed.

[0090] Understandably, after receiving the encrypted structured remote assistance command, the vehicle first decrypts the command data packet. Following a preset command field order and length, the vehicle extracts each parameter from the decrypted data packet. The vehicle then temporarily stores the extracted action type, distance, angle, speed, and command priority parameters for use in subsequent security checks.

[0091] Step S32: Perform instruction validity verification on the action type parameter, action distance parameter, action angle parameter, and action speed parameter according to the preset action constraint range, and obtain the instruction validity verification result.

[0092] It should be noted that the preset action constraint range is stored in advance on the vehicle and represents the safe allowable range for various traction action parameters. The preset action constraint range is formulated based on the vehicle's physical performance, operating scenario requirements, and safety regulations, and includes a list of allowed action types and upper and lower threshold values ​​for each parameter.

[0093] Additionally, command validity verification is a process that verifies whether the basic parameters of structured remote assistance commands meet preset security requirements. Command validity verification is the first layer of multi-layered security verification, used to quickly filter out invalid commands that clearly exceed the vehicle's capabilities or safety limits.

[0094] Additionally, the instruction validity check result is obtained after the instruction validity check is completed, and it indicates whether the basic parameters of the instruction meet the requirements. The instruction validity check result is divided into two cases: valid and invalid, and different results correspond to different subsequent processing procedures.

[0095] Understandably, after the vehicle-side obtains the parsed parameters, it retrieves the pre-stored preset action constraint ranges. The vehicle-side first checks if the action type parameter exists in the preset allowed action type list. It then sequentially compares the action distance parameter, action angle parameter, and action speed parameter to see if they are within their corresponding preset upper and lower threshold ranges. If all parameters meet the requirements, the action is deemed valid; if any parameter does not meet the requirements, the action is deemed invalid, and the corresponding instruction validity verification result is generated.

[0096] Step S33: Perform environmental risk verification based on the vehicle operation assistance data and the structured remote assistance command to obtain the environmental risk verification result.

[0097] It should be noted that environmental risk verification is a process that verifies potential safety risks after the execution of commands by combining real-time vehicle environmental data. Environmental risk verification is used to determine whether, when performing extrication actions according to commands, there will be a collision with surrounding obstacles or exceeding the road boundary.

[0098] Additionally, the environmental risk verification result is obtained after the environmental risk verification is completed, and is used to indicate whether there is any environmental safety risk in the execution of the instruction. The environmental risk verification result is divided into two cases: no risk and risk present, and different results correspond to different follow-up processing procedures.

[0099] Understandably, the vehicle-mounted system simulates and generates a local driving path based on the parameters of the structured remote assistance command. The system then combines obstacle data and road topology data from the vehicle operation assistance data to check for obstacles along the simulated path. Simultaneously, the system checks whether the simulated path exceeds the preset road boundary. If the simulated path is free of obstacles and does not exceed the road boundary, it is considered risk-free; if obstacles exist or the path exceeds the road boundary, it is considered risky, and a corresponding environmental risk verification result is generated.

[0100] In one feasible implementation, step S33 may include steps S331 to S335: Step S331: Determine the current vehicle pose, current obstacle distribution, current road boundary, and current road topology of the autonomous vehicle based on the vehicle operation assistance data.

[0101] It should be noted that the current vehicle pose refers to the real-time position and attitude data of the autonomous vehicle in space during the environmental risk verification process. The current vehicle pose includes information such as the vehicle's longitude, latitude, altitude, heading angle, and pitch angle, and serves as the reference position and attitude for predicting the command execution path.

[0102] Additionally, the current obstacle distribution refers to the real-time set of the status of all obstacles within a preset safety range around the vehicle during environmental risk verification. The current obstacle distribution includes information such as the location, size, shape, and movement status of each obstacle, providing a comprehensive reflection of the obstacle layout around the vehicle.

[0103] Additionally, the current road boundary refers to the real-time data of the physical edge and legal traffic boundary of the road where the vehicle is located during environmental risk verification. The current road boundary includes information such as the left and right edges of the road, the position of guardrails, and the boundaries of prohibited areas, which is used to define the legal passage range of vehicles.

[0104] Additionally, the current road topology refers to the real-time data of the road network structure and connectivity of the area where the vehicle is located during environmental risk verification. The current road topology includes information such as road direction, number of lanes, intersection locations, and lane connectivity, used to verify whether the path conforms to the actual road structure.

[0105] Understandably, the vehicle-mounted system extracts corresponding real-time raw data from the vehicle operation assistance data. The system then parses the raw data to obtain the current vehicle pose, current obstacle distribution, current road boundaries, and current road topology. Finally, the system converts all data to the vehicle's local coordinate system to ensure consistent spatial reference across all data.

[0106] Step S332: Predict the instruction execution path of the autonomous vehicle based on the current vehicle pose and the structured remote assistance instruction.

[0107] It should be noted that the instruction execution path refers to the complete local path that an autonomous vehicle will travel when it strictly follows the parameters of the structured remote assistance instructions. The instruction execution path includes the position, speed, and attitude information of each point on the path, and is the direct object for collision risk verification and road boundary verification.

[0108] Understandably, the vehicle-side uses its current vehicle pose as the starting point of the path. Based on the action type, distance, angle, and speed parameters in the structured remote assistance command, and combined with the vehicle's kinematic model, the vehicle-side calculates the vehicle's trajectory. The vehicle-side then generates a complete command execution path containing information on all critical path points.

[0109] Step S333: Perform a collision risk check on the instruction execution path based on the current obstacle distribution to obtain the collision risk check result.

[0110] It should be noted that collision risk verification refers to the process of checking whether there are obstacles on the command execution path and determining whether the vehicle will collide with any obstacles when executing that path. Collision risk verification identifies potential collision hazards by spatially comparing the command execution path with the current obstacle distribution, thus ensuring vehicle driving safety.

[0111] Additionally, the collision risk verification result is obtained after the collision risk verification is completed, and is used to indicate the final conclusion as to whether there is a collision risk in the instruction execution path. The collision risk verification result is divided into two cases: no collision risk and collision risk present. Different results correspond to different subsequent processing procedures.

[0112] Understandably, the vehicle iterates through all path points along the instruction execution path. The vehicle calculates the minimum spatial distance between each path point and all obstacles in the current obstacle distribution. The vehicle then determines whether the minimum distance between any path point and any obstacle is less than a preset safe distance threshold. If the distances between all path points and all obstacles are greater than or equal to the safe distance threshold, it is considered that there is no collision risk; if the distance between any path point and any obstacle is less than the safe distance threshold, it is considered that there is a collision risk, and a corresponding collision risk verification result is generated.

[0113] Step S334: Perform road boundary verification on the instruction execution path based on the current road boundary and the current road topology to obtain the road boundary verification result.

[0114] It should be noted that road boundary verification refers to checking whether the command execution path exceeds the current road's permitted passage range, and determining whether the vehicle will deviate from the legal road when executing the path. Road boundary verification combines the current road boundary and the current road topology to ensure that the vehicle always travels within a safe and legal range.

[0115] Additionally, the road boundary verification result is obtained after the road boundary verification is completed, and is used to indicate the final conclusion whether the instruction execution path exceeds the road boundary. The road boundary verification result is divided into two cases: not exceeding the boundary and exceeding the boundary. Different results correspond to different subsequent processing procedures.

[0116] Understandably, the vehicle-side system iterates through all path points along the command execution path. It performs a spatial comparison of each path point with the current road boundary to determine if any path point exceeds the road's physical edge. Simultaneously, the vehicle-side system considers the current road topology to check if the command execution path conforms to the road's lane orientation and connectivity, and whether it has entered prohibited areas. If all path points are within the road boundary and meet the road topology requirements, it is determined that the path has not exceeded the boundary; if any path point exceeds the road boundary or does not meet the road topology requirements, it is determined that the path has exceeded the boundary, and a corresponding road boundary verification result is generated.

[0117] Step S335: Generate environmental clearance information or environmental rejection information based on the collision risk verification result and the road boundary verification result, and determine the environmental clearance information or the environmental rejection information as the environmental risk verification result.

[0118] It should be noted that environmental clearance information is generated when the collision risk verification result is no collision risk and the road boundary verification result is that the road has not exceeded the boundary. This indicates that there are no environmental safety risks in the instruction execution path. Environmental clearance information allows the instruction to enter the subsequent control rights verification process.

[0119] Additionally, environmental rejection information is generated when the collision risk verification result indicates a collision risk or the road boundary verification result indicates that the path exceeds the boundary. This information indicates that there is an environmental safety risk in the command execution path. The environmental rejection information includes the specific risk type and risk location, and is used to provide feedback to the remote monitoring platform for command adjustment.

[0120] Understandably, after obtaining the collision risk verification results and road boundary verification results, the vehicle-side system makes a comprehensive judgment on the two results. If both results are passed, environmental clearance information is generated. If either result is failed, environmental rejection information is generated, specifying the specific risk type. The vehicle-side system then determines the generated environmental clearance information or environmental rejection information as the final environmental risk verification result.

[0121] Step S34: Perform dynamic driving task control authority verification based on the executing entity corresponding to the structured remote assistance instruction, and obtain the control authority verification result.

[0122] It should be noted that dynamic driving task control refers to the authority to execute all dynamic driving tasks. The allocation of dynamic driving task control determines who is responsible for the vehicle's operation and safety, and is a core element of autonomous driving compliance.

[0123] Additionally, control verification is a process that verifies whether structured remote assistance commands involve a transfer of control. Control verification ensures that the entire dynamic driving task is executed by the autonomous driving decision-making unit without transferring any control to the remote end.

[0124] Additionally, the control verification result, obtained after the control verification process, indicates whether the instruction involves a transfer of control. Control verification results are categorized as compliant or non-compliant, with different follow-up processing procedures for each result.

[0125] Understandably, the vehicle-side inspection checks the structure of the remote assistance instructions, confirming that the instructions only contain action parameter information. The vehicle-side verification instructions do not contain any signals directly controlling the vehicle's drive, braking, or steering systems. The executing entity of the vehicle-side confirmation instructions is always the autonomous driving decision-making unit; there is no content requiring the remote end to take over vehicle control. If the instruction meets the above requirements, it is deemed compliant; if there is content related to control transfer, it is deemed non-compliant, and a corresponding control verification result is generated.

[0126] Step S35: Generate instruction release information or instruction rejection information based on the instruction legality verification result, the environmental risk verification result, and the control right verification result, and determine the instruction release information or the instruction rejection information as the instruction verification result.

[0127] It should be noted that the instruction release information is generated when all safety checks pass, allowing the autonomous driving decision-making unit to execute the corresponding structured remote assistance instruction. The instruction release information indicates that the instruction fully complies with safety and compliance requirements and can proceed to the execution phase.

[0128] Additionally, a command rejection message is generated when any security check fails, prohibiting the execution of the corresponding structured remote assistance command. The command rejection message includes the specific reason for the verification failure, which is then fed back to the remote monitoring platform for processing.

[0129] Understandably, after obtaining the results of the command validity verification, environmental risk verification, and control verification, the vehicle-side system comprehensively judges these three results. If all three verifications pass, a command release message is generated. If any verification fails, a command rejection message is generated, specifying the reason for the rejection. The vehicle-side system then determines the generated command release or rejection message as the final command verification result and sends it back to the remote monitoring platform.

[0130] Step S40: When the instruction verification result is successful, remote assistance for the autonomous vehicle is performed according to the structured remote assistance instruction.

[0131] It should be noted that the Automated Driving Unit (ADU) is the core control unit of an autonomous vehicle, responsible for executing all dynamic driving tasks. The ADU possesses functions such as path planning, motion control, and safety monitoring, and is the core component for achieving autonomous driving and remote assistance in overcoming difficulties.

[0132] Additionally, Dynamic Driving Task (DDT) refers to all lateral and longitudinal motion control tasks required to complete driving operations while the vehicle is in motion. Dynamic driving tasks include steering wheel operation, throttle control, brake control, and environmental perception and decision-making.

[0133] Understandably, when the command verification result is successful, the autonomous driving decision unit will receive a structured remote assistance command. Based on the action parameters in the structured remote assistance command, the autonomous driving decision unit plans a local escape path. The autonomous driving decision unit autonomously controls the vehicle's drive, braking, and steering systems to execute the escape maneuvers according to the planned local escape path. During execution, the autonomous driving decision unit monitors the vehicle's status and changes in the surrounding environment in real time, and immediately takes emergency braking measures in the event of sudden danger.

[0134] In one feasible implementation, step S40 may include steps S41 to S45: Step S41: When the instruction verification result is successful, local path planning is performed based on the structured remote assistance instruction and the assistance scenario data to obtain the remote assistance execution path.

[0135] It should be noted that local path planning refers to the process of generating a specific driving path for a vehicle over a short distance, based on the global path and considering the vehicle's current local environment and specific task requirements. Local path planning can dynamically adjust the driving path according to real-time environmental changes and task requirements, ensuring the safe passage of the vehicle in complex environments.

[0136] Additionally, the remote assistance execution path refers to a local driving path specifically designed for vehicles to perform remote assistance maneuvers, generated based on structured remote assistance commands and assistance scenario data. The remote assistance execution path includes information such as the vehicle's precise trajectory, speed changes, and attitude adjustments, serving as the direct basis for the vehicle to perform maneuvers.

[0137] Understandably, when the instruction verification result is successful, the autonomous driving decision unit will acquire the structured remote assistance instruction and assistance scenario data. Using the current vehicle pose as the path starting point, the autonomous driving decision unit combines the action parameter requirements in the structured remote assistance instruction with the obstacle distribution and road topology information in the assistance scenario data to generate a remote assistance execution path that meets safety requirements.

[0138] Step S42: Generate drive control parameters, braking control parameters, and steering control parameters according to the remote assistance execution path.

[0139] It should be noted that drive control parameters are parameters used to control the output power of the vehicle's drive system, including motor speed, torque output, and throttle opening. Drive control parameters determine the vehicle's acceleration performance and speed, and are the core parameters for achieving longitudinal motion control of the vehicle.

[0140] Additionally, braking control parameters are used to control the braking force of a vehicle's braking system, including braking pressure, braking duration, and braking deceleration. These parameters determine the vehicle's deceleration performance and stopping accuracy, and are crucial for ensuring vehicle safety.

[0141] Additionally, steering control parameters are used to control the steering angle and speed of the vehicle's steering system, including steering wheel angle, steering angular velocity, and steering torque. These parameters determine the vehicle's steering trajectory and steering accuracy, and are the core parameters for achieving lateral motion control of the vehicle.

[0142] Understandably, after acquiring the generated remote-assisted execution path, the autonomous driving decision-making unit will decompose the path into a trajectory. Based on the position, speed, and attitude requirements of each point on the path, and in conjunction with the vehicle's dynamics model, the autonomous driving decision-making unit calculates the corresponding drive control parameters, braking control parameters, and steering control parameters at each moment.

[0143] Step S43: Control the autonomous vehicle to perform remote assistance actions according to the drive control parameters, the braking control parameters and the steering control parameters, and collect the real-time status of the vehicle performing remote assistance actions and the environmental change status.

[0144] It should be noted that remote assistance actions refer to a series of vehicle movement operations performed by the vehicle according to structured remote assistance instructions and planned remote assistance execution paths to help the vehicle get out of trouble. Remote assistance actions include basic actions such as moving forward, backward, turning, and stopping, and are the specific execution methods for the vehicle to get out of abnormal operating scenarios.

[0145] Additionally, the real-time status of a vehicle performing remote assistance actions refers to the real-time data collected on its own operating status during the process of retrieving itself from difficult situations. This data includes the vehicle's real-time speed, steering angle, braking status, battery charge, and position information, comprehensively reflecting the vehicle's real-time operating status.

[0146] Additionally, environmental change status refers to the data collected in real time regarding changes in the surrounding environment during the vehicle's extrication maneuver. This data includes changes in the position of obstacles, road conditions, and the movement of surrounding objects, and can promptly reflect unexpected situations in the environment.

[0147] Understandably, the autonomous driving decision-making unit sends the generated drive control parameters, braking control parameters, and steering control parameters to the corresponding vehicle actuators. Based on the received control parameters, the vehicle actuators execute the corresponding drive, braking, and steering operations to complete the remotely assisted action. During execution, the autonomous driving decision-making unit continuously collects the vehicle's real-time status and changes in the surrounding environment through onboard sensors.

[0148] Step S44: Perform safety monitoring on the remote assistance action based on the real-time status of the vehicle and the changes in the environment, and obtain the safety monitoring results.

[0149] It should be noted that safety monitoring refers to the continuous monitoring and analysis of the vehicle's own status and the surrounding environment during the execution of remote assistance actions. The purpose of safety monitoring is to promptly identify potential safety risks during the execution process and take corresponding safety measures to ensure the safety of the vehicle and personnel.

[0150] Additionally, safety monitoring results refer to conclusions obtained after completing safety monitoring, indicating whether there are any safety risks during the vehicle's remote assistance operations. Safety monitoring results are categorized into two types: safe and those indicating potential safety risks, with different follow-up procedures corresponding to different results.

[0151] Understandably, after acquiring the real-time vehicle status and environmental changes, the autonomous driving decision-making unit will analyze this data in real time. The unit determines whether the vehicle's real-time status is within a safe operating range and whether environmental changes will pose new safety threats to the vehicle. If the vehicle is in a normal state and there are no sudden environmental risks, it is deemed safe; if the vehicle is in an abnormal state or there are sudden environmental risks, it is deemed to have a safety risk, and corresponding safety monitoring results are generated.

[0152] Step S45: When the safety monitoring result passes, remote assistance for the autonomous vehicle is performed according to the structured remote assistance instructions.

[0153] Understandably, when the safety monitoring result is satisfactory, the autonomous driving decision-making unit will continue to execute remote assistance actions according to the structured remote assistance instructions. The autonomous driving decision-making unit continuously performs path tracking and control parameter adjustments to ensure that the vehicle accurately completes all extrication actions. Throughout the entire execution process, the safety monitoring process continues to run, monitoring the vehicle and environmental status in real time.

[0154] This embodiment provides a remote assistance method for autonomous vehicles. By automatically identifying abnormal operating scenarios and collecting full-dimensional vehicle and environmental data, it generates visualized remote scenarios and standardized structured remote assistance instructions. After undergoing three layers of safety verification—instruction legality, environmental risk, and control rights—the autonomous driving decision-making unit autonomously executes the escape action. This achieves non-interventionist remote assistance where the driver is in the loop and the vehicle is performing the action, while simultaneously meeting the requirements for autonomous driving regulatory compliance, escape process safety, and escape efficiency in complex scenarios.

[0155] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 After step S40, the remote assistance method for the autonomous vehicle further includes steps S50 to S90: Step S50: Obtain the real-time status of the autonomous vehicle, the status of environmental changes, and the progress of assistance execution during the remote assistance process.

[0156] It should be noted that the remote assistance process refers to the entire process from when the autonomous vehicle sends a remote assistance request to when the vehicle completes the extrication action and is ready to resume autonomous driving mode. The remote assistance process includes multiple stages such as command generation, safety verification, extrication execution, and status feedback, and is a complete process to help the vehicle get out of abnormal operating scenarios.

[0157] Additionally, the assistance execution progress refers to the proportion of tasks completed by the autonomous vehicle during remote assistance operations out of the total task volume. Assistance execution progress provides a clear picture of the completion status of the extrication actions, helping the remote monitoring platform to keep track of the progress of the assistance task in real time.

[0158] Understandably, during the remote assistance process, the autonomous driving decision-making unit continuously collects real-time data on the vehicle's status and environmental changes. Simultaneously, the unit calculates the current assistance progress based on the execution status of the structured remote assistance commands. The autonomous driving decision-making unit then transmits the collected real-time vehicle status, environmental changes, and calculated assistance progress back to the remote monitoring platform in real time.

[0159] Step S60: Determine whether the autonomous driving vehicle meets the conditions for autonomous driving recovery based on the real-time status of the vehicle, the environmental change status, and the progress of the assisted execution, and obtain the recovery determination result.

[0160] It should be noted that the conditions for resuming autonomous driving are pre-set and serve as the criteria for determining whether an autonomous vehicle can switch back from remote assistance mode to normal autonomous driving mode. These conditions include multiple dimensions such as normal vehicle status, no safety risks in the environment, and completion of all obstacle-avoidance maneuvers, ensuring the vehicle can operate safely and stably after resuming autonomous driving.

[0161] Additionally, the recovery determination result is obtained by the autonomous driving decision-making unit based on various conditions, indicating whether the vehicle can resume autonomous driving mode. The recovery determination result is divided into two cases: meeting the conditions for autonomous driving recovery and not meeting the conditions for autonomous driving recovery. Different results correspond to different subsequent processing procedures.

[0162] Understandably, the autonomous driving decision-making unit obtains real-time data on the vehicle's status, environmental changes, and the progress of assisted execution. This data is then compared one by one with pre-defined autonomous driving recovery conditions. If all conditions are met, the autonomous driving recovery condition is deemed met; if any condition is not met, the autonomous driving recovery condition is deemed not met, and a corresponding recovery judgment result is generated.

[0163] Step S70: When the recovery determination result meets the conditions for autonomous driving recovery, the autonomous vehicle is repositioned and the map is matched to obtain vehicle recovery positioning data.

[0164] It should be noted that repositioning refers to the process by which an autonomous vehicle, after completing the extrication maneuver, re-determines its precise position and attitude on the map. Repositioning corrects any positioning deviations that may occur during the extrication process, ensuring that the vehicle has an accurate positional reference once autonomous driving is resumed.

[0165] Additionally, map matching refers to the process of comparing and matching real-time vehicle location data with pre-stored high-precision map data. Map matching can associate the vehicle's location results with elements such as roads and lanes in the map, ensuring that the vehicle can accurately identify its road position and lane information.

[0166] Additionally, vehicle repositioning data, obtained after repositioning and map matching, provides the vehicle's precise location and attitude data when resuming autonomous driving mode. This data includes the vehicle's longitude, latitude, altitude, heading angle, and lane position, serving as the core foundational data for resuming autonomous driving operation.

[0167] Understandably, when the recovery determination result meets the conditions for autonomous driving recovery, the autonomous driving decision-making unit will initiate the repositioning process. The autonomous driving decision-making unit collects real-time positioning data through the onboard GPS, inertial measurement unit, and multi-source perception sensors. The autonomous driving decision-making unit then performs map matching between the collected real-time positioning data and high-precision map data to correct positioning deviations and obtain the vehicle's restored positioning data.

[0168] Step S80: Restore the autonomous driving operation mode of the autonomous vehicle based on the vehicle recovery positioning data to obtain the autonomous driving recovery status.

[0169] It should be noted that autonomous driving mode refers to an operating mode in which the autonomous vehicle's decision-making unit performs all dynamic driving tasks without human driver intervention. In autonomous driving mode, the vehicle can autonomously complete all driving operations, including path planning, motion control, and safety monitoring, to achieve normal driving tasks.

[0170] Additionally, the autonomous driving recovery status refers to the operational status of an autonomous vehicle after it has successfully switched back from remote assistance mode to autonomous driving mode. The autonomous driving recovery status includes information such as whether the mode switch was successful, the vehicle's current operational status, and whether the autonomous driving functions are functioning normally, indicating whether the vehicle has regained its normal operational capabilities.

[0171] Understandably, after acquiring the vehicle's restored positioning data, the autonomous driving decision-making unit will update the vehicle's location reference to match the restored positioning data. The autonomous driving decision-making unit then reloads the global driving path and initiates the autonomous driving decision-making and control process. The autonomous driving decision-making unit verifies that all autonomous driving functions are operating normally, completes the mode switch, and obtains the autonomous driving recovery status.

[0172] Step S90: Generate remote assistance feedback data based on the real-time vehicle status, the assistance execution progress, and the autonomous driving recovery status to provide remote assistance feedback.

[0173] It should be noted that the remote assistance feedback data is a dataset containing relevant information from the entire remote assistance process. This data includes real-time vehicle status, assistance progress, autonomous driving recovery status, and information on abnormal scenarios, serving as a crucial basis for the remote monitoring platform to record and evaluate the assistance task.

[0174] Additionally, assistance feedback refers to the process of sending relevant information about this remote assistance to the remote monitoring platform. Assistance feedback allows the remote monitoring platform to fully record all aspects of the assistance task, providing data support for subsequent data analysis and optimization.

[0175] Understandably, after acquiring the vehicle's real-time status, the progress of the assistance execution, and the autonomous driving recovery status, the autonomous driving decision-making unit will organize and summarize this data. The unit will then supplement this data with information such as the type of abnormal scenario, command execution details, and time consumption for this remote assistance, generating complete remote assistance feedback data. Finally, the unit will send this feedback data to the remote monitoring platform, completing the assistance feedback for this remote assistance session.

[0176] This embodiment provides a remote assistance method for autonomous vehicles. By collecting vehicle status, environmental changes and execution progress in real time during the remote assistance process, it automatically determines the conditions for autonomous driving to resume. After completing vehicle repositioning and map matching, it smoothly switches to autonomous driving operation mode and generates complete remote assistance feedback data. This achieves safety control and operational continuity assurance throughout the remote assistance process, while providing traceable task records and data support for subsequent process optimization for the remote monitoring platform.

[0177] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the remote assistance method for autonomous vehicles of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0178] This application also provides a remote assistance device for autonomous vehicles, please refer to... Figure 3 The remote assistance device for the autonomous vehicle includes: The data acquisition module 10 is used to acquire remote assistance requests generated by autonomous vehicles in abnormal operating scenarios, and to collect vehicle operation assistance data according to the remote assistance requests to obtain assistance scenario data. The instruction generation module 20 is used to generate a remote scenario based on the assistance scenario data, and to generate structured remote assistance instructions based on the remote scenario. The security verification module 30 is used to perform vehicle-side security verification based on the structured remote assistance command and the vehicle operation assistance data, and obtain the command verification result. The remote assistance module 40 is used to provide remote assistance to the autonomous vehicle according to the structured remote assistance instruction when the instruction verification result is successful.

[0179] The remote assistance device for autonomous vehicles provided in this application, employing the remote assistance method for autonomous vehicles in the above embodiments, can solve the technical problem that existing remote assistance methods for autonomous vehicles struggle to simultaneously address the requirements of compliant vehicle control management and complex scenario handling in abnormal operating scenarios, resulting in insufficient adaptability of remote assistance for autonomous light trucks. Compared with the prior art, the beneficial effects of the remote assistance device for autonomous vehicles provided in this application are the same as those of the remote assistance method for autonomous vehicles provided in the above embodiments, and other technical features in the remote assistance device for autonomous vehicles are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0180] In one embodiment, the data acquisition module 10 is further configured to acquire vehicle traffic status data generated by the autonomous vehicle in the autonomous driving activated state; Based on the vehicle traffic status data, determine whether the autonomous vehicle is in an abnormal operating scenario, and obtain the abnormal operating scenario determination result; When the abnormal operation scenario determination result indicates that the system is in an abnormal operation scenario, a remote assistance request is generated. According to the remote assistance request, vehicle pose data, surrounding environment data, obstacle data, road topology data, vehicle status data and map data are collected, and vehicle operation assistance data is generated based on the vehicle pose data, surrounding environment data, obstacle data, road topology data, vehicle status data and map data. Based on the vehicle operation assistance data, data synchronization and scene association are performed to obtain assistance scene data.

[0181] In one embodiment, the instruction generation module 20 is further configured to perform scene fusion based on vehicle pose data, surrounding environment data, obstacle data, road topology data, vehicle status data and map data in the assistance scene data to obtain remote scene basic data; The remote scene is obtained by reconstructing the on-site environment based on the aforementioned remote scene basic data; The remote assistance operation data is received according to the remote scenario, and the assistance action type, assistance action distance, assistance action angle, assistance action speed and assistance action priority are determined according to the remote assistance operation data. The instruction fields are encapsulated according to the type of assistance action, the distance of the assistance action, the angle of the assistance action, the speed of the assistance action, and the priority of the assistance action to obtain a structured remote assistance instruction.

[0182] In one embodiment, the security verification module 30 is further configured to parse the structured remote assistance command to obtain action type parameters, action distance parameters, action angle parameters, action speed parameters, and command priority parameters; The legality of the command is verified by the action type parameter, the action distance parameter, the action angle parameter, and the action speed parameter according to the preset action constraint range, and the result of the command legality verification is obtained. Environmental risk verification is performed based on the vehicle operation assistance data and the structured remote assistance instructions to obtain the environmental risk verification result. Based on the executing entity corresponding to the structured remote assistance instruction, a dynamic driving task control right verification is performed to obtain the control right verification result; Based on the legality verification result of the instruction, the environmental risk verification result, and the control verification result, an instruction release information or an instruction rejection information is generated, and the instruction release information or the instruction rejection information is determined as the instruction verification result.

[0183] In one embodiment, the safety verification module 30 is further configured to determine the current vehicle pose, current obstacle distribution, current road boundary, and current road topology of the autonomous vehicle based on the vehicle operation assistance data; Predict the instruction execution path of the autonomous vehicle based on the current vehicle pose and the structured remote assistance instructions; Based on the current obstacle distribution, the command execution path is subjected to collision risk verification to obtain the collision risk verification result; Based on the current road boundary and the current road topology, the instruction execution path is checked for road boundary to obtain the road boundary check result. Based on the collision risk verification result and the road boundary verification result, environmental clearance information or environmental rejection information is generated, and the environmental clearance information or the environmental rejection information is determined as the environmental risk verification result.

[0184] In one embodiment, the remote assistance module 40 is further configured to perform local path planning based on the structured remote assistance instruction and the assistance scenario data to obtain a remote assistance execution path when the instruction verification result is a verification pass. Drive control parameters, braking control parameters, and steering control parameters are generated based on the remote assistance execution path; The autonomous vehicle is controlled to perform remote assistance actions based on the drive control parameters, the braking control parameters, and the steering control parameters, and the real-time status of the vehicle performing remote assistance actions and the environmental change status are collected. Based on the real-time status of the vehicle and the changing environmental conditions, the remote assistance action is monitored for safety, and the safety monitoring results are obtained. When the safety monitoring results are passed, remote assistance for the autonomous vehicle is provided according to the structured remote assistance instructions.

[0185] In one embodiment, the remote assistance module 40 is further configured to acquire the real-time vehicle status, environmental change status, and assistance execution progress of the autonomous vehicle during the remote assistance process. Based on the real-time status of the vehicle, the changing status of the environment, and the progress of the assisted execution, determine whether the autonomous vehicle meets the conditions for autonomous driving recovery, and obtain a recovery determination result; When the recovery determination result is that the autonomous driving recovery conditions are met, the autonomous vehicle is repositioned and the map is matched to obtain vehicle recovery positioning data; The autonomous driving operation mode of the autonomous vehicle is restored based on the vehicle's restored positioning data, thus obtaining the autonomous driving recovery status; Remote assistance feedback data is generated based on the vehicle's real-time status, the progress of the assistance execution, and the autonomous driving recovery status to provide remote assistance feedback.

[0186] This application provides a remote assistance device for an autonomous vehicle, the remote assistance device for an autonomous vehicle comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the remote assistance method for an autonomous vehicle as described in Embodiment 1 above.

[0187] The following is for reference. Figure 4 This document illustrates a structural schematic diagram of a remote assistance device suitable for implementing the embodiments of this application for autonomous vehicles. The remote assistance device for autonomous vehicles in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The remote assistance device for autonomous vehicles shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0188] likeFigure 4 As shown, the remote assistance device for an autonomous vehicle may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the remote assistance device for the autonomous vehicle. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the remote assistance device of an autonomous vehicle to communicate wirelessly or wiredly with other devices to exchange data. While the figure shows remote assistance devices for autonomous vehicles with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0189] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0190] The remote assistance device for autonomous vehicles provided in this application, employing the remote assistance method for autonomous vehicles in the above embodiments, can solve the technical problem that existing remote assistance methods for autonomous vehicles struggle to simultaneously address the requirements of compliant vehicle control management and complex scenario handling in abnormal operating scenarios, resulting in insufficient adaptability of remote assistance for autonomous light trucks. Compared with the prior art, the beneficial effects of the remote assistance device for autonomous vehicles provided in this application are the same as those of the remote assistance method for autonomous vehicles provided in the above embodiments, and other technical features in the remote assistance device for autonomous vehicles are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0191] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0192] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0193] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the remote assistance method for an autonomous vehicle in the above embodiments.

[0194] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, RAM (Random Access Memory), ROM (Read Only Memory), Erasable Programmable Read Only Memory (EPROM), optical fiber, CD-ROM (CD-Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0195] The aforementioned computer-readable storage medium may be included in the remote assistance device of the autonomous vehicle; or it may exist independently and not be installed in the remote assistance device of the autonomous vehicle.

[0196] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by a remote assistance device for an autonomous vehicle, the remote assistance device for the autonomous vehicle causes the following: it acquires a remote assistance request generated by the autonomous vehicle in an abnormal operating scenario, and collects vehicle operation assistance data based on the remote assistance request to obtain assistance scenario data; it generates a remote scenario based on the assistance scenario data, and generates a structured remote assistance instruction based on the remote scenario; it performs vehicle-side security verification based on the structured remote assistance instruction and the vehicle operation assistance data to obtain an instruction verification result; and when the instruction verification result is a successful verification, it performs remote assistance for the autonomous vehicle based on the structured remote assistance instruction.

[0197] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including LAN (Local Area Network) or WAN (Wide Area Network)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0198] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0199] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0200] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned remote assistance method for autonomous vehicles. This addresses the technical problem that existing remote assistance methods for autonomous vehicles struggle to balance compliance management of vehicle control with the handling of complex scenarios in abnormal operating conditions, resulting in insufficient adaptability of remote assistance for autonomous light trucks. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the remote assistance method for autonomous vehicles provided in the above embodiments, and will not be elaborated upon here.

[0201] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the remote assistance method for an autonomous vehicle as described above.

[0202] The computer program product provided in this application can solve the technical problem that existing remote assistance methods for autonomous driving are unable to simultaneously meet the requirements of compliant management of vehicle control and handling of complex scenarios under abnormal operating conditions, resulting in insufficient adaptability of remote assistance for autonomous driving light trucks. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the remote assistance method for autonomous vehicles provided in the above embodiments, and will not be repeated here.

[0203] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for remote assistance of an autonomous vehicle, characterized in that, The method includes: The system acquires remote assistance requests generated by autonomous vehicles in abnormal operating scenarios, and collects vehicle operation assistance data based on the remote assistance requests to obtain assistance scenario data. A remote scenario is generated based on the assistance scenario data, and a structured remote assistance instruction is generated based on the remote scenario; Vehicle-side security verification is performed based on the structured remote assistance command and the vehicle operation assistance data to obtain the command verification result; When the instruction verification result is successful, remote assistance for the autonomous vehicle is performed according to the structured remote assistance instruction.

2. The method as described in claim 1, characterized in that, The steps of obtaining remote assistance requests generated by autonomous vehicles in abnormal operating scenarios and collecting vehicle operation assistance data based on the remote assistance requests to obtain assistance scenario data include: Obtain the vehicle traffic status data generated by the autonomous vehicle in the autonomous driving activated state; Based on the vehicle traffic status data, determine whether the autonomous vehicle is in an abnormal operating scenario, and obtain the abnormal operating scenario determination result; When the abnormal operation scenario determination result indicates that the system is in an abnormal operation scenario, a remote assistance request is generated. According to the remote assistance request, vehicle pose data, surrounding environment data, obstacle data, road topology data, vehicle status data and map data are collected, and vehicle operation assistance data is generated based on the vehicle pose data, surrounding environment data, obstacle data, road topology data, vehicle status data and map data. Based on the vehicle operation assistance data, data synchronization and scene association are performed to obtain assistance scene data.

3. The method as described in claim 1, characterized in that, The steps of generating a remote scenario based on the assistance scenario data and generating structured remote assistance instructions based on the remote scenario include: Based on the vehicle pose data, surrounding environment data, obstacle data, road topology data, vehicle status data, and map data in the assisted scene data, scene fusion is performed to obtain remote scene basic data; The remote scene is obtained by reconstructing the on-site environment based on the aforementioned remote scene basic data; The remote assistance operation data is received according to the remote scenario, and the assistance action type, assistance action distance, assistance action angle, assistance action speed and assistance action priority are determined according to the remote assistance operation data. The instruction fields are encapsulated according to the type of assistance action, the distance of the assistance action, the angle of the assistance action, the speed of the assistance action, and the priority of the assistance action to obtain a structured remote assistance instruction.

4. The method as described in claim 1, characterized in that, The step of performing vehicle-side security verification based on the structured remote assistance command and the vehicle operation assistance data to obtain the command verification result includes: The structured remote assistance command is parsed to obtain the action type parameter, action distance parameter, action angle parameter, action speed parameter, and command priority parameter; The legality of the command is verified by the action type parameter, the action distance parameter, the action angle parameter, and the action speed parameter according to the preset action constraint range, and the result of the command legality verification is obtained. Environmental risk verification is performed based on the vehicle operation assistance data and the structured remote assistance instructions to obtain the environmental risk verification result. Based on the executing entity corresponding to the structured remote assistance instruction, a dynamic driving task control right verification is performed to obtain the control right verification result; Based on the legality verification result of the instruction, the environmental risk verification result, and the control verification result, an instruction release information or an instruction rejection information is generated, and the instruction release information or the instruction rejection information is determined as the instruction verification result.

5. The method as described in claim 4, characterized in that, The step of performing environmental risk verification based on the vehicle operation assistance data and the structured remote assistance command to obtain the environmental risk verification result includes: The current vehicle pose, current obstacle distribution, current road boundary, and current road topology of the autonomous vehicle are determined based on the vehicle operation assistance data. Predict the instruction execution path of the autonomous vehicle based on the current vehicle pose and the structured remote assistance instructions; Based on the current obstacle distribution, the command execution path is subjected to collision risk verification to obtain the collision risk verification result; Based on the current road boundary and the current road topology, the instruction execution path is checked for road boundary to obtain the road boundary check result. Based on the collision risk verification result and the road boundary verification result, environmental clearance information or environmental rejection information is generated, and the environmental clearance information or the environmental rejection information is determined as the environmental risk verification result.

6. The method as described in claim 1, characterized in that, The step of providing remote assistance to the autonomous vehicle based on the structured remote assistance instruction when the instruction verification result is successful includes: When the instruction verification result is successful, local path planning is performed based on the structured remote assistance instruction and the assistance scenario data to obtain the remote assistance execution path; Drive control parameters, braking control parameters, and steering control parameters are generated based on the remote assistance execution path; The autonomous vehicle is controlled to perform remote assistance actions based on the drive control parameters, the braking control parameters, and the steering control parameters, and the real-time status of the vehicle performing remote assistance actions and the environmental change status are collected. Based on the real-time status of the vehicle and the changing environmental conditions, the remote assistance action is monitored for safety, and the safety monitoring results are obtained. When the safety monitoring results are passed, remote assistance for the autonomous vehicle is provided according to the structured remote assistance instructions.

7. The method as described in claim 1, characterized in that, After the step of remotely assisting the autonomous vehicle according to the structured remote assistance instruction when the instruction verification result is successful, the method further includes: The system acquires the real-time status of the autonomous vehicle, changes in the environment, and the progress of the assistance process during the remote assistance process. Based on the real-time status of the vehicle, the changing status of the environment, and the progress of the assisted execution, determine whether the autonomous vehicle meets the conditions for autonomous driving recovery, and obtain a recovery determination result; When the recovery determination result is that the autonomous driving recovery conditions are met, the autonomous vehicle is repositioned and the map is matched to obtain vehicle recovery positioning data; The autonomous driving operation mode of the autonomous vehicle is restored based on the vehicle's restored positioning data, thus obtaining the autonomous driving recovery status; Remote assistance feedback data is generated based on the vehicle's real-time status, the progress of the assistance execution, and the autonomous driving recovery status to provide remote assistance feedback.

8. A remote assistance device for an autonomous vehicle, characterized in that, The device includes: The data acquisition module is used to acquire remote assistance requests generated by autonomous vehicles in abnormal operating scenarios, and to collect vehicle operation assistance data based on the remote assistance requests to obtain assistance scenario data. The instruction generation module is used to generate a remote scenario based on the assistance scenario data, and to generate structured remote assistance instructions based on the remote scenario. The security verification module is used to perform vehicle-side security verification based on the structured remote assistance command and the vehicle operation assistance data, and obtain the command verification result. The remote assistance module is used to provide remote assistance to the autonomous vehicle according to the structured remote assistance instruction when the instruction verification result is successful.

9. A remote assistance device for an autonomous vehicle, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the remote assistance method for an autonomous vehicle as claimed in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the remote assistance method for an autonomous vehicle as described in any one of claims 1 to 7.

Images