Unmanned vehicle autonomous decision method and system based on end-side cloud cooperation

Abstract

The application discloses an unmanned vehicle autonomous decision-making method and system based on end-side cloud cooperation, and relates to the technical field of unmanned vehicle intelligent decision-making, and the method comprises the following steps: responding to a patrol task instruction and performing semantic analysis and task decomposition to obtain a subtask queue, performing topological adaptation based on a park semantic map and the subtask queue to obtain an initial patrol path; determining a sudden abnormal event by extracting abnormal features of multi-modal environment perception data in real time; dividing a first task branch line and a second task branch line according to the task correlation degree of uncompleted subtasks and the sudden abnormal event; determining an execution priority according to an abnormal level attribute and generating a task execution sequence; and performing path re-planning based on the task execution sequence and a real-time position to generate a target planning path. The application realizes dynamic decomposition and adaptive path re-planning of the patrol task, and improves autonomous operation efficiency and emergency response capability in a complex scene.

Description

Technical Field

[0001] This invention relates to the field of intelligent decision-making technology for unmanned vehicles, specifically to an autonomous decision-making method and system for unmanned vehicles based on edge-cloud collaboration. Background Technology

[0002] Previously, in scenarios such as park security monitoring and port inspection, unmanned vehicle inspection operations typically relied on pre-set fixed routes and structured instructions for automated patrolling. However, actual inspection environments are highly dynamic and uncertain, including unexpected obstacles, sudden personnel intrusions, or equipment malfunctions. Most existing unmanned vehicle control systems only possess basic obstacle avoidance functions or simple task interruption mechanisms, lacking the ability to understand the semantics of natural language dispatch instructions. Furthermore, when faced with sudden abnormal events, they cannot autonomously assess the impact on the current inspection task based on the level and type of the anomaly. This results in vehicles either mechanically executing the original plan, delaying the response time, or simply pausing the task and waiting for remote human intervention. This lack of dynamic task deconstruction and autonomous decision-making capabilities makes it difficult for unmanned vehicles to truly replace humans in completing fully autonomous inspection operations in complex scenarios, severely restricting inspection efficiency and risk response speed.

[0003] The information disclosed in the background section is only intended to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] The purpose of this invention is to address the problems of rigid execution of unmanned vehicle inspection tasks and lack of dynamic autonomous decision-making capabilities in existing technologies. This application achieves dynamic deconstruction and adaptive path replanning of inspection tasks through an unmanned vehicle autonomous decision-making method and system based on edge-cloud collaboration.

[0005] In a first aspect, one technical solution provided in this embodiment of the invention is: an autonomous decision-making method for unmanned vehicles based on edge-cloud collaboration, comprising the following steps: In response to the inspection task instruction, the system performs semantic parsing and task deconstruction on the inspection task instruction to obtain a subtask queue containing several subtasks. Based on the pre-built park semantic map, the system performs topology adaptation with the subtask queue to obtain the initial inspection path. Real-time extraction of abnormal features from multimodal environmental perception data along the initial inspection path to identify sudden abnormal events that characterize the anomaly level and spatial location attributes; In response to the identified sudden abnormal event, unfinished subtasks are extracted from the subtask queue. Based on the task correlation between each unfinished subtask and the sudden abnormal event, the unfinished subtasks are divided into a first task branch for handling the sudden abnormal event and a second task branch for continuing to execute the unfinished inspection task. The execution priority of each subtask is determined based on the anomaly level attribute of the sudden abnormal event. The first task branch and the second task branch are sorted according to the execution priority to generate a task execution sequence. Based on the task execution sequence and the real-time location of the unmanned vehicle, the park semantic map is invoked to perform path replanning and generate a target planned path. The unmanned vehicle then responds to the target planned path to perform inspection tasks.

[0006] As an optional approach, the steps of semantically parsing and deconstructing the inspection task instructions to obtain a subtask queue containing several subtasks are as follows: The inspection task instructions are obtained and their types are identified. The instruction types include natural language scheduling instructions and preset structured task instructions. If identified as a natural language scheduling instruction, a lightweight visual language model deployed on edge computing nodes is invoked to perform semantic understanding and intent extraction on the natural language scheduling instruction to obtain semantic elements. Based on these elements, the natural language scheduling instruction is decomposed into several sub-tasks with temporal dependencies. The semantic elements include action type, target object, and spatial range. If it is identified as a preset structured task instruction, it will be directly parsed into several sub-tasks according to the predefined task flow in the preset structured task instruction; The subtasks obtained from disassembly or parsing are arranged in the order of execution to generate the subtask queue.

[0007] As an optional approach, the step of obtaining the initial inspection path by performing topology adaptation based on the pre-built park semantic map and the sub-task queue is as follows: Extract the spatial points corresponding to each subtask from the subtask queue and construct a point sequence by combining it with the current position of the unmanned vehicle; Starting from adjacent points in the point sequence, search for connected paths in the road topology network of the park semantic map, and then splice the searched connected paths end to end according to the order of the point sequence to obtain the initial inspection path.

[0008] As an optional solution, the multimodal environmental perception data includes at least two of the following: visual image data collected by an onboard camera, point cloud data collected by a lidar, and radar echo data collected by a millimeter-wave radar.

[0009] As an optional approach, the steps of dividing the unfinished subtasks into a first task branch for handling the sudden abnormal event and a second task branch for continuing the execution of the unfinished inspection task, based on the task correlation between each unfinished subtask and the sudden abnormal event, are as follows: Obtain the spatial location and task type corresponding to each unfinished subtask, as well as the spatial location and anomaly type of the sudden abnormal event; Calculate the spatial distance between the spatial location of each unfinished subtask and the spatial location of the sudden abnormal event, and the type matching degree between the task type of each unfinished subtask and the abnormal type of the sudden abnormal event. Based on the spatial distance and the type matching degree, determine the task correlation degree between each unfinished subtask and the sudden abnormal event. Unfinished subtasks with a task relevance exceeding a preset threshold are assigned to the first task branch, while unfinished subtasks with a task relevance not exceeding the preset threshold are assigned to the second task branch.

[0010] As an optional approach, the step of sorting the first task branch and the second task branch according to the execution priority to generate a task execution sequence is as follows: Obtain the anomaly level attribute of the sudden abnormal event, and positively map the execution priority of the anomaly handling subtask in the first task branch with the level of the anomaly level attribute. The execution priority is determined by the original order of the unfinished subtasks in the subtask queue in the second task branch, wherein the execution priority of any subtask in the first task branch is higher than the execution priority of any subtask in the second task branch. The subtasks in the first task branch and the second task branch are arranged in descending order of execution priority to generate the task execution sequence.

[0011] As an optional approach, the steps of using the park semantic map to perform path replanning and generate the target planned path based on the task execution sequence and the real-time location of the unmanned vehicle are as follows: Construct a target point sequence with the current position of the unmanned vehicle as the starting point and the spatial points corresponding to each sub-task in the task execution sequence as the path points; In the road topology network of the semantic map of the park, the shortest connected path between adjacent points in the target point sequence is searched and spliced ​​in order. Duplicate road segments in the spliced ​​path are removed and reconnected, and the resulting path is used as the target planning path.

[0012] Secondly, an embodiment of the present invention also provides a technical solution: an autonomous decision-making system for unmanned vehicles based on edge-cloud collaboration, applicable to the autonomous decision-making method for unmanned vehicles based on edge-cloud collaboration as described in the first aspect, comprising: The system includes a perception and execution subsystem deployed on the autonomous vehicle, an intelligent decision-making subsystem deployed on edge computing nodes, and a scheduling and management subsystem deployed in the cloud. The perception and execution subsystem is used to collect multimodal environmental perception data and respond to the planned path to control the autonomous vehicle's driving. The intelligent decision-making subsystem is used to parse the inspection task instructions to generate a sub-task queue and plan an initial inspection path, as well as to identify sudden abnormal events based on the multimodal environmental perception data, and divide the unfinished sub-tasks into a first task branch and a second task branch according to the sudden abnormal events, and then sort them to generate a task execution sequence. Based on the task execution sequence and the real-time position of the unmanned vehicle, the system performs path replanning to generate a target planned path. The scheduling and management subsystem is used to issue inspection task instructions and receive reported abnormal event information and unmanned vehicle operation status information.

[0013] As an optional solution, the intelligent decision-making subsystem includes: The task parsing module is used to perform semantic parsing and task deconstruction on the inspection task instructions to generate a sub-task queue; An anomaly perception module is used to generate sudden anomaly events carrying anomaly level attributes and spatial location attributes based on the multimodal environmental perception data. The task decision module is used to divide the unfinished subtasks into a first task branch and a second task branch according to the task correlation between the unfinished subtasks and the sudden abnormal event, and to sort and generate a task execution sequence. The path planning module is used to plan the initial inspection path and generate the target planned path by replanning the path based on the task execution sequence and the real-time position of the unmanned vehicle.

[0014] As an optional solution, a lightweight visual language large model is also included in the task parsing module, which is used to parse natural language scheduling instructions into structured task descriptions and split them into sub-task queues with temporal dependencies.

[0015] The present invention offers at least the following substantial benefits: The technical solution provided in this application, through semantic parsing and task deconstruction of inspection task instructions, can transform abstract scheduling intentions into a queue of executable sub-tasks. Combined with a semantic map of the park, it generates an initial path, achieving end-to-end mapping from natural language to physical actions and lowering the threshold for human-computer interaction. Furthermore, by extracting abnormal features from multimodal environmental perception data in real time, and upon identifying sudden abnormal events, the remaining tasks are dynamically divided into a first task branch for handling the abnormality and a second task branch for continuing the inspection based on task relevance. Then, based on the abnormality level attribute, the tasks of each branch are prioritized and their paths are replanned. This breaks the limitation of traditional unmanned vehicles that can only passively execute according to preset routes, enabling the vehicle to possess human-like perception-decision-action closed-loop capabilities. Under the premise of ensuring that core risks are handled first, it can automatically recover and complete the remaining inspection tasks, significantly improving the autonomous operation efficiency and emergency response capabilities of unmanned vehicles in complex dynamic environments. Meanwhile, the edge-cloud collaborative architecture ensures the organic combination of real-time inference of lightweight large models at the edge and global scheduling in the cloud, taking into account both the timeliness of decision-making and the scalability of the system.

[0016] The above description of the invention is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0017] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. The drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0018] Figure 1 This is a flowchart of the autonomous decision-making method for unmanned vehicles based on edge-cloud collaboration in Embodiment 1 of the present invention.

[0019] Figure 2 This is a block diagram of the autonomous decision-making system for unmanned vehicles based on edge-cloud collaboration, according to Embodiment 2 of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only one preferred embodiment of this invention and are only used to explain this invention. They do not limit the scope of protection of this invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0021] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations (or steps) can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it may also have additional steps not included in the figures; the process may correspond to a method, function, procedure, subroutine, subroutine, etc.

[0022] Example 1: As Figure 1 As shown in the figure, the autonomous decision-making method for unmanned vehicles based on edge-cloud collaboration provided in this embodiment of the invention aims to solve the problems of rigid task execution and lack of autonomous adaptability of traditional unmanned vehicles in complex dynamic environments. Specifically, it includes steps S1 to S5.

[0023] Step S100: In response to the inspection task instruction, the inspection task instruction is semantically parsed and the task is deconstructed to obtain a subtask queue containing several subtasks. The initial inspection path is obtained by topology adaptation based on the pre-built park semantic map and the subtask queue.

[0024] Specifically, this embodiment is executed by an edge computing node or cloud server deployed on the autonomous vehicle, depending on the source and complexity of the instruction. When the edge computing node receives the inspection task instruction, it first performs semantic parsing and task deconstruction on the instruction. As an optional implementation, the specific process of performing semantic parsing and task deconstruction on the inspection task instruction to obtain a subtask queue containing several subtasks includes: obtaining the inspection task instruction and identifying its instruction type, which includes natural language scheduling instructions and preset structured task instructions.

[0025] In practical applications, pre-set structured task instructions usually refer to standardized JSON or XML format task packages that are pre-configured in the scheduling management subsystem, such as standard inspection plans containing fixed coordinate points and pre-set action codes; while natural language scheduling instructions refer to spoken commands input by dispatchers through voice or text, such as checking the gathering of people at the west perimeter wall or checking the vehicle information of weighbridge No. 3.

[0026] Furthermore, if the instruction is identified as a natural language scheduling instruction, a lightweight visual language model deployed on edge computing nodes is invoked to perform semantic understanding and intent extraction on the natural language scheduling instruction to obtain semantic elements. Based on these elements, the natural language scheduling instruction is decomposed into several sub-tasks with temporal dependencies. The semantic elements include action type, target object, and spatial range.

[0027] Specifically, the lightweight Visual Language Model (VLM), after fine-tuning and quantization for park inspection scenarios, can run in real time at the edge. For example, when the input command is "go to the west perimeter wall to check for crowd gathering," the VLM model extracts the action type as "check / move," the target object as "crowd gathering state," and the spatial range element as "west perimeter wall area" through an attention mechanism. Subsequently, the edge computing node maps and decomposes this semantic element into a sequence of subtasks with strict temporal dependencies based on a pre-set task template library: subtask 1 is "navigate to the nearest access point at the west perimeter wall," subtask 2 is "cruise at low speed along the wall path," subtask 3 is "start the crowd gathering detection algorithm and record video," and subtask 4 is "report the check results." This process of transforming unstructured natural language into a structured temporal sequence of subtasks enables autonomous vehicles to understand vague human intentions and translate them into machine-executable operations, significantly reducing the threshold for human-computer interaction. It should be understood that although this example uses the verification of personnel gatherings, in other embodiments, this parsing logic is also applicable to scenarios such as equipment failure troubleshooting and temporary material transportation, as long as the above three elements can be extracted from natural language.

[0028] Furthermore, if the instruction is identified as a pre-defined structured task instruction, it is directly parsed into several sub-tasks according to the predefined task flow in the pre-defined structured task instruction. This ensures both the efficiency and determinism of routine automated inspection operations, while avoiding the uncertainty risks that may arise from large-scale model inference. Finally, the sub-tasks obtained from the decomposition or parsing are arranged in the order of execution to generate a sub-task queue.

[0029] Furthermore, after acquiring the subtask queue, the system performs topology adaptation based on the pre-built park semantic map and the subtask queue to obtain the initial inspection path. Specifically, this includes: extracting the spatial points corresponding to each subtask from the subtask queue and constructing a point sequence based on the current location of the autonomous vehicle; using adjacent points in the point sequence as starting points, searching for connected paths in the road topology network of the park semantic map, and then concatenating the searched connected paths end-to-end according to the order of the point sequence to obtain the initial inspection path. Specifically, the park semantic map contains not only geometric road information but also topological information such as lane attributes, traffic rules, and point of interest (POI) semantic labels. Edge computing nodes use graph search algorithms such as A* or Dijkstra to calculate the optimal connected path between adjacent points in the road topology network. For example, if the subtask queue contains [current location P0, west entrance P1, weighbridge P2], the system will search for path segments from P0 to P1 and from P1 to P2 respectively, and then smoothly concatenate them.

[0030] It is important to note that the construction process of the park's semantic map includes two stages: initial mapping and incremental updates. In the initial mapping stage, autonomous vehicles equipped with LiDAR and multi-view cameras conduct a full-coverage scan along all accessible roads within the park, collecting point cloud data and visual images. These are then stitched together using a SLAM algorithm to generate a globally consistent 3D road base map. Subsequently, dispatchers configure lane attributes, traffic rules, and point-of-interest (POI) semantic tags for the road network through a visual editing interface, forming a park semantic map containing geometric, topological, and semantic layers. In the incremental update stage, autonomous vehicles continuously perceive environmental changes during routine inspections. When new roads, temporary barriers, or other inconsistencies with the existing map are detected, edge computing nodes upload the data of the changed areas to the cloud for verification and map updates. The incremental update package is then distributed to each vehicle to synchronize the local map. It should be understood that manual semantic annotation in the initial mapping can also be assisted by an automated semantic segmentation model, and then verified and corrected by dispatchers.

[0031] Step S200: Extract abnormal features from the multimodal environmental perception data on the initial inspection path in real time to determine sudden abnormal events that characterize the abnormality level attribute and spatial location attribute.

[0032] As the autonomous vehicle travels along the initial inspection path, the onboard perception and execution subsystem continuously collects environmental data. As an optional implementation, the multimodal environmental perception data includes at least two of the following: visual image data collected by onboard cameras, point cloud data collected by lidar, and radar echo data collected by millimeter-wave radar.

[0033] Specifically, the anomaly perception module running on the edge computing nodes performs fusion analysis on the aforementioned multimodal data. For example, in a patrol road scenario, when the visual image identifies a human figure in the fenced area, and the lidar point cloud detects a vertical obstacle in that area, the system determines it as an "illegal entry by personnel" event through spatiotemporal alignment and confidence fusion, and assigns it a "high-risk" anomaly level attribute and precise three-dimensional spatial coordinates according to preset rules. Similarly, in a weighbridge inspection scenario, if the millimeter-wave radar detects a vehicle lingering beyond the designated time in the weighbridge area, and the visual system does not identify any personnel, it is determined as an "abnormal operation" event and assigned a "medium-risk" level. This multimodal fusion perception mechanism effectively overcomes the limitations of a single sensor under conditions such as changes in lighting and occlusion, improving the accuracy and robustness of anomaly event identification. It should be understood that the anomaly level classification is not limited to high / medium / low levels; continuous numerical scoring or other grading systems can also be used, as long as they can characterize the urgency of the event.

[0034] In step S300, in response to the identified sudden abnormal event, unfinished subtasks are extracted from the subtask queue. Based on the task correlation between each unfinished subtask and the sudden abnormal event, the unfinished subtasks are divided into a first task branch for handling the sudden abnormal event and a second task branch for continuing to execute the unfinished inspection task.

[0035] When an edge computing node confirms a sudden abnormal event, it immediately triggers a task replanning process. This includes: obtaining the spatial location and task type of each incomplete subtask, as well as the spatial location and abnormality type of the sudden abnormal event; calculating the spatial distance between the spatial location of each incomplete subtask and the spatial location of the sudden abnormal event, and the type matching degree between the task type of each incomplete subtask and the abnormality type of the sudden abnormal event; determining the task correlation degree between each incomplete subtask and the sudden abnormal event based on the spatial distance and type matching degree; assigning incomplete subtasks with a task correlation degree exceeding a preset threshold to the first task branch, and assigning incomplete subtasks with a task correlation degree not exceeding the preset threshold to the second task branch.

[0036] Specifically, the formula for calculating the task relevance R is: ;in, This represents the Euclidean distance or road network distance between the target location of the unfinished sub-task and the spatial location of the sudden abnormal event. This is a preset distance normalization factor used to adjust the sensitivity to distance attenuation. Spatial distance weighting coefficient; and These represent the subtask type and the exception event type, respectively. This is a type matching function. It takes a value of 1 when both belong to the same business domain (such as both being security-related or both being device-related), and otherwise takes a value of 0 or a lower base value. This is the type matching weight coefficient. For example, when a "fence damage" event occurs (anomaly type of security facility), for the incomplete "fence integrity inspection" sub-task (task type of security facility), its type matching degree... Furthermore, if the spatial distance is relatively short, the calculated correlation degree... Extremely high, exceeding a preset threshold (e.g., 0.7), this subtask is assigned to the first task branch, used to go to the damage point for close-up photographic evidence collection; while for the incomplete "Inspection of Weighbridge No. 3" subtask (task type is inspection), its type matching degree... Even if the spatial distance is not far, the degree of correlation It is also below the threshold, so it is assigned to the second task branch and awaits further execution. Through this quantitative calculation mechanism, the system can automatically and objectively select the handling tasks most relevant to the current anomaly, avoiding the one-sidedness of manual intervention or simple rule matching.

[0037] Step S400: Determine the execution priority of each subtask based on the anomaly level attribute of the sudden abnormal event, sort the first task branch and the second task branch according to the execution priority, and generate the task execution sequence.

[0038] As an optional implementation, S400 specifically includes: obtaining the anomaly level attribute of the sudden abnormal event; positively mapping the execution priority of the anomaly handling subtasks in the first task branch to the high and low of the anomaly level attribute; using the original arrangement order of each unfinished subtask in the subtask queue in the second task branch as the execution priority, wherein the execution priority of any subtask in the first task branch is higher than the execution priority of any subtask in the second task branch; and uniformly arranging each subtask in the first task branch and the second task branch according to the execution priority from high to low to generate a task execution sequence.

[0039] Specifically, the aforementioned prioritization rules establish the principle of absolute priority for handling anomalies. For example, if a sudden anomaly involves the intrusion of personnel at a "high-risk" level, subtasks such as "on-site expulsion" and "tracking and monitoring" in the first task branch will be assigned the highest priority sequence (e.g., Prio 1, 2); while the original subtasks in the second task branch, such as "road patrol" and "equipment reading," regardless of their original order, will have their priority suppressed after the first branch (e.g., Prio 3, 4...). This mechanism ensures that, under conditions of limited resources or time constraints, autonomous vehicles can unconditionally prioritize responses to high-risk events, preventing the escalation of safety incidents due to mechanically executing the original plan. Simultaneously, preserving the original order within the second branch ensures that, after the anomaly handling is completed, routine inspection tasks can resume execution with minimal logical overhead, maintaining system continuity.

[0040] In step S500, based on the task execution sequence and the real-time location of the unmanned vehicle, the semantic map of the park is invoked to perform path replanning, generate the target planned path, and the unmanned vehicle responds to the target planned path to perform the inspection task.

[0041] As an optional implementation, step S500 specifically includes: constructing a target point sequence with the current position of the unmanned vehicle as the starting point and the spatial points corresponding to each sub-task in the task execution sequence as the waypoints; searching for the shortest connected paths between adjacent points in the target point sequence in the road topology network of the park semantic map and splicing them in order; and performing elimination and reconnection processing on the duplicate road segments in the spliced ​​path, and using the obtained path as the target planning path.

[0042] Specifically, since the task execution sequence already includes newly inserted anomaly handling points and existing remaining inspection points, and the order has been rearranged, a globally optimal path needs to be regenerated. Edge computing nodes start with the vehicle's current real-time pose and connect to the rearranged task points sequentially. During path stitching, the system introduces a duplicate route removal algorithm: if a section overlaps with the original planned route is found during the return trip from the anomaly handling point, the redundant backtracking path is automatically pruned, and the vehicle is directly merged into the original route via a shortcut or the nearest connection point. For example, after a vehicle finishes handling the fence damage in the western area, it does not need to return to its initial starting point to proceed to the next inspection point; instead, it can directly navigate from the western exit to the next nearest inspection point. This dynamic replanning and path optimization mechanism not only ensures the physical executability of the new task sequence but also minimizes unnecessary mileage and improves overall inspection efficiency.

[0043] Step S500: Based on the task execution sequence and the real-time location of the unmanned vehicle, the park semantic map is invoked for path replanning to generate a target planned path. The unmanned vehicle responds to the target planned path to execute the inspection task. Option 1, as an optional implementation, specifically includes: constructing a target point sequence starting from the current location of the unmanned vehicle and using the spatial points corresponding to each sub-task in the task execution sequence as waypoints; searching for the shortest connected paths between adjacent points in the target point sequence in the road topology network of the park semantic map and concatenating them in order; removing and reconnecting duplicate road segments in the concatenated path; and using the resulting path as the target planned path. Option 7: Specifically, since the task execution sequence already includes newly inserted anomaly handling points and existing remaining inspection points, and the order has been rearranged, a globally optimal path needs to be regenerated. Edge computing nodes start with the vehicle's current real-time pose and connect to the rearranged task points sequentially. During the route stitching process, the system specifically introduces a duplicate route removal algorithm: if a route overlapping with the original planned route is found during the return trip from the anomaly handling point, the redundant backtracking path is automatically trimmed, and the route is directly merged into the original route via a shortcut or the nearest connecting point. For example, after a vehicle has dealt with the fence damage in the western area, it does not need to return to the initial starting point to proceed to the next inspection point; instead, it can directly navigate from the western area exit to the next nearest inspection point. This dynamic replanning and route optimization mechanism not only ensures the physical executability of the new task sequence but also minimizes unnecessary mileage and improves overall inspection efficiency.

[0044] Through steps S100 to S500, this embodiment achieves a complete closed loop from natural language instruction understanding, multimodal anomaly perception, dynamic task deconstruction and reorganization, to adaptive path replanning. The entire process relies on an edge-cloud collaborative architecture, pushing the high-computing-power-requirement semantic understanding and decision-making logic down to edge computing nodes, ensuring millisecond-level response speeds while retaining the global scheduling capabilities of the cloud, enabling the autonomous vehicle to truly possess human-like autonomous cognition and adaptive operational capabilities.

[0045] Example 2 like Figure 2 As shown, this embodiment provides an autonomous decision-making system for unmanned vehicles based on edge-cloud collaboration. This system is applicable to the autonomous decision-making method for unmanned vehicles based on edge-cloud collaboration as described in Embodiment 1.

[0046] Specifically, the system's physical architecture is clearly divided into three parts: a perception and execution subsystem deployed on the autonomous vehicle, an intelligent decision-making subsystem deployed on edge computing nodes, and a scheduling and management subsystem deployed in the cloud. This three-layer collaborative architecture, combining edge and cloud, aims to address the limitations of traditional single-layer architectures: compared to a pure cloud architecture, pushing core decision-making logic to edge computing nodes significantly reduces communication latency and ensures autonomous operation in weak network environments; compared to a pure vehicle-side architecture, edge nodes possess stronger computing power and storage resources, enabling real-time inference and complex decision-making for lightweight large models, while retaining the cloud's global scheduling and data review capabilities, thus balancing the system's real-time responsiveness and global control.

[0047] The perception and execution subsystem is mainly used to collect multimodal environmental perception data and respond to plan the path to control the autonomous vehicle's driving.

[0048] Specifically, this subsystem integrates various sensing devices such as vehicle-mounted cameras, LiDAR, millimeter-wave radar, infrared thermal imaging, and acoustic sensors. It can collect multimodal raw data from the park environment, including visual images, point clouds, radar echoes, and audio, around the clock, and transmit this data in real-time to edge computing nodes via high-speed vehicle Ethernet or a dedicated data bus. Simultaneously, the subsystem includes a chassis motion controller and a gimbal attitude controller, which receive the target planning path and control commands generated by the intelligent decision-making subsystem and convert them into low-level execution signals such as motor speed, steering angle, and gimbal pitch / rotation angle, enabling precise navigation and proactive interaction of the autonomous vehicle. It should be understood that the perception and execution subsystem is not only the data acquisition end but also the physical execution end of the decision results. Through a closed-loop feedback mechanism, it transmits real-time vehicle position, speed, battery level, and other status information back to the intelligent decision-making subsystem, providing real-time data for dynamic replanning.

[0049] The intelligent decision-making subsystem is deployed on edge computing nodes to parse inspection task instructions to generate a sub-task queue and plan the initial inspection path. It also identifies sudden abnormal events based on multimodal environmental perception data, divides unfinished sub-tasks into first and second task branches based on the sudden abnormal events, and sorts them to generate a task execution sequence. Finally, it performs path replanning based on the task execution sequence and the real-time position of the unmanned vehicle to generate the target planned path.

[0050] As an optional implementation, the subsystem can be implemented using an edge computing box or an in-vehicle industrial control computer equipped with a high-performance AI acceleration chip to ensure sufficient inference computing power with limited power consumption.

[0051] In terms of functional logic, the intelligent decision-making subsystem further includes a task parsing module, an anomaly perception module, a task decision-making module, and a path planning module. The task parsing module is used to perform semantic parsing and task deconstruction on inspection task instructions to generate a sub-task queue. Its specific execution process corresponds to step S100 in Embodiment 1. That is, it receives instructions from the scheduling management subsystem or the local human-machine interface, and transforms the abstract scheduling intent into a machine-executable sequence of atomic operations through a built-in natural language processing algorithm or structured parser.

[0052] The anomaly perception module is used to generate sudden anomaly events carrying anomaly level attributes and spatial location attributes based on multimodal environmental perception data. Its specific execution process corresponds to step S200 in Embodiment 1. That is, it performs spatiotemporal alignment and fusion analysis on the multi-source heterogeneous data uploaded by the perception execution subsystem, uses a pre-trained deep learning model to identify specific events such as personnel intrusion, fence damage, and equipment malfunction, and outputs standardized anomaly event objects for downstream modules to call.

[0053] The task decision module is used to divide unfinished subtasks into first and second task branches based on the task relevance between unfinished subtasks and sudden abnormal events, and to generate a task execution sequence in order. Its specific execution process corresponds to steps S300 and S400 in Implementation Example 1. Internally, it maintains a task relevance calculation model and priority sorting rules, which can complete the dynamic reorganization of remaining tasks within milliseconds, ensuring that high-risk events are handled with priority.

[0054] The path planning module is used to plan the initial inspection path and generate the target planned path by replanning the path based on the task execution sequence and the real-time position of the unmanned vehicle. Its specific execution process corresponds to steps S100 and S500 in Example 1. This module loads the semantic map of the park, uses a graph search algorithm to calculate the optimal connected path, and performs duplicate segment removal and smoothing processing, finally outputting a path point sequence that can be directly executed by the chassis controller.

[0055] Through the modular design described above, the intelligent decision-making subsystem achieves a complete closed loop from perception to decision-making to planning, and the modules interact with each other through standardized data interfaces, which facilitates subsequent functional expansion and algorithm iteration.

[0056] The scheduling and management subsystem is deployed on a cloud server and is used to issue inspection task instructions and receive reported abnormal event information and unmanned vehicle operation status information.

[0057] Specifically, this subsystem provides a visual command interface for dispatchers, supporting various task issuance methods such as natural language voice input and mouse clicks. User commands are transmitted to the edge computing nodes of designated vehicles via network communication links (such as 4G / 5G / Wi-Fi). Simultaneously, the subsystem continuously receives details of abnormal events, handling results, real-time trajectories, and health status reported by each vehicle, used for global situational awareness, historical data review, and multi-vehicle collaborative dispatch. For example, when multiple alarms occur simultaneously within the park, the dispatch management subsystem can automatically allocate the nearest and available vehicle to handle the situation based on the real-time location and task load of each vehicle, avoiding task conflicts and resource waste.

[0058] It should be understood that although the scheduling and management subsystem in this embodiment is deployed on a public cloud or private cloud server, in other embodiments it can also be deployed on a local edge server in the park, as long as it can achieve global information aggregation and instruction distribution across vehicles.

[0059] To address the localization requirements of natural language instructions, this embodiment also includes a lightweight visual language model deployed in the task parsing module, which is used to parse natural language scheduling instructions into structured task descriptions and split them into sub-task queues with temporal dependencies.

[0060] Specifically, to ensure the real-time inference performance of the large model on edge computing nodes, this lightweight visual language model has undergone targeted quantization compression and operator optimization. For example, using INT8 or FP16 mixed-precision quantization technology, the model size is compressed to less than 1 / 4 of its original size while maintaining the model's semantic understanding accuracy, and memory usage is reduced by more than 60%. At the same time, it is adapted to edge-specific AI inference engines (such as TensorRT, ONNX Runtime, etc.), making full use of the GPU or NPU acceleration units of edge computing nodes to achieve a single inference time of less than 200 milliseconds. This edge-side deployment allows autonomous vehicles to complete the end-to-end conversion from "checking the gathering of people at the western perimeter wall" to the sub-task sequence of "navigation-cruising-detection-reporting" without relying on cloud APIs. This not only eliminates the impact of network fluctuations on instruction parsing but also avoids the privacy and security risks of uploading sensitive voice / image data to the cloud, truly realizing the localization, real-time operation, and autonomy of embodied intelligence.

[0061] Furthermore, the lightweight visual language large model operates as follows in this embodiment. The model consists of a visual encoder, a large language model backbone network, and a cross-modal projection layer. The visual encoder maps the input image into a sequence of visual features, and the cross-modal projection layer aligns the visual features to the embedding space of the language model, participating in attention calculation together with the text token.

[0062] During the inference phase, when an edge computing node receives a natural language scheduling instruction, it first segments the instruction into a sub-word-level token sequence using a token segmenter, and then maps it into token embedding vectors via an embedding layer. Subsequently, the token embedding vectors, along with optional visual feature prefixes, are input into the multi-layer Transformer decoder of the large language model backbone network. Each decoder layer captures the contextual dependencies between generated tokens through a masked self-attention mechanism and fuses visual feature information through a cross-attention mechanism, progressively refining high-level representations containing semantic understanding. The model predicts output tokens one by one in an autoregressive manner until a complete structured output sequence is generated. This structured output sequence follows a predefined JSON format specification, containing semantic element fields such as action type, target object, and spatial range. The post-processing parser in the task parsing module performs format validation and field extraction, ultimately transforming it into a structured subtask description that can be consumed by downstream modules.

[0063] To adapt the general pre-trained model to the specific semantics of park inspection scenarios, this embodiment performs domain-specific fine-tuning on the lightweight visual language model. Specifically, a dataset containing instruction-response pairs for park inspection scenarios is constructed. Each data point consists of a natural language scheduling instruction and its corresponding standard structured task description, covering typical business scenarios such as perimeter patrol, weighing inspection, personnel verification, and equipment inspection. The backbone network of the large language model is efficiently fine-tuned on this dataset, employing a low-rank adaptive method to update only a small number of key parameters in the model. This maintains the general semantic understanding capability while endowing the model with the ability to accurately parse park-specific terminology and business logic. For example, the fine-tuned model can accurately understand the semantic connotations of domain-specific verbs such as "verification," "inspection," and "weighing," and map them to their corresponding standard action type codes.

[0064] In terms of edge-side inference optimization, in addition to the aforementioned mixed-precision quantization, this embodiment also introduces a key-value cache management mechanism and an operator fusion strategy. The key-value cache management mechanism caches multi-head attention key-value pairs already computed during inference, avoiding repeated computation of historical tokens in the autoregressive generation stage, reducing the computational complexity of the generation stage from the square of the sequence length to linear. The operator fusion strategy merges multiple fine-grained operators in the model computation graph into coarse-grained fusion operators, reducing memory read / write overhead and kernel startup latency. These optimization methods, combined with quantization compression, significantly reduce the end-to-end inference time of the lightweight visual language model on edge computing nodes, meeting the latency requirements for real-time decision-making in autonomous vehicles. It should be understood that the components of the above operating mechanism are not the only implementation methods. For example, in other embodiments, the visual encoder can be replaced with a convolutional neural network structure, and domain fine-tuning can adopt alternative solutions such as full-parameter fine-tuning or cue-based fine-tuning, as long as the end-to-end conversion from natural language scheduling instructions to structured subtask descriptions can be achieved.

[0065] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the specific device can be divided into different functional modules to complete all or part of the functions described above.

[0066] In the embodiments provided in this application, it should be understood that the disclosed structures and methods can be implemented in other ways. For example, the structural embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another structure, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between structures or units, and may be electrical, mechanical, or other forms.

[0067] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0068] Furthermore, in the embodiments of this application, the functional units can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0069] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0070] The specific embodiments described above are preferred embodiments of the autonomous decision-making method and system for unmanned vehicles based on edge-cloud collaboration of the present invention, and are not intended to limit the specific scope of the present invention. The scope of the present invention includes but is not limited to the specific embodiments described above. All equivalent changes made in accordance with the shape and structure of the present invention are within the protection scope of the present invention.

Claims

1. An autonomous decision-making method for unmanned vehicles based on edge-cloud collaboration, characterized in that, Includes the following steps: In response to the inspection task instruction, the system performs semantic parsing and task deconstruction on the inspection task instruction to obtain a subtask queue containing several subtasks. Based on the pre-built park semantic map, the system performs topology adaptation with the subtask queue to obtain the initial inspection path. Real-time extraction of abnormal features from multimodal environmental perception data along the initial inspection path to identify sudden abnormal events that characterize the anomaly level and spatial location attributes; In response to the identified sudden abnormal event, unfinished subtasks are extracted from the subtask queue. Based on the task correlation between each unfinished subtask and the sudden abnormal event, the unfinished subtasks are divided into a first task branch for handling the sudden abnormal event and a second task branch for continuing to execute the unfinished inspection task. The execution priority of each subtask is determined based on the anomaly level attribute of the sudden abnormal event. The first task branch and the second task branch are sorted according to the execution priority to generate a task execution sequence. Based on the task execution sequence and the real-time location of the unmanned vehicle, the park semantic map is invoked to perform path replanning and generate a target planned path. The unmanned vehicle then responds to the target planned path to perform inspection tasks.

2. The autonomous decision-making method for unmanned vehicles based on edge-cloud collaboration according to claim 1, characterized in that: The steps for semantic parsing and task deconstructing the inspection task instructions to obtain a subtask queue containing several subtasks are as follows: The inspection task instructions are obtained and their types are identified. The instruction types include natural language scheduling instructions and preset structured task instructions. If identified as a natural language scheduling instruction, a lightweight visual language model deployed on edge computing nodes is invoked to perform semantic understanding and intent extraction on the natural language scheduling instruction to obtain semantic elements. Based on these elements, the natural language scheduling instruction is decomposed into several sub-tasks with temporal dependencies. The semantic elements include action type, target object, and spatial range. If it is identified as a preset structured task instruction, it will be directly parsed into several sub-tasks according to the predefined task flow in the preset structured task instruction; The subtasks obtained from disassembly or parsing are arranged in the order of execution to generate the subtask queue.

3. The autonomous decision-making method for unmanned vehicles based on edge-cloud collaboration according to claim 2, characterized in that: The steps for obtaining the initial inspection path by performing topology adaptation based on the pre-built park semantic map and the sub-task queue are as follows: Extract the spatial points corresponding to each subtask from the subtask queue and construct a point sequence by combining it with the current position of the unmanned vehicle; Starting from adjacent points in the point sequence, search for connected paths in the road topology network of the park semantic map, and then splice the searched connected paths end to end according to the order of the point sequence to obtain the initial inspection path.

4. The autonomous decision-making method for unmanned vehicles based on edge-cloud collaboration according to claim 1, characterized in that: The multimodal environmental perception data includes at least two of the following: visual image data collected by an onboard camera, point cloud data collected by a lidar, and radar echo data collected by a millimeter-wave radar.

5. The autonomous decision-making method for unmanned vehicles based on edge-cloud collaboration according to claim 1, characterized in that: The steps for dividing the unfinished subtasks into a first task branch for handling the sudden abnormal event and a second task branch for continuing the execution of the unfinished inspection task, based on the task correlation between each unfinished subtask and the sudden abnormal event, are as follows: Obtain the spatial location and task type corresponding to each unfinished subtask, as well as the spatial location and anomaly type of the sudden abnormal event; Calculate the spatial distance between the spatial location of each unfinished subtask and the spatial location of the sudden abnormal event, and the type matching degree between the task type of each unfinished subtask and the abnormal type of the sudden abnormal event. Based on the spatial distance and the type matching degree, determine the task correlation degree between each unfinished subtask and the sudden abnormal event. Unfinished subtasks with a task relevance exceeding a preset threshold are assigned to the first task branch, while unfinished subtasks with a task relevance not exceeding the preset threshold are assigned to the second task branch.

6. The autonomous decision-making method for unmanned vehicles based on edge-cloud collaboration according to claim 1, characterized in that: The steps for sorting the first task branch and the second task branch according to the execution priority to generate a task execution sequence are as follows: Obtain the anomaly level attribute of the sudden abnormal event, and positively map the execution priority of the anomaly handling subtask in the first task branch with the level of the anomaly level attribute. The execution priority is determined by the original order of the unfinished subtasks in the subtask queue in the second task branch, wherein the execution priority of any subtask in the first task branch is higher than the execution priority of any subtask in the second task branch. The subtasks in the first task branch and the second task branch are arranged in descending order of execution priority to generate the task execution sequence.

7. The autonomous decision-making method for unmanned vehicles based on edge-cloud collaboration according to claim 1, characterized in that: The steps for generating the target planned path by calling the semantic map of the park based on the task execution sequence and the real-time location of the unmanned vehicle are as follows: Construct a target point sequence with the current position of the unmanned vehicle as the starting point and the spatial points corresponding to each sub-task in the task execution sequence as the path points; In the road topology network of the semantic map of the park, the shortest connected path between adjacent points in the target point sequence is searched and spliced ​​in order. Duplicate road segments in the spliced ​​path are removed and reconnected, and the resulting path is used as the target planning path.

8. An autonomous decision-making system for unmanned vehicles based on edge-cloud collaboration, applicable to the autonomous decision-making method for unmanned vehicles based on edge-cloud collaboration as described in any one of claims 1 to 7, characterized in that: include: The system includes a perception and execution subsystem deployed on the autonomous vehicle, an intelligent decision-making subsystem deployed on edge computing nodes, and a scheduling and management subsystem deployed in the cloud. The perception and execution subsystem is used to collect multimodal environmental perception data and respond to the planned path to control the autonomous vehicle's driving. The intelligent decision-making subsystem is used to parse the inspection task instructions to generate a sub-task queue and plan an initial inspection path, as well as to identify sudden abnormal events based on the multimodal environmental perception data, and divide the unfinished sub-tasks into a first task branch and a second task branch according to the sudden abnormal events, and then sort them to generate a task execution sequence. Based on the task execution sequence and the real-time position of the unmanned vehicle, the system performs path replanning to generate a target planned path. The scheduling and management subsystem is used to issue inspection task instructions and receive reported abnormal event information and unmanned vehicle operation status information.

9. The autonomous decision-making system for unmanned vehicles based on edge-cloud collaboration according to claim 8, characterized in that, The intelligent decision-making subsystem includes: The task parsing module is used to perform semantic parsing and task deconstruction on the inspection task instructions to generate a sub-task queue; An anomaly perception module is used to generate sudden anomaly events carrying anomaly level attributes and spatial location attributes based on the multimodal environmental perception data. The task decision module is used to divide the unfinished subtasks into a first task branch and a second task branch according to the task correlation between the unfinished subtasks and the sudden abnormal event, and to sort and generate a task execution sequence. The path planning module is used to plan the initial inspection path and generate the target planned path by replanning the path based on the task execution sequence and the real-time position of the unmanned vehicle.

10. The autonomous decision-making system for unmanned vehicles based on edge-cloud collaboration according to claim 9, characterized in that, It also includes a lightweight visual language large model deployed in the task parsing module, used to parse natural language scheduling instructions into structured task descriptions and split them into sub-task queues with temporal dependencies.

Images