Unmanned aerial vehicle highway intelligent inspection method and system based on patrol requirements

Abstract

The application discloses a method and system for intelligent highway inspection based on patrol requirements, and belongs to the fields of unmanned aerial vehicles, intelligent transportation systems and artificial intelligence. The method comprises the following steps: obtaining natural language instructions to perform semantic analysis and generate a structured task vector; constructing and generating an adaptive inspection strategy based on the task vector and by fusing real-time data; distributing the strategy to unmanned aerial vehicles for autonomous inspection and identifying and generating classified event alarms; and finally responding to the alarms and triggering a closed-loop inspection task report. The application adopts a technical path combining multi-level reasoning based on a large language model and dynamic behavior strategy synthesis, and through the construction of a complete cognitive and action framework from semantic intention understanding, behavior decision-making to closed-loop response, it can realize autonomous generation and intelligent execution of complex and fuzzy inspection tasks, and significantly improve the automation level, response speed and decision intelligence of highway inspection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicles, intelligent transportation systems and artificial intelligence, and in particular to an intelligent highway inspection method and system for unmanned aerial vehicles based on patrol requirements. BACKGROUND

[0002] Highways are the main arteries of national transportation, and their safety and smoothness are of great importance. Unmanned aerial vehicle inspection, as a new and alternative technology to traditional manual inspection, is increasingly used in remote and dynamic monitoring of road conditions, traffic incidents and ancillary facilities, which is of great significance to road safety and traffic management efficiency.

[0003] However, existing unmanned aerial vehicle inspection systems rely on preset fixed routes or simple trigger rules, making it difficult to respond flexibly to real-time, diverse and even ambiguous patrol requirements. In addition, the links of task planning, event detection and emergency disposal are usually fragmented, lacking an intelligent framework that can dynamically and integrally generate and dispatch patrol strategies, detection algorithms and response actions according to patrol intentions. SUMMARY

[0004] To solve the above problems, the present application provides an intelligent highway inspection method and system for unmanned aerial vehicles based on patrol requirements, which adopts a technical path combining multi-level reasoning based on large language models and dynamic behavior strategy synthesis. By constructing a complete cognitive and action framework from semantic intention understanding, behavior decision-making to closed-loop response, autonomous generation and intelligent execution of complex and ambiguous patrol tasks can be achieved, significantly improving the automation level, response speed and decision intelligence of highway inspection.

[0005] The above objectives can be achieved through the following solutions:

[0006] The intelligent highway inspection method for unmanned aerial vehicles based on patrol requirements includes obtaining natural language instructions of patrol requirements, performing semantic analysis and intention decomposition on the natural language instructions, and generating a structured task vector. Based on the structured task vector and by integrating real-time traffic and environmental perception data for collaborative planning, an adaptive inspection strategy is constructed and generated. The adaptive inspection strategy is distributed to the unmanned aerial vehicle for autonomous inspection, and the multi-modal data obtained by the unmanned aerial vehicle is analyzed in real time to identify and generate classified event alarms. In response to the classified event alarms, the corresponding disposal methods are matched and triggered from a preset multi-level response plan library to generate a closed-loop inspection task report.

[0007] Optionally, the generating the structured task vector comprises: performing multi-path reasoning on the natural language instruction by using a large language model to generate a candidate task hypothesis; obtaining objective data for verifying the candidate task hypothesis from an external real-time data source for the candidate task hypothesis to obtain external calibration evidence; performing posterior probability calculation and weighted fusion on the candidate task hypothesis based on the external calibration evidence to generate a structured task vector.

[0008] Optionally, the constructing and generating the adaptive patrol strategy comprises: constructing and generating a probabilistic task graph based on the patrol target and the probability distribution in the structured task vector; performing real-time strategy calculation based on the probabilistic task graph, the real-time traffic, and the environmental perception data to generate a time-series behavior strategy; and performing path instantiation on the probabilistic task graph according to the time-series behavior strategy to construct and generate an adaptive patrol strategy.

[0009] Optionally, the method further comprises: performing path planning deduction on the probabilistic task graph under the candidate task hypothesis to generate an expected task completion degree index; performing statistical analysis on the expected task completion degree index to calculate and generate a graph confidence index of the probabilistic task graph; and if the graph confidence index is lower than a preset robustness threshold, taking the candidate task hypothesis as a new constraint to iteratively optimize the node transition probability of the probabilistic task graph.

[0010] Optionally, the identifying and generating the classified event alarm comprises: obtaining multi-modal data in a historical normal patrol state to train a data generator model; performing reverse mapping and reconstruction on real-time obtained multi-modal data by using the data generator model to calculate and generate optimal reconstruction data; quantitatively analyzing reconstruction errors between multi-modal data and the optimal reconstruction data, and classifying a region where the reconstruction error exceeds a preset dynamic threshold as an event to identify and generate a classified event alarm.

[0011] Optionally, the calculating and generating the optimal reconstruction data comprises: defining a reconstruction loss function based on a latent space of the data generator model by using a difference between multi-modal data and synthetic data output by the data generator model; performing iterative exploration on the latent space based on a gradient of the reconstruction loss function and coupling a random noise term to generate a latent space state sample; performing statistical moment estimation on the latent space state sample to calculate and determine a first latent space vector, and inputting the first latent space vector into the data generator model for forward propagation to generate optimal reconstruction data.

[0012] Optionally, the generating the closed-loop inspection task report comprises: fusing the classified event alarm and the structured task vector to construct a comprehensive decision context; inputting the comprehensive decision context into a large language model to generate a dynamic response strategy through multi-target reasoning; executing a disposal mode according to the dynamic response strategy, and recording a disposal process to generate a closed-loop inspection task report.

[0013] Optionally, the method further comprises: projecting the timing behavior strategy in the structured task vector to calculate and generate a strategy-intention coverage matrix; performing information entropy analysis on the strategy-intention coverage matrix to quantify and generate a cognitive coordination index; if the cognitive coordination index is lower than a preset coordination threshold, taking the candidate task hypothesis as a penalty term to generate a cognitive bias feedback signal.

[0014] Optionally, the method further comprises: using the cognitive bias feedback signal to dynamically reshape a reward function of the strategy generation network model to generate an updated reward function; using the updated reward function to perform online training on the strategy generation network model to generate an updated strategy generation network model.

[0015] Based on the same inventive concept, the application also provides an intelligent highway inspection system for unmanned aerial vehicles based on patrol requirements, which comprises: a task semantic analysis module for obtaining natural language instructions of patrol requirements, performing semantic analysis and intention decomposition on the natural language instructions, and generating a structured task vector; an adaptive strategy generation module for generating an adaptive inspection strategy based on the structured task vector and fusing real-time traffic and environmental perception data for collaborative planning; an autonomous inspection and event identification module for distributing the adaptive inspection strategy to unmanned aerial vehicles for autonomous inspection, and performing real-time analysis on multi-modal data obtained by the unmanned aerial vehicles to identify and generate classified event alarms; and a closed-loop response and report generation module for responding to the classified event alarms, matching and triggering corresponding disposal modes from a preset multi-level response plan library, and generating a closed-loop inspection task report.

[0016] Compared with the prior art, the application has the following advantages:

[0017] 1. The application uses a large language model to perform deep semantic understanding on natural language instructions and dynamically generates a structured task vector containing multiple possibilities. It upgrades the inspection task from executing a preset path to autonomously synthesizing an adaptive inspection strategy containing a flight path, an algorithm and a behavior logic according to a probabilistic task intention, thereby fundamentally improving the intelligent level and flexibility in dealing with ambiguous and complex tasks;

[0018] 2、The application is not simply matched from the static pre-plan library when responding to the alarm, but fuses the real-time classified event alarm with the initial structured task vector, constructs a comprehensive decision context, and uses a large language model to reason again to generate the optimal disposal strategy. This mechanism makes the emergency response no longer isolated, but deeply coupled with the top-level intention of the task, significantly improving the accuracy and scenario adaptability of the disposal decision.

[0019] Other features and advantages of the present application will be set forth in the following description, and in part will be apparent from the description, or can be learned by practice of the application. The objects and other advantages of the present application will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without any creative effort.

[0021] Figure 1 is a patrol demand based unmanned aerial vehicle expressway intelligent patrol method schematic diagram of the embodiment of the present application.

[0022] Figure 2 is a candidate task hypothesis Bayesian posterior probability updating schematic diagram of the embodiment of the present application.

[0023] Figure 3 is a probabilistic task atlas and optimal behavior strategy instantiation schematic diagram of the embodiment of the present application.

[0024] Figure 4 is an abnormal event classification decision boundary based on reconstruction error of the embodiment of the present application.

[0025] Figure 5 is a patrol demand based unmanned aerial vehicle expressway intelligent patrol system structure schematic diagram of the embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort are within the protection scope of the present application.

[0027] Referring to Figure 1 One embodiment of the present application proposes a high-speed highway intelligent patrol method and system based on patrol requirements, which adopts a technical path combining multi-level reasoning based on large language models and dynamic behavior strategy synthesis. By constructing a complete cognitive and action framework from semantic intention understanding, behavior decision-making to closed-loop response, autonomous generation and intelligent execution of complex and fuzzy patrol tasks can be realized, significantly improving the automation level, response speed and decision intelligence of highway patrol.

[0028] The method of the embodiment specifically includes:

[0029] Obtain natural language instructions of patrol requirements, perform semantic analysis and intention decomposition on the natural language instructions, and generate a structured task vector;

[0030] Based on the structured task vector, and in combination with real-time traffic and environmental perception data, perform collaborative planning, construct and generate an adaptive patrol strategy;

[0031] Distribute the adaptive patrol strategy to the unmanned aerial vehicle for autonomous patrol, and perform real-time analysis on the multi-modal data obtained by the unmanned aerial vehicle, identify and generate a classified event alarm;

[0032] In response to the classified event alarm, match and trigger the corresponding disposal mode from a pre-set multi-level response plan library, and generate a closed-loop patrol task report.

[0033] The technical path combining multi-level reasoning based on large language models and dynamic behavior strategy synthesis is adopted. By constructing a complete cognitive and action framework from semantic intention understanding, behavior decision-making to closed-loop response, autonomous generation and intelligent execution of complex and fuzzy patrol tasks can be realized, significantly improving the automation level, response speed and decision intelligence of highway patrol.

[0034] Optionally, the generation of the structured task vector includes:

[0035] Use a large language model to perform multi-path reasoning on the natural language instructions to generate a candidate task hypothesis;

[0036] Specifically, this step aims to convert the natural language instruction input by the user, which may have ambiguity or ambiguity, into a set of explicit machine-processable alternatives. A large language model (LLM) receives the natural language instruction as input. Through a special prompt engineering technology, the model is guided not to directly output a unique answer, but to carry out multi-path and divergent reasoning to generate a set of logically mutually exclusive candidate task hypotheses. Each candidate task hypothesis is a possible and complete structured interpretation of the user's true intention, which contains key entities such as patrol route, monitoring target, and task priority.

[0037] For the candidate task hypothesis, objective data for verifying the candidate task hypothesis is obtained from an external real-time data source, obtaining external validation evidence;

[0038] Specifically, this step aims to seek data support from the objective world for the multiple subjective hypotheses generated in the previous step. A data verification process will automatically call one or more external Application Programming Interfaces (APIs) for each candidate task hypothesis. For example, if a candidate task hypothesis is "congestion occurs on the Beijing-Shanghai Expressway in Suzhou", the process will automatically call a real-time map service API to obtain the current average speed and traffic flow data of the route; if the assumption is "bad weather", the weather service API will be called to obtain rainfall and wind speed data. These objective data obtained from external real-time data that can confirm or disprove the corresponding hypothesis collectively constitute external validation evidence.

[0039] Based on the external validation evidence, posterior probability calculation and weighted fusion are performed on the candidate task hypotheses to generate a structured task vector.

[0040] Specifically, this step is the core of realizing the probabilistic and intelligent inference of user intent. A Bayesian inference process takes the initial probability of each candidate task hypothesis as the prior probability. Then, using the external validation evidence obtained in the previous step, the posterior probability of each candidate task hypothesis is calculated through a Bayesian updating rule. The core idea of Bayesian updating can be represented by the formula:

[0041] ,

[0042] where, is the posterior probability of the th candidate task hypothesis after observing the external validation evidence E; is its prior probability; is the likelihood of observing the evidence when the hypothesis likelihoods. Finally, by fusing each candidate task hypothesis with its corresponding posterior probability, a final structured task vector is generated, which not only contains the task entity, but also contains the confidence assessment of various possibilities, as shown in Figure 2

[0043] Optionally, the constructing and generating the adaptive patrol strategy comprises:

[0044] constructing and generating a probabilistic task graph based on the patrol targets and probability distribution in the structured task vector;

[0045] Specifically, this step aims to transform the generated structured task vector, which contains probabilistic interpretations of the user's multiple possible intentions, into a graph structure for behavior decision-making. A graph construction process defines each patrol target in the candidate task hypothesis as a node in the graph. The edges between nodes represent the possibility of switching between different hypotheses or transferring between geographically adjacent patrol areas. The process uses the posterior probabilities contained in the structured task vector as the initial weights of the nodes and edges. Through this step, a weighted, directed probabilistic task graph that can fully represent the task possibility space is constructed and generated.

[0046] performing real-time strategy calculation based on the probabilistic task graph, the real-time traffic, and the environmental perception data to generate a time-series behavior strategy;

[0047] Specifically, the purpose of this step is to generate an optimal behavior strategy after considering the uncertainty of the task and the dynamic changes in the real world. A strategy calculation process uses a deep reinforcement learning (DRL) model, such as a policy network. The network takes the topological structure and weights of the probabilistic task graph, as well as real-time traffic and environmental perception data, as input. Its goal is to learn a policy function that can maximize the long-term cumulative reward, which combines multiple optimization objectives such as task completion, flight safety, and energy consumption. Finally, the time-series behavior strategy output by the policy network defines the probability of selecting a certain "atomic behavior" for the UAV when facing any possible state at each future time step.

[0048] According to the time-series behavior strategy, the probabilistic task graph is path-instantiated to construct and generate an adaptive patrol strategy.

[0049] ​Specifically, this step is to transform the abstract behavior policy generated in the previous step into a concrete, executable inspection task plan. A path instantiation process will traverse the probabilistic task graph according to the optimal action sequence given by the temporal behavior policy, thereby determining an optimal inspection path. At the same time, this process will configure specific sensor parameters and intelligent detection algorithms that need to be mounted and activated for each waypoint on the path according to the requirements of the sensor actions in the temporal behavior policy. A complete plan containing this instantiated optimal path and the sensor and algorithm configuration bound to it constitutes the final adaptive inspection strategy, as shown in Figure 3 The figure shows how the application determines an optimal behavior policy path through intelligent decision-making from a task graph containing multiple possibilities.

[0050] Optionally, the method further comprises:

[0051] Path planning deduction is performed on the probabilistic task graph under the candidate task hypothesis to generate an expected task completion degree index;

[0052] Specifically, this step aims to conduct a comprehensive "stress test" on the generated preliminary action framework. A simulation deduction process will traverse each high-probability candidate task hypothesis generated. For each candidate task hypothesis, the process will simulate a complete path planning on the probabilistic task graph, and calculate a quantitative expected task completion degree index according to the goal defined by the hypothesis. The index can be a normalized score that integrates factors such as expected completion time, target coverage rate, and resource consumption.

[0053] Statistical analysis is performed on the expected task completion degree index to calculate and generate a graph confidence index of the probabilistic task graph;

[0054] Specifically, this step aims to quantify the robustness of the preliminary action framework generated in the previous step. A statistical analysis process will calculate the variance or standard deviation of the expected task completion degree index generated in the previous step under all candidate task hypotheses. A lower variance means that the current probabilistic task graph has good adaptability to the user's multiple possible intentions, i.e., strong robustness. Conversely, a higher variance means that the graph is very "fragile" and can only serve the most mainstream hypothesis well, but performs poorly on other possibilities. The final generated graph confidence index is designed to be inversely proportional to the variance.

[0055] If the graph confidence index is lower than the preset robustness threshold, the candidate task hypothesis is used as a new constraint to iteratively optimize the node transition probability of the probabilistic task graph.

[0056] Specifically, this step is the closed-loop process for enhancing the robustness of the task graph. First, the confidence index of the graph generated in the previous step is compared with a robustness threshold. If the index is below the threshold, it indicates that the current probabilistic task graph is not robust enough. At this point, an iterative optimization process is triggered. This process identifies candidate task hypotheses that lead to lower expected task completion metrics and transforms them into a new set of hard constraints that must be satisfied. Subsequently, an optimizer, aiming to satisfy these new constraints, re-solves and adjusts the transition probabilities between nodes in the probabilistic task graph, thereby generating a new, robust version of the probabilistic task graph that is better adapted to all high-probability user intentions.

[0057] Optionally, the identification and generation of categorized event alarms includes:

[0058] Acquire multimodal data under historical normal inspection conditions and train a data generator model;

[0059] Specifically, this step aims to build a generative model capable of deep learning and reproducing the inherent distribution patterns of "normal" inspection data. In this step, a Generative Adversarial Network (GAN) framework is used for model training. This framework includes a generator and a discriminator. The training process is a dynamic, adversarial game: the generator continuously attempts to generate synthetic data indistinguishable from real normal data, while the discriminator continuously learns to more accurately distinguish between real and synthetic data. The goal of this training process is to minimize an adversarial loss function, an exemplary loss function that can be expressed by the formula:

[0060] ,

[0061] in, It is the overall value function; It is a generator; It is a discriminator; It comes from a real, normal data distribution. The sample; It comes from a prior noise distribution The sample; It is the discriminator that determines The probability of the actual data; The generator is based on noise The generated synthetic data. After training, the resulting generator is the data generator model, which has the ability to generate highly realistic and normal multimodal data from a low-dimensional latent space vector.

[0062] The real-time acquired multi-modal data is inversely mapped and reconstructed by the data generator model to calculate and generate optimal reconstructed data.

[0063] Specifically, the goal of this step is to find the best projection of any new real-time data onto the manifold of "normal" data, i.e., the "most similar normal data". An inverse reconstruction process converts this task into an optimization problem. First, a reconstruction loss function is defined by the difference between the real-time multi-modal data and the synthetic data output by the data generator model. Then, based on the gradient of the reconstruction loss function and coupled with a controlled random noise term, the latent space of the data generator model is iteratively explored through a Langevin dynamics sampling process to find the optimal latent space vector that minimizes the reconstruction loss function. Finally, the optimal latent space vector is input into the data generator model for a forward propagation, and the output is the optimal reconstructed data.

[0064] The reconstruction error between the multi-modal data and the optimal reconstructed data is quantitatively analyzed, and the event classification is performed on the region where the reconstruction error exceeds the preset dynamic threshold, to identify and generate a classified event alarm.

[0065] Specifically, this step is the final abnormality judgment link. An error analysis process calculates the difference between the real-time multi-modal data and its corresponding optimal reconstructed data, i.e., the reconstruction error. When a real-time data contains abnormal patterns, the data generator model has never learned such patterns in training, so it cannot generate a "normal" data similar to it, resulting in a significant increase in reconstruction error. By comparing the size of the reconstruction error with a dynamically set threshold, the presence of an anomaly can be determined. Further, by analyzing the distribution characteristics of the reconstruction error in different data dimensions, such as whether the texture part error of image data is large or the frequency spectrum part error of sound data is large, the identified abnormal events can be preliminarily classified, and finally classified event alarms containing event type, confidence, and spatiotemporal location are identified and generated, such as Figure 4 As shown, it shows how the system intelligently distinguishes different types of events such as "normal state", "traffic accident", and "illegal parking" in a two-dimensional feature space composed of "image reconstruction error" and "sound reconstruction error" through a nonlinear decision boundary.

[0066] Optionally, the calculating and generating optimal reconstructed data comprises:

[0067] Based on the latent space of the data generator model, a reconstruction loss function is defined using the difference between the multi-modal data and the synthetic data output by the data generator model.

[0068] Specifically, this step aims to construct a mathematical metric that can accurately quantify the difference between any two pieces of multi-modal data. A loss function definition process not only considers the direct difference in pixels or numerical values between the real-time acquired multi-modal data and the synthetic data output by the data generator model, but also creatively introduces a perception loss term. The perception loss term extracts high-dimensional features of the two pieces of data through a pre-trained deep neural network and calculates the distance between these high-dimensional features. By weighted summing the direct difference loss and the perception loss, a more comprehensive reconstruction loss function can be constructed that focuses on both low-level details and high-level semantic similarity.

[0069] Based on the gradient of the reconstruction loss function and coupled with a random noise term, the latent space is iteratively explored to generate latent space state samples;

[0070] Specifically, this step aims to find the optimal latent space vector that minimizes the reconstruction loss function through an efficient global optimization process. An iterative exploration process uses a Langevin dynamics sampling method derived from statistical physics. In each iteration, the update of a candidate vector in the latent space not only moves a small step in the opposite direction of the gradient of the reconstruction loss function, but is also subject to an additional random noise term with controlled intensity sampled from a Gaussian distribution. The introduction of this random noise term allows the search process to escape the "trap" of local optimal solutions, enabling more extensive exploration in the entire latent space and ultimately converging to the neighborhood of the global optimal solution, generating a set of latent space state samples surrounding the optimal solution.

[0071] Statistical moment estimation is performed on the latent space state samples to calculate and determine the first latent space vector, and the first latent space vector is input into the data generator model for forward propagation to generate optimal reconstruction data.

[0072] Specifically, this step is the final step of determining the optimal latent vector and completing the reconstruction. A statistical estimation process processes a set of latent space state samples generated at the end of the iterative exploration in the previous step in a stable state. By calculating the mean of the set of samples, a latent space vector as the optimal estimate can be obtained. This optimal latent space vector determined after sufficient exploration and optimization is used as the final input for a forward propagation calculation through the data generator model, and the output is the optimal reconstruction data closest to the original real-time multi-modal data and the best "normal state" reconstruction version.

[0073] Optionally, the generating the closed-loop inspection task report comprises:

[0074] The classified event alarm and the structured task vector are fused to construct a comprehensive decision context.

[0075] Specifically, this step aims to provide a complete information input for subsequent intelligent decision-making, which contains both "task intention" and "real situation". A context construction process will structurally integrate the generated classification event alarm representing the current specific abnormal event and the generated structured task vector representing the user's original patrol intention at the data level. The final generated comprehensive decision context is a machine-readable data object containing complex information such as "(task priority: high, focus target: congestion) + (real-time event: multiple car pileup at K105, confidence: 98%)".

[0076] Input the comprehensive decision context into a large language model for multi-objective reasoning to generate a dynamic response strategy;

[0077] Specifically, this step is the core of the present application to realize advanced intelligent decision-making. A large language model receives the comprehensive decision context constructed in the previous step as part of its prompt. The model is trained or fine-tuned to understand professional knowledge in the field of traffic inspection and emergency disposal. Through its powerful context understanding and multi-objective reasoning capabilities, the model can go beyond simple rule matching, comprehensively evaluate the urgency of the current situation, the importance of the initial task, and the available disposal resources, and thus generate an optimal and personalized dynamic response strategy. The strategy is not a single action, but a structured behavior scheme containing multiple specific disposal actions and their recommended execution sequence.

[0078] According to the dynamic response strategy, perform the disposal method and record the disposal process to generate a closed-loop inspection task report.

[0079] Specifically, this step is the final execution and recording link to complete the entire "perception-decision-action" closed loop. A task execution module will analyze the dynamic response strategy generated in the previous step and convert each disposal action in it into a call instruction for specific hardware or software systems. After all disposal actions are executed, a report generation process will compile and structurally store all information of this task, including the initial natural language instruction, the generated structured task vector, the identified classification event alarm, the generated dynamic response strategy, and the execution record and results of each disposal action, to finally generate a closed-loop inspection task report that is complete in information and fully traceable.

[0080] Optionally, the method further comprises:

[0081] Project the timing behavior strategy in the structured task vector, calculate and generate a strategy-intention coverage matrix;

[0082] Specifically, this step aims to quantify the extent to which a generated, concrete behavioral strategy can satisfy the various possibilities contained in the user's original intent. A coverage calculation process first projects the generated temporal behavioral strategy onto a multivariate probability hypothesis space defined by the generated structured task vector. For each candidate task hypothesis, the process evaluates the degree to which the temporal behavioral strategy covers its core objective. For example, if a candidate task hypothesis is "patrol a 5-kilometer stretch of highway," and the generated temporal behavioral strategy only covers 3 kilometers of that stretch, its coverage is 60%. By repeating this evaluation process for all high-probability candidate task hypotheses, a policy-intent coverage matrix can be constructed. The calculation of a matrix element can be expressed by the formula:

[0083] ,

[0084] in, These are elements in the policy-intent coverage matrix, representing the first... Each temporal behavior strategy assumes the core target region for the i-th candidate task. Coverage rate; Is this behavioral strategy in The spatial location of the drone at any given moment; It is an indicator function that is 1 when the drone is within the target area and 0 otherwise. Target area Total size.

[0085] Information entropy analysis is performed on the strategy-intent coverage matrix to quantify and generate a cognitive coordination index;

[0086] Specifically, this step aims to assess, from an information theory perspective, whether the "focus" and "comprehensiveness" of the current decision match the "uncertainty" of the user's intent. An information entropy analysis process normalizes each row of the policy-intent coverage matrix generated in the previous step, obtaining a probability distribution that describes the "attention allocation" of a single behavioral policy across all possible intents. Subsequently, the Shannon entropy of this probability distribution is calculated and weighted by combining it with the posterior probabilities of each candidate task hypothesis in the structured task vector to obtain the final cognitive coordination index. An exemplary calculation function is shown in the formula:

[0087] ,

[0088] in, It is the final cognitive coordination index; the higher the value, the better the coordination between strategy and intention. It is the total number of candidate task hypotheses; It is the calculated number a posteriori probability of a candidate task hypothesis; is an element in the normalized policy-intent coverage matrix, representing the coverage of a policy over the hypothesis .

[0089] If the cognitive coordination index is lower than a preset coordination threshold, the candidate task hypothesis is taken as a penalty term to generate a cognitive bias feedback signal.

[0090] Specifically, to generate a cognitive bias feedback signal according to the cognitive coordination index, this step is the decision-making link to complete the “meta-learning” feedback loop. First, compare the cognitive coordination index generated in the previous step with a coordination threshold. If the index is lower than the threshold, it indicates that there is a “mismatch” between the current decision-making behavior and the user's cognition of the intent. For example, too much attention may be paid to the task hypothesis with the highest probability, while other possibilities are completely ignored. At this time, a feedback signal generation process will be activated, which will package those “ignored” candidate task hypotheses with low coverage and their corresponding low coverage into a structured cognitive bias feedback signal. The signal is used as a special “penalty term” to guide and optimize the behavior preferences of the future decision-making model, so that it makes more comprehensive and coordinated decisions in subsequent tasks.

[0091] Optionally, the method further comprises:

[0092] using the cognitive bias feedback signal to dynamically reshape the reward function of the policy generation network model to generate an updated reward function;

[0093] Specifically, this step aims to convert the quantified “cognitive mismatch” degree into direct “penalty” or “incentive” for subsequent decision-making behavior. A reward function reshaping process receives the cognitive bias feedback signal. The signal indicates which candidate task hypotheses were “ignored” in the previous strategy. The process generates an updated reward function by adding a dynamic penalty term related to the feedback signal to the original reward function of the deep reinforcement learning model. The design of the penalty term aims to give a significant negative reward to the policy generation network if it generates a behavior strategy that again ignores these important candidate task hypotheses in subsequent training, thereby forcing it to adjust its decision-making preferences.

[0094] using the updated reward function to perform online training on the policy generation network model to generate an updated policy generation network model.

[0095] Specifically, this step is the execution link of the whole meta-learning closed loop, and realizes the evolution of the model "decision wisdom". An online reinforcement learning training process is started, which uses the updated reward function as its core optimization target. In the training, the policy generation network model learns how to generate a new timing behavior policy through continuous "trial and exploration", which not only meets the original task target, but also strives to maximize the reward function which is dynamically reshaped and contains the reward of "cognitive coordination". Through this online training process, the original policy generation network model is replaced by an updated policy generation network model which has more comprehensive decision preferences and more robust behavior when facing ambiguous intentions.

[0096] Embodiment 1:

[0097] In order to verify the feasibility of the application in implementation, the application is applied to the intelligent inspection management of a section of full-length 50 kilometers of newly opened intelligent expressway. The expressway section passes through hilly and urban-rural junctions, contains multiple long tunnels, viaducts and complex ramp hubs, and its traffic flow has high dynamicity and uncertainty. The traditional inspection method relying on manual preset fixed flight path and simple rule triggering is difficult to cope with the complex patrol requirements and emergencies of this section.

[0098] In order to verify the beneficial effects of the application, a test period of 3 months is selected for all-weather intelligent and automated inspection of the section. The control group uses the traditional unmanned aerial vehicle inspection system, which manually plans the flight path by the operator according to the instructions of the monitoring center, and uses the rule-based event alarm logic. The experimental group completely deploys the method and system described in the application, and carries out all-weather intelligent inspection through an unmanned aerial vehicle autonomous airport and a cluster of unmanned aerial vehicles carrying multi-modal sensors.

[0099] In this embodiment, when the operator of the control center inputs a fuzzy natural language instruction such as "G50 Expressway K1520 section westbound, evening peak seems to be a bit congested, pay attention, see if there is an accident" through voice, a large language model immediately performs multi-path reasoning on it and generates multiple candidate task hypotheses, such as {Hypothesis A: regular evening peak congestion, initial probability: 0.7}, {Hypothesis B: minor traffic accident, initial probability: 0.2}, {Hypothesis C: construction ahead of time, initial probability: 0.1}. The system immediately calls the real-time traffic API of the external map service to obtain that the average speed of the K1520 section is significantly lower than the historical same period data, and through Bayesian inference, the posterior probability of each hypothesis is updated, for example, the probability of {Hypothesis B: minor traffic accident} is increased to 0.65, and a structured task vector focusing on "accident investigation" and "traffic diversion" is finally generated.

[0100] Subsequently, based on the structured task vector, a probabilistic task graph is constructed, which not only contains the key patrol of K1520 section, but also assigns lower observation probability to adjacent ramp exits based on the uncertainty in the task vector. At the same time, a strategy generation network model combines real-time wind speed data and traffic flow to generate an initial time series behavior strategy. Before execution, a robustness verification module is started, which deduces the strategy under the candidate task assumption that "K1520 section of the main road is completely blocked", and finds that the expected task completion degree is very low. Therefore, the system automatically optimizes the node transition probability of the probabilistic task graph, adds an alternative path for the UAV to approach the core area from the air above the adjacent ramp, and generates a more robust adaptive patrol strategy.

[0101] During the patrol execution process, the multi-modal data obtained by the UAV along the route is input into a data generator model based on GAN training. At K1522, the reconstruction error of a video frame significantly exceeds the dynamic threshold, and the system determines it as an anomaly. By analyzing the source of the error, it is mainly caused by a stationary hot target in the image that does not conform to the normal vehicle kinematics, and a classification event alarm is generated, type "emergency lane abnormal parking". During the entire test period, this method has also successfully identified previously unseen abnormal types, such as a small amount of falling rocks caused by landslides.

[0102] Finally, the "emergency lane abnormal parking" alarm, together with the initial structured task vector that has increased the "accident probability", is input into the large language model for multi-objective reasoning. The model determines that this abnormal parking is most likely related to a potential traffic accident. Therefore, it does not generate the usual "illegal parking evidence" instruction, but generates a higher priority dynamic response strategy, which includes: 1. instruct the UAV to immediately adjust its attitude and use the zoom lens to record multiple-angle, high-definition video of the target vehicle and surrounding road conditions; 2. turn on the on-board intercom to call the target vehicle to confirm whether there are injured persons; 3. link with Gaode Map to push a "traffic accident ahead, please slow down" prompt to the oncoming vehicles. All processes are recorded to generate a complete closed-loop patrol task report.

[0103] After 3 months of comparative testing, the invention has shown significant technical advantages in task understanding accuracy, anomaly event detection ability, and intelligent level of emergency disposal. See Tables 1, 2, and 3 for specific data.

[0104] Table 1 Comparison of intelligent task understanding and planning efficiency

[0105]

[0106] Table 2 Abnormal event detection performance comparison table

[0107]

[0108] Table 3 Emergency response effect comparison table

[0109]

[0110] Based on the same inventive concept, the present application also provides a highway unmanned aerial vehicle intelligent patrol inspection system based on patrol requirements, as shown in the accompanying drawings, the system comprises: Figure 5

[0111] A task semantic analysis module is configured to obtain a natural language instruction of the patrol requirement, perform semantic analysis and intention decomposition on the natural language instruction, and generate a structured task vector.

[0112] An adaptive strategy generation module is configured to perform collaborative planning based on the structured task vector and in combination with real-time traffic and environmental perception data, construct and generate an adaptive patrol strategy.

[0113] An autonomous patrol and event identification module is configured to distribute the adaptive patrol strategy to an unmanned aerial vehicle for autonomous patrol, and perform real-time analysis on multi-modal data obtained by the unmanned aerial vehicle, identify and generate a classified event alarm.

[0114] A closed-loop response and report generation module is configured to respond to the classified event alarm, match and trigger a corresponding handling method from a pre-set multi-level response plan library, and generate a closed-loop patrol task report.

[0115] It should be noted that the function division and information interaction between the above-mentioned various modules are logical, and can be integrated in the same software platform or distributed in physical implementation. The connection between them represents data flow and control flow, and aims to cooperatively achieve the building energy consumption dynamic optimization target of the present application. The above-mentioned is only an exemplary embodiment of the present application, and cannot limit the protection scope of the present application.​

Claims

1. A method for intelligent inspection of expressway by UAV based on patrol demand, characterized in that, The method comprises: acquiring natural language instructions of patrol requirements, performing semantic analysis and intention decomposition on the natural language instructions, and generating a structured task vector; wherein the generation of the structured task vector comprises: using a large language model to perform multi-path reasoning on the natural language instructions to generate a candidate task hypothesis; obtaining objective data for verifying the candidate task hypothesis from an external real-time data source for the candidate task hypothesis to obtain external calibration evidence; based on the external calibration evidence, performing posterior probability calculation and weighted fusion on the candidate task hypothesis to generate a structured task vector; based on the structured task vector, and in cooperation with real-time traffic and environmental perception data, constructing and generating an adaptive patrol strategy; distributing the adaptive patrol strategy to a UAV for autonomous patrol, and performing real-time analysis on multi-modal data obtained by the UAV to identify and generate a classified event alarm; in response to the classified event alarm, matching and triggering a corresponding disposal method from a pre-set multi-level response plan library to generate a closed-loop patrol task report; wherein the generation of the closed-loop patrol task report comprises: fusing the classified event alarm and the structured task vector to construct a comprehensive decision context; inputting the comprehensive decision context into a large language model for multi-objective reasoning to generate a dynamic response strategy; and executing the disposal method according to the dynamic response strategy and recording the disposal process to generate a closed-loop patrol task report.

2. The method for intelligent patrol inspection of the highway based on the patrol requirement of the unmanned aerial vehicle according to claim 1, characterized in that, The construction and generation of the adaptive patrol strategy comprises: based on the patrol target and probability distribution in the structured task vector, constructing and generating a probabilistic task graph; based on the probabilistic task graph, the real-time traffic and the environmental perception data, performing real-time strategy calculation to generate a time sequence behavior strategy; wherein the generation of the time sequence behavior strategy comprises: inputting the topological structure and weight of the probabilistic task graph, and real-time traffic and environmental perception data into a pre-set strategy generation network model to obtain a time sequence behavior strategy; according to the time sequence behavior strategy, performing path instantiation on the probabilistic task graph to construct and generate an adaptive patrol strategy.

3. The method for intelligent patrol inspection of the highway based on the patrol requirement of the unmanned aerial vehicle according to claim 2, characterized in that, The method further comprises: performing path planning deduction on the probabilistic task graph under the candidate task hypothesis to generate an expected task completion degree index; statistically analyzing the expected task completion degree index to calculate and generate a graph confidence index of the probabilistic task graph; if the graph confidence index is lower than a pre-set robustness threshold, taking the candidate task hypothesis as a new constraint to iteratively optimize the node transition probability of the probabilistic task graph.

4. The method for intelligent patrol inspection of the highway based on patrol requirements according to claim 1, characterized in that, The identification and generation of the classified event alarm comprises: obtaining multi-modal data under a historical normal patrol state to train a data generator model; performing reverse mapping and reconstruction on real-time obtained multi-modal data through the data generator model to calculate and generate optimal reconstruction data; quantitatively analyzing reconstruction errors between multi-modal data and the optimal reconstruction data, and classifying events in a region where the reconstruction errors exceed a pre-set dynamic threshold to identify and generate a classified event alarm.

5. The method for intelligent patrol inspection of the highway based on patrol requirements according to claim 4, characterized in that, The computing and generating optimal reconstruction data comprises: Based on the latent space of the data generator model, a reconstruction loss function is defined by using the difference between the multi-modal data and the synthetic data output by the data generator model; Based on the gradient of the reconstruction loss function, and coupled with a random noise term, the latent space is iteratively explored to generate latent space state samples; Statistical moment estimation is performed on the latent space state samples, a first latent space vector is calculated and determined, and the first latent space vector is input into the data generator model for forward propagation to generate optimal reconstruction data.

6. The method for intelligent patrol inspection of the highway based on patrol requirements according to claim 3, characterized in that, The method further comprises: Projecting the time-series behavior policy in the structured task vector, calculating and generating a policy-intent coverage matrix; Performing information entropy analysis on the policy-intent coverage matrix, quantifying and generating a cognitive coordination index; If the cognitive coordination index is lower than a preset coordination threshold, the candidate task hypothesis is taken as a penalty term to generate a cognitive bias feedback signal.

7. The method for intelligent patrol inspection of the highway based on patrol requirements of the unmanned aerial vehicle according to claim 6, characterized in that, The method further comprises: Using the cognitive bias feedback signal, dynamically reshaping the reward function of the policy generation network model to generate an updated reward function; Using the updated reward function, online training the policy generation network model to generate an updated policy generation network model.

8. The intelligent patrol system of the highway based on patrol requirements of the unmanned aerial vehicle, applied to the intelligent patrol method of the highway based on patrol requirements of the unmanned aerial vehicle as claimed in any one of claims 1-7, characterized in that, The system comprises: A task semantic analysis module for obtaining a natural language instruction of a patrol requirement, performing semantic analysis and intent decomposition on the natural language instruction, and generating a structured task vector; wherein the generation of the structured task vector comprises: using a large language model to perform multi-path reasoning on the natural language instruction to generate a candidate task hypothesis; obtaining objective data for verifying the candidate task hypothesis from an external real-time data source to obtain external verification evidence; based on the external verification evidence, performing posterior probability calculation and weighted fusion on the candidate task hypothesis to generate a structured task vector; An adaptive policy generation module for generating an adaptive patrol strategy based on the structured task vector and fusing real-time traffic and environmental perception data for collaborative planning; An autonomous patrol and event identification module for distributing the adaptive patrol strategy to a UAV for autonomous patrol, and performing real-time analysis on the multi-modal data obtained by the UAV to identify and generate a classified event alarm; A closed-loop response and report generation module for responding to the classified event alarm, matching and triggering a corresponding handling method from a preset multi-level response plan library to generate a closed-loop patrol task report; wherein the generation of the closed-loop patrol task report comprises: fusing the classified event alarm and the structured task vector to build a comprehensive decision context; inputting the comprehensive decision context into a large language model for multi-objective reasoning to generate a dynamic response strategy; executing the handling method according to the dynamic response strategy and recording the handling process to generate a closed-loop patrol task report.

Images