A highway unmanned aerial vehicle autonomous inspection method and system

Abstract

The present invention provides a method and system for autonomous inspection of highway drones, which relates to the field of data processing technology. The method includes: receiving a preset inspection path from the inspection start point to the inspection end point; collecting drone sensor information and drone angular velocity information during the inspection process; performing obstacle avoidance when the drone sensor information detects an obstacle, otherwise, detecting whether it is yawed; generating an obstacle avoidance direction based on the potential field principle, and performing obstacle avoidance according to the obstacle avoidance direction; calculating the drone yaw angle according to the drone angular velocity information, and judging whether it is yawed based on the drone yaw angle; correcting the drone heading according to the yaw angle, and inspecting along the adjusted drone heading; collecting real-time video frames of highways; constructing a highway abnormal scene recognition model, and outputting abnormal scenes; uploading abnormal scenes to the gimbal, and issuing abnormal reminders. The autonomous inspection efficiency and inspection accuracy are improved, and the frequency of manual intervention and deployment costs are reduced.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and system for autonomous inspection of highways by unmanned aerial vehicles. Background Art

[0002] Autonomous inspection by drones on highways refers to the method of using drones to inspect and monitor highways. Drones are equipped with high-definition cameras and other sensors to automatically perform inspection tasks and conduct real-time monitoring of traffic flow, vehicle anomalies, road conditions, accidents, etc. on highways. Through preset routes or real-time remote control, drones can quickly cover a large area and collect necessary information for subsequent processing and analysis. Autonomous inspections can improve monitoring efficiency and emergency response speed. Compared with traditional inspection methods, they can greatly reduce labor costs and time costs. Especially during peak hours or when an accident occurs, drones can quickly reach the scene of the incident and provide first-hand on-site data to help traffic management departments make decisions quickly and effectively prevent and reduce the impact of emergencies such as traffic congestion and accidents.

[0003] However, the existing drone autonomous inspection methods still require too much human intervention, especially obstacle avoidance and yaw correction problems that drones cannot complete autonomously. In addition, various sudden scenarios on highways will be encountered during the inspection process. The existing drone-mounted recognition model has the problem of insufficient scene recognition rate, which seriously hinders the efficiency of emergency handling of abnormal scenarios, thereby leading to increased inspection costs and reduced practicality. Summary of the invention

[0004] In order to solve the problem that the existing drone autonomous inspection method still requires too much manual intervention, especially the obstacle avoidance and yaw correction problems that the drone cannot complete autonomously, and various sudden scenes on the highway will be encountered during the inspection process, the existing drone-mounted recognition model has the problem of insufficient scene recognition rate, which seriously hinders the efficiency of sudden handling of abnormal scenes, thereby leading to the technical problems of increased inspection costs and reduced practicability, the present invention provides a highway drone autonomous inspection method and system.

[0005] The technical solution provided by the embodiment of the present invention is as follows:

[0006] First aspect

[0007] An embodiment of the present invention provides a method for autonomous inspection of a highway by a drone, comprising:

[0008] S1: Receive the preset inspection path from the inspection starting point to the inspection end point;

[0009] S2: Perform inspections from the inspection starting point along the preset inspection route, and collect drone sensor information and drone angular velocity information during the inspection process;

[0010] S3: If the drone sensor information detects an obstacle, proceed to step S4, otherwise proceed to step S5;

[0011] S4: Combine the inspection endpoint and the obstacle, generate the obstacle avoidance direction based on the potential field principle, and perform obstacle avoidance according to the obstacle avoidance direction;

[0012] S5: Calculate the yaw angle of the drone according to the angular velocity information of the drone, and determine whether the drone is yawed based on the yaw angle of the drone. If so, proceed to step S6, otherwise, proceed to step S7;

[0013] S6: Correct the heading of the drone according to the yaw angle, and perform inspection along the adjusted heading of the drone;

[0014] S7: Collecting real-time video frames of the highway during the inspection process;

[0015] S8: constructing a highway abnormal scene recognition model, wherein the highway abnormal scene recognition model includes a basic feature extraction module, a deep feature extraction module, a fully connected layer and an activation layer connected in sequence, wherein the deep feature extraction module includes a plurality of parallel submodules connected in sequence, wherein the parallel submodule includes a parallel first convolution layer and a second convolution layer, wherein the first convolution layer is composed of a first rectangular frame and a second rectangular frame with interchangeable length and width in parallel, and the second convolution layer includes a multi-frequency attention unit;

[0016] S9: inputting the real-time video frame of the highway into the highway abnormal scene recognition model, and outputting the abnormal scene;

[0017] S10: Upload the abnormal scene to the gimbal and issue an abnormal reminder.

[0018] Second aspect

[0019] An embodiment of the present invention provides a highway drone autonomous inspection system, comprising:

[0020] processor;

[0021] A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method for autonomous highway drone inspection as described in the first aspect is implemented.

[0022] The third aspect

[0023] An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for autonomous highway drone inspection as described in the first aspect is implemented.

[0024] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0025] In the present invention, based on basic information such as drone sensor information and drone angular velocity information obtained by the drone, obstacles are continuously detected during the inspection process, and obstacle avoidance directions are generated in combination with the inspection endpoint and the obstacle to avoid the obstacle. The invention has high immediacy and dynamically responds to environmental changes, while reducing dependence on complex sensor systems, thereby improving flight safety and efficiency. In the process of obstacle avoidance, the drone tends to the inspection endpoint. Based on the principle of potential field, the drone can generate a reverse force when an obstacle is detected, so that the drone naturally orbits around the obstacle, thereby achieving smooth and continuous obstacle avoidance flight. The drone is calculated during obstacle avoidance or normal inspection. The yaw angle can autonomously determine whether to deviate from the preset route, increase the inspection accuracy and the accuracy of collecting abnormal scenes, and set up a continuous feature extraction module and a parallel first rectangular frame and second rectangular frame with interchangeable length and width to perform multi-scale feature extraction on the collected video frames to improve the recognition accuracy of abnormal scenes on highways and reduce the computing resource consumption caused by full-scale feature extraction to a certain extent, increase the endurance of the UAV. The whole process can greatly reduce the number of manual interventions, complete inspections autonomously and efficiently, and improve the recognition accuracy of abnormal scenes, thereby reducing the number of false alerts, reducing the cost of UAV deployment, and increasing the practicality of UAVs. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 A schematic diagram of a process flow of a method for autonomous inspection of a highway by a drone provided by an embodiment of the present invention;

[0028] Figure 2 A schematic diagram of the structure of a highway abnormal scene recognition model provided by an embodiment of the present invention;

[0029] Figure 3 A schematic diagram of a structure of a first convolutional layer for feature extraction provided in an embodiment of the present invention;

[0030] Figure 4 A schematic structural diagram of a highway drone autonomous inspection system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0032] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0033] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0034] In the embodiments of the present invention, sometimes the subscripts such as W 1 It may be mistakenly written as a non-subscript form such as W1. When the difference is not emphasized, the meanings they express are the same.

[0035] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0036] Reference Manual Attached Figure 1 , shows a flow chart of a method for autonomous highway drone inspection provided by an embodiment of the present invention.

[0037] The embodiment of the present invention provides a method for autonomous inspection of highways by drones, which can be implemented by autonomous inspection equipment for highways by drones, and the autonomous inspection equipment for highways by drones can be a terminal or a server. The processing flow of the method for autonomous inspection of highways by drones can include the following steps:

[0038] S1: Receive the preset inspection path from the inspection start point to the inspection end point.

[0039] The preset inspection path refers to the flight route that has been set before the drone performs the highway inspection mission. This path defines the specific flight trajectory of the drone from the inspection starting point to the end point, including the take-off point, flight route, key inspection area and landing point. Specifically, the preset inspection path covering all key areas (such as interchanges, bridges, tunnel entrances, etc.) can be planned using highway map data and geographic information systems (GIS).

[0040] S2: Perform inspections from the inspection starting point along the preset inspection route, and collect drone sensor information and drone angular velocity information during the inspection process.

[0041] Among them, the drone sensor information includes data collected by various sensors carried by the drone. The drone angular velocity information refers to the data of the rotation rate of each axis of the drone during flight, which is usually measured by a gyroscope sensor. The angular velocity information is used to monitor and adjust the attitude of the drone, including yaw, pitch and roll. Specifically, the drone angular velocity information can be collected through the drone's angular velocity sensor. Among them, the angular velocity sensor is also called a gyroscope sensor, which is a necessary sensor for each drone or motion robot. Collecting drone sensors and angular velocity information can achieve comprehensive monitoring and precise inspection of highways. Such data collection provides a rich source of information for subsequent data analysis and anomaly detection, ensuring the efficiency and accuracy of inspections. Real-time monitoring of the attitude and position of the drone also enhances the safety of flight, and can adjust the flight path in time to avoid potential obstacles and dangers. In addition, through the automated inspection process, the manpower requirements and related costs are greatly reduced, and the economic efficiency of inspections is improved.

[0042] S3: If the drone sensor information detects an obstacle, go to step S4, otherwise go to step S5.

[0043] It should be noted that when the drone's sensor detects an obstacle, the obstacle avoidance program is automatically triggered. If no obstacle is detected, the regular inspection program continues, which significantly enhances the flexibility and safety of drone inspections. By automatically detecting and responding to environmental changes, the drone can quickly adjust the flight plan when necessary to ensure that collisions with obstacles are avoided, while maintaining the continuity and efficiency of the inspection mission to the greatest extent possible and improving environmental adaptability.

[0044] S4: Combine the inspection endpoint and the obstacle, generate the obstacle avoidance direction based on the potential field principle, and perform obstacle avoidance according to the obstacle avoidance direction.

[0045] Among them, the potential field principle is a path planning and obstacle avoidance method. This method assumes that there is a virtual "potential field" around the robot, in which obstacles generate repulsive forces and the target position generates attractive forces. The robot calculates its movement direction based on the combined force of these forces, avoids obstacles and moves towards the target.

[0046] By applying the potential field principle to automatically generate obstacle avoidance directions, the drone is able to immediately and effectively adjust its flight path when an obstacle is detected to ensure safe bypassing of the obstacle. The benefits of this approach include enhancing the autonomy and adaptability of the drone, enabling it to operate flexibly in different environments while maximizing inspection efficiency and safety. In addition, the potential field-based obstacle avoidance strategy can be dynamically adjusted in real time to adapt to environmental changes, thereby improving the reliability and efficiency of drones when performing complex tasks. With automatic obstacle avoidance, drones can continue to conduct efficient inspection activities without human intervention, which is particularly important for covering vast or difficult-to-reach highway areas.

[0047] In a possible implementation, S4 specifically includes:

[0048] S401: Obtain the inspection endpoint position and obstacle position.

[0049] S402: Calculate the first distance from the drone to the inspection endpoint and the second distance from the drone to the obstacle location:

[0050]

[0051]

[0052] Among them, d 1 represents the first distance, d 2 represents the second distance, p and (x, y) represent the current position of the drone, p g and (x g ,y g ) represents the inspection end point, p z and (x z ,y z ) indicates the obstacle location.

[0053] S403: Calculate the drone gravitational potential and the drone repulsive potential based on the first distance and the second distance:

[0054]

[0055]

[0056] Among them, d 0 represents the maximum detection distance of the drone sensor, ζ and ε represent the drone gravitational potential adjustment coefficient and the drone repulsive potential adjustment coefficient, respectively.

[0057] It should be noted that the meaning of the repulsive potential in the formula is that when the second distance, that is, the distance between the drone and the obstacle, is less than the maximum detection distance of the drone sensor, the repulsive potential is activated, and the intensity increases as it approaches the obstacle. The characteristic of this potential energy function is that when the distance between the drone and the obstacle is equal to the maximum detection distance of the drone sensor, the repulsive potential is zero, and when it approaches zero (that is, the drone is very close to the obstacle), the repulsive potential increases rapidly, generating a strong repulsive force to prevent the object from colliding with the obstacle.

[0058] S404: Calculate the total potential field of the drone based on the drone's gravitational potential and the drone's repulsive potential:

[0059] E(p)=E 引力 (p)+E 斥力 (p)

[0060] Where E(p) represents the total potential field of the UAV.

[0061] Among them, the attraction generated by the drone's gravitational potential will point to the target point because it is a negative gradient, that is, it points from the high potential energy area to the low potential energy area. The repulsive force generated by the drone's repulsive potential will prompt the drone to stay away from obstacles because it is also a negative gradient, that is, it points from the low potential energy area (close to the obstacle) to the high potential energy area (away from the obstacle).

[0062] S405: Based on the total potential field of the drone, determine the virtual force field of the drone, where the virtual force field of the drone is the negative gradient of the total potential field of the drone:

[0063] D(p)=-▽E(p)

[0064] Where D(p) represents the virtual force field of the drone and ▽ represents the gradient operator.

[0065] It should be noted that the compound force generated by the UAV’s virtual force field combines the effects of pointing toward low potential energy areas (attraction) and away from high potential energy areas (repulsion), helping the navigation system avoid obstacles while moving toward the target with greater safety.

[0066] S406: Using the direction of the virtual force field as an obstacle avoidance direction to avoid obstacles.

[0067] It should be noted that by combining the inspection endpoint and the obstacle position, the potential field principle is used to automatically plan the obstacle avoidance route of the drone. The specific execution details range from obtaining the position, calculating the distance to generating the potential field and determining the obstacle avoidance direction, providing the drone with an efficient and reliable autonomous obstacle avoidance mechanism that can dynamically respond to environmental changes, calculate the gravitational potential and repulsive potential in real time, and ensure that the drone can effectively move toward the predetermined endpoint while avoiding obstacles. This potential field-based obstacle avoidance algorithm not only enhances the autonomous navigation capability of the drone, but also greatly improves the safety and efficiency of flight. Through the negative gradient virtual force field, the drone can naturally move along a safe and optimal path, optimize the path planning, reduce the risk of collision, and ensure the continuity and accuracy of the task. This adaptive obstacle avoidance technology is particularly suitable for complex or changeable flight environments, such as highway inspections, and can effectively respond to various emergencies and ensure the efficient execution of inspection work.

[0068] S5: Calculate the yaw angle of the drone according to the angular velocity information of the drone, and determine whether the drone is yawed based on the yaw angle of the drone. If so, proceed to step S6, otherwise, proceed to step S7.

[0069] It should be noted that during the obstacle avoidance process, the drone will adjust the heading angle to bypass the obstacle. If only under the action of step S4, the drone will deviate from the highway route, that is, the preset inspection route. Therefore, the yaw angle should be calculated at all times during the obstacle avoidance process, and the obstacle avoidance should be carried out under the intervention of the yaw angle. The obstacle avoidance can be completed safely with the smallest obstacle avoidance path, and the preset route can be quickly returned to inspect the highway. In addition, after bypassing the obstacle according to the obstacle avoidance path and returning to the preset inspection path, the heading angle will remain unchanged. If the heading is not adjusted in time, the drone will also fall into continuous yaw and correction. After obtaining the yaw angle, the drone can automatically correct the heading according to the yaw angle, thereby avoiding the yaw caused by the obstacle avoidance process. In addition, if the real-time navigation information is lost during the inspection process, the real-time yaw correction can be completed based on this technology, which can be applied to various complex environments, increase the environmental adaptability of the drone, and can smoothly complete the automatic inspection of the highway without human intervention.

[0070] In a possible implementation, calculating the yaw angle of the drone according to the angular velocity information of the drone specifically includes:

[0071] The quaternion describing the UAV's movement attitude is updated according to the UAV's angular velocity information, where the quaternion includes a scalar part representing the rotation amplitude and a first vector, a second vector, and a third vector representing the rotation angle:

[0072]

[0073]

[0074]

[0075] Among them, ω x ,ω y and ω z They represent the x-axis angular velocity, the y-axis angular velocity and the z-axis angular velocity respectively, q=[q 0 ,q 1 ,q 2 ,q 3 ] represents a quaternion, q 0 ,q 1 ,q 2 ,q 3 denote the scalar part, the first vector, the second vector and the third vector respectively, Δt denotes the sampling interval between the kth sampling and the k+1th sampling, q k+1 and q k They represent the k-th sampling quaternion and the k+1-th sampling quaternion respectively, and |||| represents the modulo operator.

[0076] It should be noted that the purpose of dividing the quaternion by its own norm is to keep the quaternion as a unit quaternion, ensuring that it can correctly represent the rotation without introducing scaling. If normalization is not performed, the norm of the quaternion may gradually deviate from 1 due to calculation errors, which will affect the accuracy and stability of the rotation.

[0077] Convert the updated quaternion to a rotation matrix and calculate the drone's yaw angle based on the rotation matrix:

[0078]

[0079] yaw=arctan2(r 21 ,r 11 ),

[0080] Among them, R represents the rotation matrix, yaw represents the yaw angle of the drone, arctan2(r 21 ,r 11 ) represents the calculation of the rotation matrix r 21 =2(q 1 q 2 +q 0 q 3 ) elements and Four-quadrant inverse tangent of the element.

[0081] Among them, the four-quadrant inverse tangent value is also called the two-parameter inverse tangent value.

[0082] It should be noted that the method of using quaternions to update the attitude of the drone and converting it into a rotation matrix to calculate the yaw angle avoids the universal lock problem caused by the Euler angle representation compared to the traditional method of direct calculation using Euler angles, ensuring the continuity and stability of the rotation, which is particularly critical for the drone to perform precise autonomous navigation in complex spaces, because any rotation discontinuity or jump may cause navigation errors, affecting the safety and accuracy of mission execution. Secondly, quaternions are more numerically stable when performing continuous rotation and complex motion pattern calculations, and are not easily affected by cumulative calculation errors. In addition, after converting the quaternion into a rotation matrix, the calculation of the yaw angle can directly use matrix operations, which has been optimized on many modern computing platforms and can get results faster, thereby improving the real-time and response speed of the entire system.

[0083] S6: Correct the heading of the drone according to the yaw angle, and perform inspection along the adjusted heading of the drone.

[0084] S7: Collect real-time video frames of the highway during the inspection process.

[0085] It should be noted that skipping the video frame collection of the yaw and obstacle avoidance process is to avoid collecting yaw or obstacle avoidance video frames and allocate more computing resources to drone obstacle avoidance and effective highway video frame analysis. On the one hand, it improves the high-speed maneuverability of the drone in the obstacle avoidance process, and on the other hand, it reduces the consumption of irrelevant video frames on the drone's limited power.

[0086] Reference Manual Attached Figure 2 , showing a structural schematic diagram of a highway abnormal scene recognition model provided by an embodiment of the present invention.

[0087] Figure 2 In the figure, Conv 7x7 represents the Conv 7x7 convolutional layer, BatchNorm represents the batch normalization layer, ReLU represents the ReLU activation layer, and MaxPool represents the maximum pooling layer. Figure 2 A and B in the figure represent the first convolutional layer and the second convolutional layer respectively, the arrow direction represents the data flow, and n represents n parallel submodules, the specific number can be increased or decreased as needed. It can effectively extract the basic features of the image, and the multi-level deep feature extraction module further enhances the recognition ability of complex scenes. It accelerates feature analysis through parallel processing, improves the processing speed and accuracy of the model, and not only improves the performance of the model, but also ensures real-time performance.

[0088] Reference Manual Attached Figure 3 , shows a structural schematic diagram of a first convolutional layer for feature extraction provided by an embodiment of the present invention.

[0089] Figure 3In the figure, the dotted line indicates the specific size of the corresponding rectangular box. Figure 3 It can be seen that the height and width of the first rectangular box and the second rectangular box are swapped, and the direction of the arrow represents the data flow. This feature extraction method not only improves the accuracy of feature extraction, but also avoids the problem of excessive consumption of computing resources in the full-image feature extraction method.

[0090] S8: Build a highway abnormal scene recognition model.

[0091] Among them, the highway abnormal scene recognition model includes a basic feature extraction module, a deep feature extraction module, a fully connected layer and an activation layer connected in sequence, wherein the deep feature extraction module includes a plurality of parallel sub-modules connected in sequence, wherein the parallel sub-module includes a parallel first convolution layer and a second convolution layer, wherein the first convolution layer is composed of a first rectangular frame and a second rectangular frame with interchanged length and width in parallel, and the second convolution layer includes a multi-frequency attention unit.

[0092] Specifically, the second convolutional layer includes a Conv 1x1 convolutional layer, a batch normalization layer, a Conv 3x3 convolutional layer, and a multi-frequency attention unit, which are connected sequentially.

[0093] It should be noted that the constructed highway abnormal scene recognition model uses an advanced deep learning architecture to identify and analyze images collected from drone cameras. The model uses basic feature extraction modules and deep feature extraction modules, including parallel convolutional layers and multi-frequency attention units, to effectively capture and analyze complex features in images. Such an architecture can accurately identify various abnormal scenes, such as traffic accidents, road damage or traffic jams, so as to quickly respond and take necessary management measures. In addition, the use of parallel processing and multi-frequency attention units enhances the processing speed and recognition accuracy of the model, allowing drones to perform efficient and accurate analysis in real time during flight, significantly improving the efficiency and effectiveness of highway safety management.

[0094] Specifically, the first convolution layer is composed of two rectangular convolution kernels (P×Q and Q×P) in different directions, which capture features in both width and height directions, thereby obtaining better spatial feature expression. The two convolution kernels have a rectangular shape, one focusing on the vertical direction (e.g., P×Q when P>Q), and the other focusing on the horizontal direction (Q×P). This design enables the convolution operation to more flexibly adapt to the changes in different sizes and directions of images or feature maps. Using rectangular convolution kernels in different directions can expand the receptive field of the model, making it perform better in both vertical and horizontal directions. Compared with using a large square convolution kernel (such as P×P), using two rectangular convolution kernels can reduce the amount of computation while maintaining a larger receptive field, because the total number of parameters of the rectangular convolution kernel is less than that of the large square convolution kernel. This structure is particularly suitable for processing highway images that have obvious features in a specific direction.

[0095] In one possible implementation, the basic feature extraction module includes a Conv 7x7 convolution layer, a batch normalization layer, a ReLU activation layer, and a maximum pooling layer connected in sequence.

[0096] It should be noted that the use of a basic feature extraction module consisting of a Conv 7x7 convolution layer, a batch normalization layer, a ReLU activation layer, and a maximum pooling layer connected in sequence helps to effectively extract key visual features from highway video frames. The benefit of this structure is that a larger convolution kernel (7x7) can capture a wider range of contextual information and enhance the expressiveness of features. The batch normalization layer helps to accelerate the convergence speed of model training and improve the stability of the model during training. The ReLU activation layer enhances the nonlinear processing capability of the network and helps to capture complex feature patterns. The maximum pooling layer reduces the number of parameters through dimensionality reduction operations, improves computational efficiency, and retains the most significant feature information. The design of this module ensures that the entire abnormal scene recognition model maintains high computational efficiency while ensuring accuracy, making it suitable for application in real-time systems.

[0097] In one possible implementation, the multi-frequency attention unit is specifically used for:

[0098] Divide the input features by channel dimension to obtain multiple feature components:

[0099]

[0100] F i ∈R C′×K×K ,i∈[0,n-1]

[0101]

[0102] Among them, C represents the number of input feature channels, C' represents the number of feature channels after division, K×K represents that the height and width of the channel space dimension feature are both K, and F i represents the i-th eigencomponent, R represents the real number domain, and n represents the number of partitions.

[0103] Perform a zigzag scan on the characteristic component to obtain the frequency component of the characteristic component.

[0104] Among them, zigzag scanning is a content selection method that starts from the upper left corner and selects frequencies along a diagonal pattern.

[0105] Combined with the frequency component index of the frequency component, the feature component is transformed into a two-dimensional DCT to obtain a DCT transformation vector:

[0106]

[0107] Among them, T i represents the i-th DCT transform vector obtained by the two-dimensional DCT transform 2DDCT of the i-th feature component, i∈[0,n-1], u i and v i Respectively represent the frequency component index of the i-th feature component in different dimensions, c(u i ,v i ) represents a constant normalization factor affected by the frequency component index, M represents the maximum frequency component index, (x, y) represents the index coordinates, cos represents the sine function, and π represents pi.

[0108] Among them, the normalization factor for each coefficient ensures that the energy is consistent before and after the transformation, which is very important for DCT transformation because it means that no matter how much the transformation is, the reconstructed image will maintain the energy level of the original image, thereby minimizing the distortion caused by compression.

[0109] The DCT transformation vector is input into the sigmoid activation function in sequence to output the attention output features of the multi-frequency attention unit:

[0110] A=sigmoid(fc(cat(T 0 ,T 1 ,...,T n-1 )))

[0111]

[0112] Among them, A represents the attention vector, sigmoid represents the sigmoid activation function, cat represents the connection operation, fc represents the fully connected layer, l represents the channel index of the multi-frequency attention unit, and f l The lth channel, a lrepresents the lth element of the attention vector, Represents the attention output feature.

[0113] It should be noted that the channel attention module usually includes a global average pooling operation to assign a single scalar weight to each channel. The initial operation is to average the information of all positions in the spatial dimension into a single value. Because the final weight acts on the entire channel, it is necessary to calculate the weight based on the entire channel information. In addition, the purpose is to use the correlation between channels rather than the spatial distribution correlation. Using global average pooling to suppress spatial distribution information helps to calculate weights more accurately. However, due to the simplicity of global average pooling, it is difficult to successfully extract complex information from multiple inputs, resulting in the loss of important information. The multi-frequency attention unit regards the acquisition of scalar weights as a compression process while retaining the overall representation ability of the channel. Discrete cosine transform (DCT) is applied to compress the channel, and then multiple frequency components are used to achieve channel attention, which retains more useful information and enhances the model's discrimination and learning capabilities.

[0114] In a possible implementation manner, after S8, the method further includes:

[0115] The highway abnormal scene recognition model is pruned and the inter-layer connections are optimized.

[0116] It should be noted that through pruning, the model can remove redundant or unimportant parameters and connections, thereby reducing the complexity of the model and the computing resources required for runtime, and improving the running speed and efficiency of the model. Model pruning can simplify model complexity and computing resource consumption, reduce the use of limited power of optimized drones, and thus improve the endurance of drones.

[0117] In a possible implementation, the highway abnormal scene recognition model is pruned to optimize the inter-layer connection, specifically including:

[0118] The objective function for measuring the loss value of the highway abnormal scene recognition model after pruning is established under the test data set. The specific objective function is:

[0119]

[0120] w∈R m

[0121] c∈{0,1} m

[0122] ||c|| 0 ≤k

[0123] Among them, min means taking the minimum value, L means the total loss function, represents the Hadamard product, w∈Rm represents the weight parameter set of the highway abnormal scene recognition model with m elements, R represents the real number domain, c represents the auxiliary indicator variable with m elements, c = 1 represents the connection weight parameter, when c = 0 represents the non-connection weight parameter, num represents the total number of samples in the test data set D, represents the i-th sample (x i ,y i )’s loss value, |||| 0 represents the L0 norm, and k represents the expected weight density level.

[0124] Among them, the objective function disconnects each weight parameter through an auxiliary indicator variable. When c = 1, it means connecting the weight parameter, and when c = 0, it means not connecting the weight parameter. The importance of each connection can be determined by measuring its impact on the loss function. j =1 means there is a connection j, c j = 0 means that there is no connection j, so the impact of connection j on the model loss can be adjusted by changing c j The value of while keeping other values ​​unchanged is measured, the formula ||c|| 0 ≤k, the L0 norm constraint limits the number of non-zero elements in vector c to at most k. For example, k can be 100, which means that at most only 100 weights can be retained and the rest are pruned.

[0125] Calculate the contribution of each connection to the objective function:

[0126]

[0127] Where, ΔL j (w; D) represents the contribution of the jth connection to the total loss function L in the weight parameter set w and the test data set D, d j (w; D) represents the loss sensitivity of the jth connection in the total loss function of the weight parameter set w and the test dataset D, c j represents the auxiliary indicator variable for the j-th connection, represents partial derivative, δ represents a decimal close to 0, e j represents the unit vector used to simulate the disturbance, which is 1 in the jth position and 0 in other positions, and lim represents the limiting process in which the parameter δ approaches 0.

[0128] By calculating c j The rate of change of the loss function when a small change occurs, quantifying the importance of the weight.

[0129] The loss sensitivity is normalized to represent the connection importance:

[0130]

[0131] Among them, s j represents the connection importance of the j-th connection, and || represents the absolute value.

[0132] Each connection is retained in descending order of connection importance, where the number of retained connections is the expected weight density level.

[0133] It should be noted that by identifying and deleting unimportant connections to reduce the complexity and computational burden of the model, this method can dynamically evaluate and adjust the importance of each connection by introducing a binary mask c and applying sparsity constraints. The specific process includes: first, using partial derivatives (the magnitude of the derivative) to quantify the impact of each connection on the model loss. Then, important connections are retained based on the normalized magnitude of the derivative, and unimportant connections are cut off. This method not only improves the operating efficiency of the model, but also helps reduce the risk of overfitting and makes the model more robust. This self-adjustment ability makes the adaptive pruning network particularly suitable for drone inspection scenarios with limited resources and the need for efficient operation, improving the endurance of drones.

[0134] S9: Input the real-time video frames of the highway into the highway abnormal scene recognition model, and output the abnormal scene.

[0135] It should be noted that the practice of inputting real-time video frames of highways into the abnormal scene recognition model has greatly improved the effectiveness and accuracy of inspections. By analyzing video data in real time, the model can quickly identify and report potential anomalies and dangers. This real-time recognition function enables traffic management departments to respond quickly and take timely measures to avoid potential traffic accidents and reduce traffic congestion, thereby greatly improving the safety and smoothness of highways.

[0136] In a possible implementation, abnormal scenarios include traffic congestion, traffic accidents, pedestrians on the highway, illegal parking, and road damage.

[0137] S10: Upload the abnormal scene to the gimbal and issue an abnormal reminder.

[0138] Understandably, uploading the identified abnormal scenarios to the PTZ and issuing abnormal alerts ensures that key information can be quickly conveyed to the traffic management center and relevant decision makers, enabling them to take appropriate response measures in a timely manner.

[0139] Specifically, by using advanced drone technology and real-time data processing in the process of highway inspection, the automation level and efficiency of inspection have been greatly improved. Specifically, the drone automatically performs tasks according to the preset inspection path, collects and analyzes sensor and angular velocity data in real time, ensures accurate control of flight attitude and timely detection of obstacles, and intelligently plans obstacle avoidance routes based on potential field principles to ensure flight safety. In addition, by introducing multi-frequency attention units and efficient deep learning models, drones can identify and process abnormal scenes in real time, reduce human errors, and enhance response speed. This highly automated inspection method not only improves inspection accuracy, but also significantly reduces operating costs and manpower requirements, making highway maintenance and management more efficient and safe. In addition, abnormal situations can be quickly reported and handled, greatly improving road safety and traffic efficiency.

[0140] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0141] In the present invention, based on basic information such as drone sensor information and drone angular velocity information obtained by the drone, obstacles are continuously detected during the inspection process, and obstacle avoidance directions are generated in combination with the inspection endpoint and the obstacle to avoid the obstacle. The invention has high immediacy and dynamically responds to environmental changes, while reducing dependence on complex sensor systems, thereby improving flight safety and efficiency. In the process of obstacle avoidance, the drone tends to the inspection endpoint. Based on the principle of potential field, the drone can generate a reverse force when an obstacle is detected, so that the drone naturally orbits around the obstacle, thereby achieving smooth and continuous obstacle avoidance flight. The drone is calculated during obstacle avoidance or normal inspection. The yaw angle can autonomously determine whether to deviate from the preset route, increase the inspection accuracy and the accuracy of collecting abnormal scenes, and set up a continuous feature extraction module and a parallel first rectangular frame and second rectangular frame with interchangeable length and width to perform multi-scale feature extraction on the collected video frames to improve the recognition accuracy of abnormal scenes on highways and reduce the computing resource consumption caused by full-scale feature extraction to a certain extent, increase the endurance of the UAV. The whole process can greatly reduce the number of manual interventions, complete inspections autonomously and efficiently, and improve the recognition accuracy of abnormal scenes, thereby reducing the number of false alerts, reducing the cost of UAV deployment, and increasing the practicality of UAVs.

[0142] Reference Manual Attached Figure 4 , showing a structural schematic diagram of a highway UAV autonomous inspection system provided by the present invention.

[0143] The present invention also provides a highway unmanned aerial vehicle autonomous inspection system 20, which is applied to the above-mentioned highway unmanned aerial vehicle autonomous inspection method, comprising:

[0144] Processor 201.

[0145] The memory 202 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 201, the highway drone autonomous inspection method as described in the method embodiment is implemented.

[0146] The highway drone autonomous inspection system 20 provided by the present invention can execute the above-mentioned highway drone autonomous inspection method and achieve the same or similar technical effects. To avoid repetition, the present invention will not go into details.

[0147] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0148] In the present invention, based on basic information such as drone sensor information and drone angular velocity information obtained by the drone, obstacles are continuously detected during the inspection process, and obstacle avoidance directions are generated in combination with the inspection endpoint and the obstacle to avoid the obstacle. The invention has high immediacy and dynamically responds to environmental changes, while reducing dependence on complex sensor systems, thereby improving flight safety and efficiency. In the process of obstacle avoidance, the drone tends to the inspection endpoint. Based on the principle of potential field, the drone can generate a reverse force when an obstacle is detected, so that the drone naturally orbits around the obstacle, thereby achieving smooth and continuous obstacle avoidance flight. The drone is calculated during obstacle avoidance or normal inspection. The yaw angle can autonomously determine whether to deviate from the preset route, increase the inspection accuracy and the accuracy of collecting abnormal scenes, and set up a continuous feature extraction module and a parallel first rectangular frame and second rectangular frame with interchangeable length and width to perform multi-scale feature extraction on the collected video frames to improve the recognition accuracy of abnormal scenes on highways and reduce the computing resource consumption caused by full-scale feature extraction to a certain extent, increase the endurance of the UAV. The whole process can greatly reduce the number of manual interventions, complete inspections autonomously and efficiently, and improve the recognition accuracy of abnormal scenes, thereby reducing the number of false alerts, reducing the cost of UAV deployment, and increasing the practicality of UAVs.

[0149] It should be understood that the processor in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0150] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0151] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0152] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0153] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0154] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0155] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0156] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0157] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0158] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0159] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0160] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0161] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for autonomous highway drone inspection as described in the method embodiment is implemented.

[0162] A computer-readable storage medium provided by the present invention can implement the steps and effects of the highway drone autonomous inspection method of the above method embodiment. To avoid repetition, the present invention will not go into details.

[0163] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0164] There are a few points to note:

[0165] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention, and other structures may refer to the general design.

[0166] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of the layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0167] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0168] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A highway drone autonomous inspection method, characterized in that: include: S1: Receive the preset inspection path from the inspection starting point to the inspection end point; S2: Performing inspection from the inspection starting point along the preset inspection path, and collecting drone sensor information and drone angular velocity information during the inspection process; S3: If the drone sensor information detects an obstacle, proceed to step S4; otherwise, proceed to step S5; S4: combining the inspection endpoint and the obstacle, generating an obstacle avoidance direction based on a potential field principle, and performing obstacle avoidance according to the obstacle avoidance direction; S5: Calculate the yaw angle of the drone according to the angular velocity information of the drone, and determine whether the drone is yawed based on the yaw angle of the drone. If so, proceed to step S6; otherwise, proceed to step S7; S6: Correct the heading of the drone according to the yaw angle, and perform inspection along the adjusted heading of the drone; S7: Collecting real-time video frames of the highway during the inspection process; S8: constructing a highway abnormal scene recognition model, wherein the highway abnormal scene recognition model includes a basic feature extraction module, a deep feature extraction module, a fully connected layer and an activation layer connected in sequence, wherein the deep feature extraction module includes a plurality of parallel submodules connected in sequence, wherein the parallel submodule includes a parallel first convolution layer and a second convolution layer, wherein the first convolution layer is composed of a first rectangular frame and a second rectangular frame with interchanged length and width in parallel, and the second convolution layer includes a multi-frequency attention unit; S9: inputting the real-time video frame of the highway into the highway abnormal scene recognition model, and outputting the abnormal scene; S10: Upload the abnormal scene to the PTZ and issue an abnormal reminder.

2. The method for autonomous highway inspection by unmanned aerial vehicle according to claim 1 is characterized in that: The S4 specifically includes: S401: Obtaining the inspection end point position and obstacle position; S402: Calculate the first distance from the drone to the inspection endpoint and the second distance from the drone to the obstacle location respectively: Wherein, d1 represents the first distance, d2 represents the second distance, p and (x, y) represent the current position of the drone, and p g and (x g ,y g ) represents the inspection end position, p z and (x z ,y z ) represents the position of the obstacle; S403: Calculate the drone gravitational potential and the drone repulsive potential based on the first distance and the second distance: Among them, d0 represents the maximum detection distance of the drone sensor, ζ and ε represent the adjustment coefficient of the drone gravitational potential and the adjustment coefficient of the drone repulsive potential, respectively; S404: Calculate the total potential field of the drone according to the drone gravitational potential and the drone repulsive potential: E(p)=E 引力 (p)+E 斥力 (p) Wherein, E(p) represents the total potential field of the UAV; S405: Based on the total potential field of the drone, determine the drone virtual force field, wherein the drone virtual force field is the negative gradient of the total potential field of the drone: D(p)=-▽E(p) Wherein, D(p) represents the virtual force field of the drone, and ▽ represents the gradient operator; S406: Using the direction of the virtual force field as an obstacle avoidance direction to avoid the obstacle.

3. The autonomous highway inspection method using unmanned aerial vehicles according to claim 1 is characterized in that: The calculating of the yaw angle of the drone according to the angular velocity information of the drone specifically includes: A quaternion describing the movement attitude of the drone is updated according to the drone angular velocity information, wherein the quaternion includes a scalar part representing the rotation amplitude and a first vector, a second vector, and a third vector representing the rotation angle: Among them, ω x ,ω y and ω z denote the x-axis angular velocity, the y-axis angular velocity and the z-axis angular velocity respectively, q=[q0,q1,q2,q3] denotes the quaternion, q0,q1,q2,q3 denote the scalar part, the first vector, the second vector and the third vector respectively, Δt denotes the sampling interval between the k-th sampling and the k+1-th sampling, q k+1 and q k They represent the k-th sampling quaternion and the k+1-th sampling quaternion respectively, and || || represents the modulus operator; Convert the updated quaternion to a rotation matrix and calculate the drone's yaw angle based on the rotation matrix: yaw=arctan2(r 21 ,r 11 ) Where R represents the rotation matrix, yaw represents the yaw angle of the drone, arctan2(r 21 ,r 11 ) represents the calculation of the rotation matrix r 21 =2(q1q2+q0q3) elements and Four-quadrant inverse tangent of the element.

4. The method for autonomous highway inspection by unmanned aerial vehicle according to claim 1 is characterized in that: The basic feature extraction module includes a Conv 7x7 convolution layer, a batch normalization layer, a ReLU activation layer and a maximum pooling layer connected in sequence.

5. The autonomous highway inspection method using unmanned aerial vehicles according to claim 1 is characterized in that: The multi-frequency attention unit is specifically used for: Divide the input features by channel dimension to obtain multiple feature components: F i ∈R C′×K×K ,i∈[0,n-1] Among them, C represents the number of input feature channels, C' represents the number of feature channels after division, K×K represents the height and width of the channel space dimension feature are both K, and F i represents the i-th characteristic component, R represents the real number domain, and n represents the number of partitions; Performing a zigzag scan on the characteristic component to obtain a frequency component of the characteristic component; The characteristic component is subjected to a two-dimensional DCT transform in combination with the frequency component index of the frequency component to obtain a DCT transform vector: Among them, T i represents the i-th DCT transform vector obtained by the two-dimensional DCT transform 2DDCT of the i-th feature component, i∈[0,n-1], u i and v i Respectively represent the frequency component index of the i-th feature component in different dimensions, c(u i ,v i ) represents a constant normalization factor affected by the frequency component index, M represents the maximum frequency component index, (x, y) represents the index coordinates, cos represents the sine function, and π represents pi; The DCT transformation vector is sequentially input into the sigmoid activation function to output the attention output feature of the multi-frequency attention unit: A=sigmoid(fc(cat(T0,T1,...,T n-1 ))) Where A represents the attention vector, sigmoid represents the sigmoid activation function, cat represents the connection operation, fc represents the fully connected layer, l represents the channel index of the multi-frequency attention unit, and f l The lth channel, a l represents the lth element of the attention vector, Represents the attention output feature.

6. The autonomous highway inspection method using unmanned aerial vehicles according to claim 1 is characterized in that: After S8, the method further includes: The highway abnormal scene recognition model is pruned to optimize the inter-layer connections.

7. The method for autonomous highway inspection by unmanned aerial vehicle according to claim 6 is characterized in that: The pruning and growing of the highway abnormal scene recognition model to optimize the inter-layer connection specifically includes: An objective function is established to measure the loss value of the highway abnormal scene recognition model after pruning under the test data set. The objective function is specifically: w∈R m c∈{0,1} m ||c||0≤k Among them, min means taking the minimum value, L means the total loss function, represents the Hadamard product, w∈R m represents a weight parameter set of the highway abnormal scene recognition model with m elements, R represents a real number domain, c represents an auxiliary indicator variable with m elements, when c=1, it represents a connection weight parameter, when c=0, it represents a non-connection weight parameter, num represents the total number of samples in the test data set D, represents the i-th sample (x i ,y i ), || ||0 represents the L0 norm, and k represents the expected weight density level; Calculate the contribution of each connection to the objective function: Where, ΔL j (w; D) represents the contribution of the jth connection to the total loss function L in the weight parameter set w and the test data set D, d j (w; D) represents the loss sensitivity of the jth connection in the total loss function of the weight parameter set w and the test dataset D, c j represents the auxiliary indicator variable for the j-th connection, represents partial derivative, δ represents a decimal close to 0, e j represents the unit vector used to simulate the disturbance, which is 1 in the jth position and 0 in other positions, and lim represents the limiting process of the parameter δ approaching 0; The loss sensitivity is normalized to represent the connection importance: Among them, s j represents the connection importance of the jth connection, and || represents the absolute value; Each connection is retained according to the connection importance from large to small, wherein the number of retained connections is the expected weight density level.

8. The method for autonomous highway inspection by unmanned aerial vehicle according to claim 1, characterized in that: The abnormal scenarios include traffic congestion, traffic accidents, pedestrians on the highway, illegal parking and road damage.

9. A highway drone autonomous inspection system, characterized in that: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method for autonomous highway drone inspection as claimed in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for autonomous highway drone inspection as described in any one of claims 1 to 8 is implemented.

Images