Substation equipment identification method, system and device for image identification unmanned aerial vehicle route planning

Abstract

The invention discloses a transformer substation equipment identification method and system for image identification unmanned aerial vehicle route planning, and the method comprises the following operation steps: obtaining a layout plan of a transformer substation, and constructing a three-dimensional plan of transformer substation equipment for the layout plan of the transformer substation; generating an unmanned aerial vehicle flight route according to the three-dimensional plane graph; generating an unmanned aerial vehicle flight plan for the unmanned aerial vehicle flight route; when an unmanned aerial vehicle control system of the unmanned aerial vehicle flies according to the unmanned aerial vehicle flight plan, a transformer substation equipment image is collected according to the position of the transformer substation equipment in the three-dimensional planar graph; identifying the substation equipment from the substation equipment image and identifying a to-be-identified target object of the substation equipment at the target orientation; the to-be-identified target object is an electrical element on the substation equipment. According to the invention, the equipment image of the substation equipment is acquired through the unmanned aerial vehicle flight plan of the unmanned aerial vehicle, and the type of the substation equipment and the fault type when the fault occurs are further identified according to the equipment image.

Description

Technical Field

[0001] The present invention relates to the field of substation equipment identification, and in particular to a substation equipment identification method, system and device for image recognition UAV route planning. Background Art

[0002] With the rapid development of the power industry, substations are key nodes for power transmission and distribution, and their safe and efficient operation is crucial to ensuring the stability of the power grid. Traditional substation inspection methods mainly rely on manual inspections, which are not only inefficient, but also have high safety risks and are easy to miss. Especially in the complex and changeable substation environment, manual inspections are difficult to fully and timely discover equipment failures or potential safety hazards.

[0003] In recent years, drone technology has been widely used in the field of power inspection due to its advantages of high efficiency, flexibility and safety. Drones can carry a variety of sensors such as high-definition cameras, infrared thermal imagers, and lidar to conduct non-contact and long-distance inspections of substation equipment, greatly improving the efficiency and accuracy of inspections. However, to fully realize the potential of drones in substation inspections, key technical issues such as drone route planning and accurate equipment identification must be solved.

[0004] Most existing drone route planning methods are based on two-dimensional plane diagrams, ignoring the actual layout and height information of substation equipment in three-dimensional space, resulting in the drone being unable to accurately avoid obstacles during flight, posing a risk of collision. At the same time, two-dimensional planning methods cannot fully utilize the advantages of drone flight altitude and visual coverage, making it difficult to achieve comprehensive and efficient inspection of substation equipment.

[0005] In terms of equipment recognition, traditional image recognition methods often rely on manually set features and template matching. For complex and changeable substation equipment images, the recognition accuracy and robustness are low. With the rapid development of deep learning technology, image recognition methods based on deep learning models such as convolutional neural networks (CNN) have demonstrated strong learning and generalization capabilities, and can automatically learn feature representations from large amounts of data to achieve accurate recognition of substation equipment. However, how to effectively apply these advanced equipment image recognition technologies to drone inspection systems and combine them with drone route planning is still a technical problem that needs to be solved urgently. Summary of the invention

[0006] The purpose of the present invention is to provide a substation equipment identification method, system and device for image recognition UAV route planning, which solves the above-mentioned technical problems pointed out in the prior art.

[0007] The present invention provides a substation equipment identification method for image recognition UAV route planning, comprising the following steps:

[0008] Obtaining a layout plan of a substation, and constructing a three-dimensional plan of substation equipment based on the layout plan of the substation; and generating a flight route for a drone according to the three-dimensional plan;

[0009] Generate a UAV flight plan for the UAV flight route; when the UAV control system of the UAV flies according to the UAV flight plan, collect substation equipment images according to the positions of the substation equipment in the three-dimensional plane map; identify the substation equipment from the substation equipment image and identify the target object to be identified at the target position of the substation equipment;

[0010] The target object to be identified is an electrical component on the substation equipment.

[0011] Preferably, the UAV flight route is generated according to the three-dimensional plane map, and the specific operation steps are as follows:

[0012] Extracting the three-dimensional coordinates of the substation equipment from the three-dimensional plane map, and determining the three-dimensional coordinates of each substation equipment as the target position;

[0013] The particle swarm optimization algorithm is used to generate the UAV flight route for the target location of the substation equipment.

[0014] Preferably, a UAV flight plan is generated for the UAV flight route, and the specific operation steps are as follows:

[0015] Setting route parameters for the UAV flight route;

[0016] The route parameters include flight altitude parameters, camera angle parameters and drone attitude;

[0017] The drone attitude includes a roll angle, a pitch angle, and a yaw angle;

[0018] Generate a UAV flight plan based on the UAV flight route and the route parameters.

[0019] Preferably, when the drone control system of the drone flies according to the drone flight plan, the substation equipment image is collected according to the position of the substation equipment in the three-dimensional plane map, and the specific operation steps are as follows:

[0020] The substation is equipped with a fixed-point camera, and the fixed-point camera assists the drone to collect images of the substation equipment; the posture of the drone is mapped with the parameter information of the fixed-point camera to create a mapping relationship;

[0021] The parameter information of the fixed point camera includes the position, direction and height of the fixed point camera;

[0022] The drone executes the drone flight plan according to the drone control system, the drone determines the target position of the substation equipment in the drone flight route in the drone flight plan, takes the target position of the substation equipment as the drone flight target point, and determines the coordinates of the drone flight target point;

[0023] The drone is hovered at the coordinates of the drone flight target point according to the route parameters in the drone flight plan;

[0024] When the drone flies to the coordinates of the drone's flight target point, the fixed-point camera of the substation preferentially captures equipment images of the circumferential sector-shaped collection surface of the substation equipment that can be captured;

[0025] Determine, according to the mapping relationship, an area in the flight route of the drone in the flight plan of the drone where the photography of the substation equipment and the monitoring of the fixed point camera overlap;

[0026] The UAV control system controls the UAV to collect equipment images on the remaining circumferential sector-shaped collection surfaces of the substation equipment according to the monitoring overlapping area.

[0027] Preferably, the substation equipment image is identified as the substation equipment, and a substation equipment identification model is constructed; the equipment image is input into the substation equipment identification model, and the type and fault of the substation equipment in the equipment image are output.

[0028] Preferably, a substation equipment identification model is constructed, and the specific operation steps are as follows:

[0029] Use the RetinaNet model to build a substation equipment identification model;

[0030] Install multi-sensor equipment on drones and fixed-point cameras, and set the same recording parameters for drones and fixed-point cameras;

[0031] The recording parameters include the same timestamp and the status of the substation equipment at different time points;

[0032] Collecting a number of multi-angle sample equipment images, annotating the multi-angle sample equipment images with equipment types and fault types, and performing preprocessing to establish a sample data set;

[0033] Dividing the sample data set into a sample training set and a sample test set;

[0034] Extracting visual features of fixed-point cameras and visual features of drone images from the filtered sample data set;

[0035] Fusion of the visual features of the fixed point camera with the visual features of the machine set image to form a joint feature vector;

[0036] A joint loss function is defined for the joint feature vector, and a Focal Loss loss function is used to optimize the sample training set for the equipment type and fault type of the joint feature vector. The definition formula is:

[0037] FL(p t )=-α t ·(1-p t ) γ ·log(p t );

[0038] Where FL is the Focal Loss function that adjusts the equipment type and fault type in the sample training set;

[0039] pt is the probability predicted by the RetinaNet model;

[0040] αt is a balancing factor, which is used to control the weight of samples with high coverage of circumferential targets in the sample training set;

[0041] γ is a regulation factor. By adjusting the value of γ, the RetinaNet model can reduce the focus on easy-to-classify samples.

[0042] Use the optimized sample training set to train the RetinaNet model, and calculate the accuracy, recall rate, and F1-score for the device types and fault types in the optimized sample training set;

[0043] According to the accuracy, recall and calculated F1-score, the hyperparameters of the RetinaNet model are adjusted to obtain a preliminary substation equipment recognition model;

[0044] The preliminary substation equipment identification model is evaluated using a sample test set, and the accuracy of the preliminary substation equipment identification model in identifying equipment types and fault types on the sample test set is recorded to obtain the substation equipment identification model after the final test is completed.

[0045] Preferably, the hyperparameters of the RetinaNet model are adjusted according to the accuracy, recall rate and calculated F1-score to obtain a preliminary substation equipment recognition model. The specific operation steps are as follows:

[0046] Using grid search to construct a grid search object for the hyperparameters, searching the hyperparameters to obtain a hyperparameter grid and a scoring index; using a sample training set for fitting to obtain a hyperparameter grid combination; evaluating the hyperparameter grid combination to obtain an optimal hyperparameter combination;

[0047] The RetinaNet model is retrained according to the optimal hyperparameter combination to obtain the preliminary substation equipment identification model.

[0048] Preferably, the introduction of the circumferential target coverage F to further screen the sample training set specifically includes the following steps:

[0049] Determine, for the multi-angle sample device images in the sample data set, whether the circumferential target coverage F of the target to be identified in the fixed point camera is greater than the minimum limit value FJ of the circumferential target coverage;

[0050] If so, the sample data set with a coverage greater than the minimum limit value FJ of the circumferential target coverage is determined to be a circumferential target high coverage sample, and the sample data set part of the circumferential target high coverage sample is used as a sample training set;

[0051] If not, the sample data set that is less than or equal to the minimum limit value FJ of the circumferential target coverage is determined to be a circumferential target low coverage sample, and the sample data set part of the circumferential target low coverage sample is used as the sample test set.

[0052] Accordingly, the present invention also proposes a substation equipment identification system for image recognition UAV route planning, comprising: a route generation module; an identification module;

[0053] The route generation module is used to obtain a layout plan of the substation, construct a three-dimensional plan of the substation equipment based on the layout plan of the substation; and generate a flight route for the drone based on the three-dimensional plan;

[0054] The identification module is used to generate a UAV flight plan for the UAV flight route; when the UAV control system of the UAV flies according to the UAV flight plan, the substation equipment image is collected according to the position of the substation equipment in the three-dimensional plan; and the substation equipment is identified from the substation equipment image.

[0055] Compared with the prior art, the embodiments of the present invention have at least the following technical advantages:

[0056] By analyzing the above-mentioned substation equipment identification method for image recognition drone route planning provided by the present invention, it can be known that in specific applications, a plan view including the location and size data of all substation equipment is obtained through laser scanning technology and high-resolution aerial photography, and the location of the substation equipment is determined according to the plan view; then the three-dimensional coordinates of the substation equipment are extracted according to the plan view, and a three-dimensional plan view is generated according to the three-dimensional coordinates. The three-dimensional plan view can make the substation equipment a three-dimensional three-dimensional model, so as to generate a flight route for the drone according to the three-dimensional model, and fly in the substation equipment through the flight route of the drone, so as to avoid the dangerous situation of the drone touching obstacles, thereby ensuring the safe flight of the drone and the subsequent collection of images of the substation equipment, so that the captured images are clearer;

[0057] Furthermore, the flight route parameters of the drone are set, and the flight plan of the drone is generated for the flight route of the drone through the parameters of the flight altitude of the route parameters, the shooting angle, and the attitude of the drone. The flight plan of the drone can help the drone better avoid the equipment of the substation, and can also find the substation position of the drone during the flight, and collect images of the substation equipment at an appropriate height; at the same time, the substation is equipped with a fixed-point camera to assist the drone in collecting equipment images. The fixed-point camera collects equipment images by the area that the substation equipment can capture (that is, the circumferential fan-shaped collection surface of the substation equipment captured), which can save the time of the drone collecting substation equipment images; and the image capture of the substation equipment in different directions and heights is mapped to the position and height of the fixed angle of the fixed-point camera through the attitude of the drone, and the overlapping area captured by the drone and the fixed-point camera is determined according to the mapping relationship, and the overlapping area captured is removed, so that the drone collects equipment images on the circumferential fan-shaped collection surface of the substation equipment that has not been captured, thereby reducing the collection time of the drone;

[0058] Furthermore, the substation equipment recognition model is constructed by using the RetinaNet model; first, the substation equipment is collected by using drones and fixed-point monitoring cameras to collect sample equipment images at several angles, and the equipment type and fault type of the substation equipment are annotated on the sample equipment images at several angles, so as to better understand what kind of equipment the substation equipment is and what kind of fault affects the operation of the substation equipment when a fault occurs; the collected sample equipment images at several angles are preprocessed to generate a sample data set, which is then divided into a sample training set and a sample test set; thereby the constructed model is trained and the accuracy of the model is tested; the sample training set is extracted from the machine set image visual features and the fixed-point camera visual features, and a joint feature vector is formed by the two to form a circumferential coverage of the substation equipment, so that the collected substation equipment images are more comprehensive, so as to more clearly reflect the type of substation equipment and the type of fault that occurs in the subsequent process; considering that the substation equipment recognition model is to specifically identify a specific target object (or target position) at a certain position on the substation equipment, when obtaining the training set, both the fixed camera and the drone collect images to obtain the sample training set;

[0059] In practice, the sample training set of some substation equipment itself has an imbalance problem, especially when both fixed cameras and drones collect images to obtain sample training sets (sometimes it is not good to have too many samples with low coverage of circumferential targets); so after introducing the circumferential target coverage F, model training also needs to refer to the change of this factor; so further, a joint loss function is defined for the involution feature vector, and the equipment type and fault type in the sample training set are adjusted through the Focal Loss loss function to optimize the sample training set; at the same time, the balance factor and γ are adjustment factors; among them, αt is the balance factor, which is used to control the weight of samples with high coverage of circumferential targets in the sample training set; γ is the adjustment factor, which is adjusted to reduce the RetinaNet model's attention to easy-to-classify samples (easily identifiable negative samples, substation equipment is in normal state);

[0060] The Focal Loss loss function can be used to solve the problem of sample imbalance in the training set; the RetinaNet model is trained through the optimized sample training set, and the accuracy, recall rate and F1-score of the equipment type and fault type are calculated, so that different types of equipment and fault states can be identified, ensuring the effectiveness and reliability of the final substation equipment identification model in practical applications; and the hyperparameters of the RetinaNet model are adjusted according to the accuracy, recall rate and calculated F1-score to obtain a preliminary substation equipment identification model; the preliminary substation equipment identification model is evaluated through the sample test set, and the accuracy of identifying the equipment type and fault type of the sample test set is evaluated, and the substation equipment identification model after the final test is obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A schematic diagram of the main process of a method for identifying substation equipment by image recognition UAV route planning in Example 1;

[0062] Figure 2 A flowchart of the recognition of drone-collected images in a method for identifying substation equipment by image recognition drone route planning in Example 1;

[0063] Figure 3 A schematic diagram of a circumferential sector-shaped collection surface of a substation equipment collected by a fixed-point camera and a drone in a substation equipment identification method for image recognition drone route planning in Example 1;

[0064] Figure 4 A flowchart of constructing a substation equipment identification model for a substation equipment identification method for image recognition UAV route planning in Example 1;

[0065] Figure 5 A schematic diagram of the circumferential target coverage rate of a substation equipment identification method for image recognition drone route planning in Example 1;

[0066] Figure 6 A flowchart of obtaining a preliminary substation equipment identification model for a substation equipment identification method for image recognition UAV route planning in Example 1;

[0067] Figure 7 This is a flow chart of a substation equipment identification system for image recognition drone route planning in Example 2;

[0068] Figure 8 A schematic diagram of the structure of a storage medium for a substation equipment identification method using the above-mentioned image recognition UAV route planning.

[0069] Reference numerals: route generation module 10; identification module 20; processor 1110; communication interface 1120; memory 1130; computer storage medium 1140. DETAILED DESCRIPTION

[0070] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] The present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings.

[0072] like Figure 1 As shown, the present invention provides a substation equipment identification method for image recognition UAV route planning, comprising the following operating steps:

[0073] S10: Obtaining a layout plan of the substation, constructing a three-dimensional plan of substation equipment based on the layout plan of the substation; and generating a flight route for the drone according to the three-dimensional plan;

[0074] It should be noted that a detailed plan of the substation is obtained by using laser scanning technology or high-resolution aerial photography; the plan should include the location and size data of all major equipment, which are substation equipment, including transformers, circuit breakers, switches, etc.

[0075] The substation layout plan is converted into a 3D model through data fusion technology. Computer vision algorithms are used to combine the information of the 2D plan with the actual size data to generate a 3D model with height information. Structured light scanning or depth cameras are used to further accurately measure the spatial position and geometry of the equipment, thereby constructing an accurate 3D plan.

[0076] Based on the equipment position and 3D structure of the 3D plan, an optimization algorithm is applied to generate the UAV flight route. Route planning needs to consider the density of equipment, the flight altitude of the UAV, the camera's field of view coverage, and possible obstacles. Genetic algorithms or particle swarm optimization algorithms are used to optimize the route to ensure full coverage of all equipment and avoid collision risks.

[0077] Extracting the three-dimensional coordinates of substation equipment (such as transformers, switches, busbars, etc.) from the three-dimensional plane map, and determining the three-dimensional coordinates of each substation equipment as the target position;

[0078] The target position of the substation equipment is used to generate the UAV flight route through the route planning algorithm (i.e. genetic algorithm or particle swarm optimization algorithm, which will not be described in detail); three-dimensional plane diagram: represents the layout of substation equipment (such as transformers, switches, busbars, etc.) in three-dimensional space. Each device has a specific three-dimensional coordinate (x, y, z). The three-dimensional coordinates of each device are extracted from the plane diagram to determine its position in three-dimensional space as the target position; the target positions of the substation equipment are connected to generate the UAV flight route; the UAV flight route may be used by the UAV to avoid obstacles or obstructions of the substation equipment in the substation equipment, thereby ensuring the flight safety of the UAV;

[0079] S20: generating a UAV flight plan for the UAV flight route; when the UAV control system of the UAV flies according to the UAV flight plan, collecting substation equipment images according to the positions of the substation equipment in the three-dimensional plane map; identifying the substation equipment from the substation equipment image and identifying the target object to be identified at the target position of the substation equipment;

[0080] The target object to be identified is an electrical component on the substation equipment.

[0081] The above-mentioned substation equipment is generally a transformer unit or a circuit breaker control cabinet, or a high-voltage distribution equipment unit, etc. Taking the transformer unit as an example, the embodiment of the present invention not only needs to identify the type and fault of the entire transformer unit, but also needs to identify and image a part of the transformer unit at a specific position (such as damage to the circuit breaker or deformation of the magnetic circuit channel core of the transformer). Therefore, the detection of electrical components at the target position is crucial.

[0082] It should be noted that the drone control system executes the mission according to the planned drone flight plan; the drone control system monitors the drone's position and flight status in real time to ensure that the drone flies according to the preset path and adjusts the route in real time to adapt to environmental changes;

[0083] During the flight, the drone is equipped with a high-resolution camera or multi-spectral sensor to collect images of substation equipment in real time; the camera is set to autofocus and high dynamic range shooting to ensure clear images of the equipment under different lighting conditions. The image data is transmitted to the ground control center through real-time image transmission technology;

[0084] The collected equipment images are analyzed by deep learning algorithms. After being trained on a large number of substation equipment images, different types of equipment and their working status can be efficiently identified. The identification results include substation equipment category, location, operating status, etc., and are compared with the three-dimensional plan to verify the accuracy of the identification results.

[0085] Specifically, Figure 2 As shown, in step S20, a drone flight plan is generated for the drone flight route; when the drone control system of the drone flies according to the drone flight plan, the substation equipment image is collected according to the position of the substation equipment in the three-dimensional plane map; the substation equipment is identified from the substation equipment image, and the specific operation steps are as follows:

[0086] S21 sets flight route parameters for the UAV flight route;

[0087] The route parameters include flight altitude parameters, camera angle parameters and drone attitude;

[0088] The drone attitude includes roll angle, pitch angle and yaw angle (i.e., the attitude information of the drone: the roll angle is the rotation angle of the drone body relative to the horizontal plane; the pitch angle is the lifting or lowering angle of the drone head relative to the horizontal plane; the yaw angle is the rotation angle of the drone body relative to the north; the drone controls the flight direction and adjusts the flight position of the drone through the roll angle, pitch angle and yaw angle of the drone attitude);

[0089] Generate a UAV flight plan based on the UAV flight route (the UAV flight route itself is a path) and the route parameters (i.e., the UAV flight plan is the path generated by taking into account the route parameters);

[0090] It should be noted that the UAV flight route enables the UAV to avoid obstacles and equipment in the substation during flight; however, while avoiding obstacles and equipment in the substation, images of the substation equipment must be collected; therefore, it is necessary to add route parameters to the UAV flight route, which include flight altitude, shooting angle parameters, and UAV attitude (i.e., the attitude of the UAV can directional control the shooting angle of the UAV when the UAV takes pictures of the substation equipment, such as: if there is substation equipment on the left side of the UAV, the UAV will adjust its attitude to aim at the substation equipment on the left side to take multi-angle images); in this way, the UAV can complete the task of collecting images of the substation equipment during flight. The UAV needs to hover at a certain height above the substation equipment to avoid collision with the substation equipment and obstacles; and collect images of the entire local substation equipment; flight altitude: according to the height of the substation equipment, set a reasonable flight altitude to ensure clear images (for example: the height of the substation equipment is 5m, and the UAV may need to fly at a height of 10m, which not only ensures the flight safety of the UAV, but also can collect good images of the substation equipment to ensure clear images); shooting angle: adjust the shooting angle of the UAV camera according to the equipment layout to capture the best image;

[0091] By generating a flight plan for the drone, the drone can rationally plan its flight path, reduce the flight time in the air, identify and plan in advance to avoid obstacles (such as buildings, trees, etc.), and reduce the risk of collision; set a safe flight altitude according to the environment and mission requirements to reduce the risk to surrounding personnel and substation equipment; the generated route can ensure that the drone shoots the target at the best shooting angle and altitude, improving the quality of images and data; reasonable planning of the flight plan can ensure that the drone fully covers the target area and avoids missed shots and reshoots.

[0092] S22: The substation is provided with a fixed-point camera, and the fixed-point camera assists the drone in collecting images of the substation equipment;

[0093] It should be noted that the image acquisition of substation equipment can be carried out not only by drones, but also by fixed-point cameras installed in the substation itself, and the images of substation equipment that can be covered by the fixed-point cameras can be collected;

[0094] However, because the fixed-point camera is fixedly installed and the area collected is a circumferential sector in front (and the area collected is the circumferential sector in front), it may not be able to shoot some substation equipment or the circumferential sector collection surface of the substation equipment. For example, the fixed-point camera is installed at a height of 10m, but the height of the substation equipment is 5m, 8m, 10m, and 13m. The fixed-point camera can only collect images of the substation equipment at a height of 8m, and the collection height difference of the substation equipment at the height of 5m and 13m may be too large, resulting in unclear images of the substation equipment or finally causing errors in recognition; and the substation equipment at a height of 10m will have an incomplete circumferential sector collection surface during the collection process, and the image of the entire substation equipment (the entire circular sampling area) cannot be obtained;

[0095] By collecting images of substation equipment with fixed cameras, the time of collecting images of substation equipment by drones can be saved;

[0096] S23: Mapping the attitude of the drone and the parameter information of the fixed-point camera to create a mapping relationship (i.e., the attitude of the drone and the fixed-point camera both include information such as position, direction, and height. Through the spatial position, the drone and the fixed-point camera are in the same space and can be mapped through a coordinate system; the positions of the two can be described by the same coordinate system to determine their relative position relationship; further, through the observation direction, the flight direction of the drone and the observation direction of the camera can affect their field of view; by mapping the attitude of the drone with the direction of the camera, it can be determined whether the fields of view of the two overlap; then through the height influence, the height difference will affect the quality and range of the collected image; by comparing the heights of the drone and the fixed-point camera, their field of view coverage areas at different heights can be determined; so a mapping relationship can be created between the two);

[0097] The parameter information of the fixed point camera includes the position, direction and height of the fixed point camera;

[0098] It should be noted that the parameter information of the fixed-point camera is determined, and the attitude of the drone is mapped to the parameter information of the fixed-point camera to establish a connection; because the attitude of the drone can control the flight direction of the drone and the direction of collecting images of the substation equipment, as well as the substation equipment at different heights. The drone can be turned in all directions to collect images of the surrounding substation equipment; and the parameter information of the fixed-point camera (that is, the position, direction and height of the fixed-point camera) can also collect images of the substation equipment according to the height and direction of the substation equipment, so the two can be mapped to create a mapping relationship, so as to judge the areas that the two can collect (that is, the circumferential fan-shaped collection surface) in the future, and judge the overlap of the areas, so that the drone can quickly collect images of the substation equipment;

[0099] S24: The drone executes the drone flight plan according to the drone control system, and the drone determines the target position of the substation equipment in the drone flight route in the drone flight plan (that is, because the drone flight plan is obtained by setting the route parameters of the drone flight route, and the drone flight route is generated by the three-dimensional plane map (that is, the three-dimensional plane map is generated by determining the three-dimensional coordinates of the substation equipment in the plane map, and the position and size data of the substation equipment can be determined), so the position of the drone substation can be determined, and the substation equipment is determined as the target position), the target position of the substation equipment is used as the drone flight target point, and the coordinates of the drone flight target point are determined;

[0100] The drone is hovered at the coordinates of the drone flight target point according to the route parameters in the drone flight plan;

[0101] It should be noted that before the drone takes off, it is necessary to ensure that all parameters in the flight plan (such as route, flight altitude, and shooting angle) have been correctly set and meet the flight requirements, so as to ensure that the drone can accurately find the exact location of the substation equipment during flight, thereby performing image acquisition of the substation equipment at multiple angles; the drone control system automatically calculates and determines the target position of the substation equipment according to the preset route, which is based on the three-dimensional coordinates extracted from the explanation in S20. By determining the target position of the substation equipment, the drone finds the shooting location, and uses the target position of the substation equipment as the target point to be flown by the drone, and determines the coordinates of the target point to be flown by the drone, and the drone flies to the target point of the coordinates and hovers (that is, the drone first needs to determine the target position of the substation equipment in the drone flight route generated according to the three-dimensional plan, and determines the target position as the destination to be flown by the drone (that is, the drone flight target point), determines the coordinate information of the destination, and lets the drone fly to the coordinate position of the destination, and the drone at the coordinate position of the destination executes the route parameters and hovers);

[0102] S25: When the drone flies to the coordinates of the drone's flight target point, the fixed-point camera of the substation preferentially collects equipment images of the circumferential sector-shaped collection surface of the substation equipment that can be photographed (that is, because the fixed-point camera is fixed, it can only take pictures of a single side, that is, one side of a substation equipment, and cannot take pictures of other sides of the substation equipment, so the drone needs to collect images of the circumferential sector-shaped collection surfaces of the remaining substation equipment, such as Figure 3 shown);

[0103] Determine the overlapping area of ​​the substation equipment shooting and the fixed point camera monitoring in the UAV flight route in the UAV flight plan according to the mapping relationship (that is, determine the overlapping area between the area of ​​substation equipment captured in the UAV flight route and the area of ​​single surface captured by the fixed point camera according to the mapping relationship, and exclude the overlapping area (that is, the overlapping area is because the fixed point camera has already prioritized the camera area that it can shoot (that is, the fixed point camera of the substation equipment can shoot the circumferential fan-shaped collection surface), so the UAV will not collect the circumferential fan-shaped collection surface that the fixed point camera has already captured again), and the UAV shoots and collects images of the remaining three areas of the substation (that is, the three surfaces that the remaining fixed point cameras of the substation equipment have not been able to shoot); at the same time, the overlapping monitoring area needs to know the common shooting area of ​​the fixed point camera and the UAV, so the two need a connection relationship, so that the common shooting area of ​​the two (that is, the overlapping monitoring area) can be understood through the created mapping relationship; otherwise, the UAV and the fixed point camera can only know their own shooting areas, and cannot know the common overlapping monitoring area);

[0104] The drone control system controls the drone to collect equipment images on the remaining circumferential sector collection surface of the substation equipment according to the monitoring overlap area (the remaining circumferential sector collection surface is the sector collection range remaining after the fixed point camera captures the circumferential sector collection surface of the substation equipment);

[0105] It should be noted that when the drone is controlled to fly to the coordinates of the drone's flight target point (i.e., the coordinates of the substation equipment) according to the drone control system, the nearest fixed-point camera is searched, and then the drone and the fixed-point camera perform information exchange "handshake", and the fixed-point camera of the substation will give priority to collecting equipment images of the substation equipment; at the same time, according to the mapping relationship between the drone's posture and the parameters of the fixed-point camera (i.e., the drone), it is judged whether the area where the substation equipment is photographed in the drone's flight route overlaps with the area where the fixed-point camera can photograph and collect the substation equipment, which can reduce the time for the drone to collect the circumferential fan-shaped surface of the collected substation equipment;

[0106] Furthermore, the drone will fly toward the coordinates of the next drone flight target point according to the flight route. During the flight, the fixed-point camera system will transmit the device images of the circumferential fan-shaped collection surface of the substation equipment collected by itself to the drone flight control system of the drone through the network system or Bluetooth connection, thereby saving the drone's hovering time (that is, indirectly reducing the collection time); at the same time, the reason why the drone flight control system still flies toward the coordinates of the next drone flight target point according to the flight route is also because the fixed-point camera will transmit the device images of the substation equipment collected by itself to the drone flight control system of the drone, which can avoid the inability to transmit the device images collected by the fixed-point camera because the fixed-point camera is far away from the operator's control center;

[0107] S26: constructing a substation equipment recognition model; inputting the equipment image into the substation equipment recognition model, and outputting the type and fault of the substation equipment in the equipment image;

[0108] It should be noted that the substation equipment recognition model is constructed through the RetinaNet model, which is a deep learning model for target detection; the substation equipment recognition model is constructed to identify the equipment type and fault type of the equipment image of the substation equipment, so as to achieve the recognition effect. Therefore, the substation equipment recognition model not only needs to identify the type and fault of the current substation equipment, but also needs to further identify its detailed electrical components.

[0109] Specifically, Figure 4 As shown, in step S26, a substation equipment identification model is constructed, and the specific operation steps are as follows:

[0110] S261: Use the RetinaNet model to build a substation equipment identification model;

[0111] Install multi-sensor equipment on drones and fixed-point cameras (i.e., fixed-point cameras can provide stable multi-angle image data, suitable for long-term monitoring; while drones can provide dynamic perspective circumferential fan-shaped image data of substation equipment surfaces to capture the performance of equipment in different environments and conditions), and set the same recording parameters for drones and fixed-point cameras;

[0112] The recording parameters include the same timestamp (i.e., the timestamp is synchronized to ensure that different perspective information of the device is obtained at the same time point. This synchronization can improve the model's sensitivity to changes in device status and enhance the timeliness of fault detection) and the status of substation equipment at different time points (i.e., temperature, humidity);

[0113] Collect a number of multi-angle sample equipment images, annotate the multi-angle sample equipment images with equipment types (i.e., equipment types are transformers, switches, busbars, etc.) and fault types (fault types are normal state, fault state), and perform preprocessing to establish a sample data set;

[0114] The concept of circumferential target coverage F is introduced. The circumferential target coverage F = the range ratio of the target object to be identified within the circumferential sector collection area of ​​the substation equipment that can be photographed by the fixed point camera (it may be 0); for example: there is a circuit breaker at a certain target position in the transformer unit as the target object to be identified, such as Figure 5 As shown, the device type and fault type annotated on several multi-angle sample device images can be understood as identifying circuit breakers or other devices in the transformer unit;

[0115] Preset the minimum limit value FJ of the circumferential target coverage rate;

[0116] Determine, for the multi-angle sample device images in the sample data set, whether the circumferential target coverage F of the target to be identified in the fixed point camera is greater than the minimum limit value FJ of the circumferential target coverage;

[0117] If so, the sample data set greater than the minimum limit value FJ of the circumferential target coverage is determined to be a circumferential target high coverage sample, and the sample data set part of the circumferential target high coverage sample is used as a sample training set (that is, by using the circumferential target high coverage sample as the sample training set, it can be ensured that the model focuses on important targets, and the circumferential target high coverage sample is a sample with high diversity and complexity, which enhances the model's recognition ability for different targets, and pays more attention to the target to be identified, the identification equipment type and the fault type);

[0118] If not, the sample data set that is less than or equal to the minimum limit value FJ of the circumferential target coverage is determined to be a circumferential target low coverage sample, and the sample data set part of the circumferential target low coverage sample is used as a sample test set (that is, the circumferential target low coverage sample is used as a sample test set to improve the recognition rate and accuracy of the model); the above premise is that when the number of samples is small as a whole, the training set and the test set should be reasonably divided, and the sample data set part of the circumferential target high coverage sample is preferentially used as the sample training set;

[0119] It should be noted that the drones and fixed-point cameras are equipped with multi-sensor equipment to collect multi-angle sample equipment images of the substation equipment. The multi-sensor equipment can collect multi-angle sample equipment images at different angles of the substation equipment. The multi-sensor equipment can use infrared cameras, temperature sensors, etc.; the substation not only has drones to collect data, but also many fixed-point monitoring cameras, sample data, and real-time data; fixed-point cameras can provide stable image data, which is helpful to capture changes in equipment over a long period of time; images collected using fixed-point cameras should cover different angles of the equipment to increase the diversity of samples. The shooting angle and interval of the fixed-point camera can be set to ensure that the equipment image can be obtained in different time periods and environmental conditions, and the fault of the substation equipment can be identified. By checking whether the temperature and humidity of the substation equipment are higher than the threshold of normal substation equipment, it can be judged whether there is a fault problem in the substation equipment and the fault of the substation equipment can be identified. At the same time, the substation equipment can be photographed from multiple angles according to the fixed-point monitoring camera, so that the substation equipment can be understood from different directions and the fault of the substation equipment can be identified. The multi-angle sample equipment images are pre-processed to remove the effects of blur and noise caused by shooting and collection, and the data diversity is increased by using technologies such as rotation, flipping, scaling and brightness adjustment to improve the generalization ability of the model and establish a sample data set.

[0120] S262: extracting visual features of fixed-point cameras and visual features of drone images from the filtered sample data set;

[0121] Fusion of the visual features of the fixed point camera with the visual features of the machine set image to form a joint feature vector;

[0122] It should be noted that the concept of circumferential target coverage F is introduced, and the circumferential target coverage F = the range ratio of the target object to be identified (a circuit breaker at the southwest corner of a transformer unit) covered by the circumferential fan-shaped collection surface of the substation equipment that can be photographed by the fixed-point camera; sometimes the circumferential target coverage F is 0, that is, because the position of the circuit breaker is just restricted by the orientation of the fixed-point camera, the circuit breaker at the southwest corner cannot be photographed at all, so the circumferential target coverage F is 0 at this time;

[0123] The visual information in the images and video data collected by drones when performing tasks plays an important role in many application scenarios. These visual features are used in image recognition, analysis and processing technology. UAVs can obtain environmental information, perform tasks, and make autonomous decisions. The visual features of fixed-point cameras represent stable viewing angles and provide common different angles. Therefore, the circumferential target coverage rate F is introduced to determine the proportion of the range of equipment that the substation wants to collect within the circumferential sector shooting range of the fixed-point camera for the substation equipment. The circumferential target coverage rate F is used to determine the coverage range of the circumferential sector for the substation equipment. It is also necessary to introduce the concept of circumferential target coverage rate F to constrain and screen when screening sample training sets.

[0124] Fusion of the visual features of fixed-point cameras and the visual features of drone images can provide the model with richer information. For example, the acquisition of multi-angle images should include all directions of the equipment to ensure that the model can learn the visual features of the equipment at different angles; in the feature fusion stage, the visual features of the fixed-point camera are combined with the visual features of the drone's drone images. This fusion can form a complete spatial view of the substation equipment;

[0125] A number of multi-angle sample equipment images of the substation equipment are collected by using fixed-point cameras and drones. In this way, sample equipment images can be collected from different directions (or different shooting surfaces) of the substation equipment. Then, the different surfaces of the substation equipment can be judged whether there is a fault or what kind of substation equipment it is by marking the equipment type and fault type. Therefore, the visual features of the fixed-point camera (that is, the sample equipment images that can be captured by the fixed-point camera) and the visual features of the machine set image are used to describe the circumferential fan-shaped area collected relative to the equipment during the shooting process, so as to understand the range of the substation equipment to be collected) to form a joint feature vector to reflect the coverage of the collection range of the substation equipment.

[0126] S263: defining a joint loss function for the joint feature vector, using a Focal Loss loss function to optimize a sample training set for the equipment type and fault type of the joint feature vector (i.e., the joint feature vector combines visual features and visual features of the drone's machine set image. These features provide more comprehensive information, enabling the model to more accurately identify the equipment type and fault type. Using a joint loss function (i.e., a Focal Loss loss function) can ensure that the learning of these two features is simultaneously optimized during the training process, thereby improving the overall performance of the model; the joint loss function can simultaneously process multiple tasks (such as equipment type identification and fault state detection), thereby achieving multi-task learning), and the definition formula is:

[0127] FL(p t )=-α t·(1-p t ) γ ·log(p t );

[0128] Where FL is the Focal Loss function that adjusts the equipment type and fault type in the sample training set;

[0129] pt is the probability predicted by the RetinaNet model (for positive samples, pt = p, for negative samples, pt = 1-p);

[0130] αt is a balancing factor, which is used to control the weight of samples with high coverage of circumferential targets in the sample training set;

[0131] γ is an adjustment factor, which is used to reduce the RetinaNet model's attention to easy-to-classify samples (easily identifiable negative samples, substation equipment is in normal state);

[0132] It should be noted that in practice, the sample training sets of some substation equipment themselves have imbalance problems, especially when fixed cameras and drones collect images separately and then use them as sample training sets; therefore, the αt in the FocalLoss loss function is a balance factor, which can adjust the imbalance problem of samples with high coverage of surrounding targets and samples with low coverage of surrounding targets in the samples. Sometimes, it is not good to have too many samples with low coverage of surrounding targets; therefore, a balance factor is introduced in practical applications;

[0133] In addition, in practical applications, positive samples (such as fault states) are often far less than negative samples (normal states); in this case, the traditional cross entropy loss function easily causes the model to tend to predict negative samples and ignore the learning of positive samples;

[0134] The adjustment mechanism of the Focal Loss loss function is to introduce an adjustment factor. The Focal Loss loss function reduces the focus on easy-to-classify samples (negative samples that are easy to identify) and enhances the learning of difficult-to-classify samples (positive samples that are more difficult to identify).

[0135] By reducing the focus on easy-to-classify samples (such as normal state images), the Focal Loss loss function allows the RetinaNet model to focus more on difficult-to-classify fault samples during training. In the fault detection task of substation equipment, the sample training set of the fault state is usually small. Using the Focal Loss loss function can improve the recognition accuracy of the RetinaNet model for these samples.

[0136] In the above formula, pt is the probability that the RetinaNet model predicts a sample as a positive class (positive sample, fault state); if the sample is a true positive sample (for example, a fault state), then pt = p, where p is the probability that the sample output by the RetinaNet model is a positive class; if the sample is a negative sample (for example, a normal state), then pt = 1-p, that is, the probability that the RetinaNet model predicts it as a negative class (negative sample, normal state); αt is a balancing factor, which is used to control the weight of samples with high coverage of circumferential targets in the type sample; the weight of samples with high coverage of circumferential targets in the sample training set is adjusted by the balancing factor; for example, if there are fewer samples with high coverage of circumferential targets, the value of αt can be increased by 0.2, so that the RetinaNet model pays more attention to these samples during training;

[0137] By combining the factors pt, αt and γ, the Focal Loss function is introduced, which enables the RetinaNet model to learn the characteristics of minority class samples more effectively during training, thereby improving the overall detection performance, especially in the field of fault detection.

[0138] S264: Use the optimized sample training set to train the RetinaNet model, calculate the accuracy, recall rate and F1-score of the device type and fault type in the optimized sample training set;

[0139] It should be noted that the RetinaNet model is used to adjust the parameters of the model for the sample training set so that it can recognize different types of equipment and fault conditions; for example, the RetinaNet model will learn how to distinguish between equipment in normal and faulty states to obtain a perfect substation equipment recognition model; during the evaluation, the performance of the model is recorded for each type of equipment (such as transformers, switches, busbars, etc.) and each fault state (such as normal state, faulty state). Specifically, it includes: Accuracy: the proportion of correctly classified samples to the total samples; Recall: the proportion of correctly identified positive samples to all actual positive samples, reflecting the detection ability of the RetinaNet model; F1-score: the harmonic mean of accuracy and recall, which comprehensively considers the accuracy and completeness of the RetinaNet model;

[0140] If the accuracy is high but the recall is low, it means that the RetinaNet model may perform well on normal state samples, but fail to identify faulty states. If the recall is high but the accuracy is low, there may be more false positives. F1-score can help comprehensively evaluate the performance of the model.

[0141] It is emphasized that during the training and evaluation of the RetinaNet model, careful attention should be paid to the recognition capabilities of different equipment types and their fault states to ensure the effectiveness and reliability of the resulting substation equipment recognition model in practical applications;

[0142] S265: adjusting hyperparameters of the RetinaNet model according to the accuracy, recall and calculated F1-score to obtain a preliminary substation equipment recognition model;

[0143] It should be noted that hyperparameters include learning rate, batch size, and number of training rounds; hyperparameters determine the way and speed of RetinaNet model training; for example, the learning rate controls the amplitude of each parameter update, affecting the convergence speed and final performance; by adjusting hyperparameters, the complexity of the RetinaNet model can be balanced to reduce overfitting (good performance on the sample training set but poor performance on the sample test set) or underfitting (the RetinaNet model is too simple and cannot capture the characteristics of the data); after preliminary training, more detailed optimization is performed on the details of the RetinaNet model performance to ensure that the preliminary substation equipment identification model performs best in specific tasks.

[0144] S266: Evaluate the preliminary substation equipment identification model using the sample test set, record the accuracy of the preliminary substation equipment identification model in identifying the equipment type and fault type of the sample test set, and obtain the substation equipment identification model after the final test is completed;

[0145] It should be noted that an independent sample test set is used to evaluate the preliminary trained substation equipment identification model to ensure the application effect of the preliminary substation equipment identification model in real scenarios and record the recognition accuracy of the preliminary substation equipment identification model under different equipment types and fault conditions; the accuracy of the sample test set in identifying equipment types and fault types is judged by setting thresholds, and the substation equipment identification model after the final test is obtained;

[0146] Specifically, Figure 6 As shown, in step S265, the hyperparameters of the RetinaNet model are adjusted according to the accuracy, recall rate and calculated F1-score to obtain a preliminary substation equipment recognition model. The specific operation steps are as follows:

[0147] S2651: constructing a grid search object for the hyperparameters by using grid search, searching the hyperparameters to obtain a hyperparameter grid and a scoring index; fitting using a sample training set to obtain a hyperparameter grid combination; evaluating the hyperparameter grid combination to obtain an optimal hyperparameter combination;

[0148] It should be noted that grid search is an exhaustive method that systematically trains and evaluates the RetinaNet model to find the best performance hyperparameter combination by defining all possible combinations of a set of hyperparameters; constructs a grid search object for the hyperparameter learning rate, batch size, and number of training rounds to search, hyperparameter grid, and scoring indicators;

[0149] Hyperparameter grid and scoring index refers to the definition of a set of hyperparameters and their possible values ​​in machine learning. This grid is used to systematically explore different hyperparameter combinations to find the best RetinaNet model configuration; such as: learning rate: possible values: 0.001, 0.01, 0.1; batch size: possible values: 16, 32, 64; number of training rounds: possible values: 50, 100; combination generation (i.e. hyperparameter combination): (0.001, 16, 50), (0.001, 16, 100), (0.001, 32, 50), etc.; each combination represents a specific hyperparameter configuration, and grid search will train and evaluate each combination; in this way, the best hyperparameter combination can be found to improve the performance of the RetinaNet model;

[0150] When adjusting hyperparameters (i.e., finding the best hyperparameter combination), the accuracy, recall, and calculated F1-score can determine whether the RetinaNet model is overfitting. For example, if the accuracy on the sample training set is high but the recall is low, it may indicate that the RetinaNet model has learned too much on the training data and performs poorly on new data. Therefore, according to different application scenarios, the most appropriate indicators are selected for optimization to ensure that the RetinaNet model can meet the standard of high recognition accuracy in actual applications.

[0151] S2652: Retraining the RetinaNet model according to the optimal hyperparameter combination to obtain the preliminary substation equipment identification model;

[0152] It should be noted that the best hyperparameter combination is the accuracy, recall rate and calculated F1-score obtained through the sample training set, and then the best hyperparameter combination that can improve the model performance is found by adjusting the hyperparameters; therefore, the RetinaNet model can retrain the best hyperparameter combination to improve the training effect and performance of the RetinaNet model, thereby obtaining a preliminary substation equipment recognition model;

[0153] Specifically, in step S266, the step of obtaining the substation equipment identification model after the final test is completed further includes:

[0154] S2661: Integrate the trained substation equipment recognition model into the UAV control system, wherein the substation equipment recognition model recognizes substation equipment and faults based on the equipment image;

[0155] The drone control system provides feedback on the identified substation equipment and faults;

[0156] It should be noted that the trained substation equipment recognition model is installed in the drone control system, and the drone control system determines the type of substation equipment and whether there is a fault (i.e., including the type of fault (i.e., in step S261, it has been explained whether the temperature of the substation equipment is too high) through the installed infrared camera and temperature sensor and other equipment) by analyzing the collected equipment image, and feeds back the obtained analysis results to the technicians for maintenance and other operations;

[0157] Embodiment 2

[0158] like Figure 7 As shown, the present invention also invents a substation equipment identification system for image recognition drone route planning, comprising: a route generation module 10; an identification module 20;

[0159] The route generation module 10 is used to obtain a layout plan of the substation, construct a three-dimensional plan of the substation equipment based on the layout plan of the substation; and generate a flight route for the drone according to the three-dimensional plan;

[0160] The identification module 20 is used to generate a drone flight plan for the drone flight route; when the drone control system of the drone flies according to the drone flight plan, the substation equipment image is collected according to the position of the substation equipment in the three-dimensional plan; the substation equipment is identified from the substation equipment image.

[0161] Embodiment 3

[0162] On the other hand, the third embodiment is based on the method for recognizing and processing dental and maxillofacial deformation based on image scanning provided in the first embodiment of the invention, and further provides a computer storage medium 1140 (hereinafter referred to as storage medium). Figure 8 FIG. 1 is a schematic diagram of a computer storage medium structure framework provided in Embodiment 3 of the present invention, which includes:

[0163] A memory 1130, used for storing computer programs;

[0164] The communication interface 1120 is used to connect the memory 1130 to the processor 1110;

[0165] The processor 1110 is used to execute a computer program to implement a method for identifying and processing dental and maxillofacial deformation based on image scanning, as disclosed in an embodiment of any combination of the above-mentioned implementation modes.

[0166] In summary, the present invention proposes a method for identifying substation equipment by image recognition drone route planning, which uses laser scanning technology and high-resolution aerial photography to obtain a plan view containing the location and size data of all substation equipment, and determines the location of the substation equipment according to the plan view; then, the three-dimensional coordinates of the substation equipment are extracted according to the plan view, and a three-dimensional plan view is generated according to the three-dimensional coordinates. The three-dimensional plan view can make the substation equipment a three-dimensional three-dimensional model, so as to generate a flight route for the drone according to the three-dimensional model, and fly the drone in the substation equipment according to the flight route to avoid the dangerous situation of the drone touching obstacles, thereby ensuring the safe flight of the drone and the subsequent acquisition of images of the substation equipment, so that the captured images are clearer;

[0167] Furthermore, the flight route parameters of the drone are set, and the flight plan of the drone is generated for the flight route of the drone through the parameters of the flight altitude of the route parameters, the shooting angle, and the attitude of the drone. The flight plan of the drone can help the drone better avoid the equipment of the substation, and can also find the substation position of the drone during the flight, and collect images of the substation equipment at an appropriate height; at the same time, the substation is equipped with a fixed-point camera to assist the drone in collecting equipment images. The fixed-point camera collects equipment images by the area that the substation equipment can capture (that is, the circumferential fan-shaped collection surface of the substation equipment captured), which can save the time of the drone collecting substation equipment images; and the image capture of the substation equipment in different directions and heights is mapped to the position and height of the fixed angle of the fixed-point camera through the attitude of the drone, and the overlapping area captured by the drone and the fixed-point camera is determined according to the mapping relationship, and the overlapping area captured is removed, so that the drone collects equipment images on the circumferential fan-shaped collection surface of the substation equipment that has not been captured, thereby reducing the collection time of the drone;

[0168] Furthermore, the substation equipment recognition model is constructed by using the RetinaNet model; first, the substation equipment is collected by using drones and fixed-point monitoring cameras to collect sample equipment images at several angles, and the equipment type and fault type of the substation equipment are annotated on the sample equipment images at several angles, so as to better understand what the substation equipment is and what faults affect the operation of the substation equipment when a fault occurs; the sample equipment images collected at several angles are preprocessed to generate a sample data set, which is then divided into a sample training set and a sample test set; thereby the constructed model is trained and the accuracy of the model is tested; the visual features of the machine image and the visual features of the fixed-point camera are extracted from the sample training set, and a joint feature vector is formed by the two to form a circumferential coverage of the substation equipment, so that the collected substation equipment images are more comprehensive, so as to more clearly reflect the type of substation equipment and the type of fault that occurs in the subsequent process; considering that the substation equipment recognition model is to specifically identify a specific target object (or target position) at a certain position on the substation equipment, when obtaining the training set, both the fixed camera and the drone collect images to obtain the sample training set;

[0169] In practice, the sample training set of some substation equipment itself has an imbalance problem, especially when both fixed cameras and drones collect images to obtain sample training sets (sometimes it is not good to have too many samples with low coverage of circumferential targets); so after introducing the circumferential target coverage F, model training also needs to refer to the change of this factor; so further, a joint loss function is defined for the involution feature vector, and the equipment type and fault type in the sample training set are adjusted through the Focal Loss loss function to optimize the sample training set; at the same time, the balance factor and γ are adjustment factors; among them, αt is the balance factor, which is used to control the weight of samples with high coverage of circumferential targets in the sample training set; γ is the adjustment factor, which is adjusted to reduce the RetinaNet model's attention to easy-to-classify samples (easily identifiable negative samples, substation equipment is in normal state);

[0170] The Focal Loss loss function can be used to solve the problem of sample imbalance in the training set; the RetinaNet model is trained through the optimized sample training set, and the accuracy, recall rate and F1-score of the equipment type and fault type are calculated, so that different types of equipment and fault states can be identified, ensuring the effectiveness and reliability of the final substation equipment identification model in practical applications; and the hyperparameters of the RetinaNet model are adjusted according to the accuracy, recall rate and calculated F1-score to obtain a preliminary substation equipment identification model; the preliminary substation equipment identification model is evaluated through the sample test set, and the accuracy of identifying the equipment type and fault type of the sample test set is evaluated, and the substation equipment identification model after the final test is obtained.

[0171] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. A person skilled in the art may modify the technical solutions described in the above embodiments, or replace part or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying substation equipment for image recognition UAV route planning, characterized in that: The steps include: Obtaining a layout plan of a substation, and constructing a three-dimensional plan of substation equipment based on the layout plan of the substation; and generating a flight route for a drone according to the three-dimensional plan; generating a UAV flight plan for the UAV flight route; when the UAV control system of the UAV flies according to the UAV flight plan, collecting substation equipment images according to the positions of the substation equipment in the three-dimensional plane map; Identify the substation equipment and the target object to be identified at the target position of the substation equipment in the image of the substation equipment; The target object to be identified is an electrical component on the substation equipment.

2. According to the method of claim 1, the substation equipment identification method for image recognition drone route planning is characterized in that: Generate a UAV flight route based on the three-dimensional plane map. The specific steps are as follows: Extracting the three-dimensional coordinates of the substation equipment from the three-dimensional plane map, and determining the three-dimensional coordinates of each substation equipment as the target position; The particle swarm optimization algorithm is used to generate the UAV flight route for the target location of the substation equipment.

3. According to the method of substation equipment identification for image recognition drone route planning as described in claim 2, it is characterized in that: Generate a UAV flight plan for the UAV flight route. The specific steps are as follows: Setting route parameters for the UAV flight route; The route parameters include flight altitude parameters, camera angle parameters and drone attitude; The drone attitude includes a roll angle, a pitch angle, and a yaw angle; Generate a UAV flight plan based on the UAV flight route and the route parameters.

4. According to the method of substation equipment identification for image recognition UAV route planning as described in claim 3, it is characterized in that: When the drone control system of the drone flies according to the drone flight plan, the substation equipment image is collected according to the position of the substation equipment in the three-dimensional plane map. The specific operation steps are as follows: The substation is equipped with a fixed-point camera, and the fixed-point camera assists the drone to collect images of the substation equipment; the posture of the drone is mapped with the parameter information of the fixed-point camera to create a mapping relationship; The parameter information of the fixed point camera includes the position, direction and height of the fixed point camera; The drone executes the drone flight plan according to the drone control system, the drone determines the target position of the substation equipment in the drone flight route in the drone flight plan, takes the target position of the substation equipment as the drone flight target point, and determines the coordinates of the drone flight target point; The drone is hovered at the coordinates of the drone flight target point according to the route parameters in the drone flight plan; When the drone flies to the coordinates of the drone's flight target point, the fixed-point camera of the substation preferentially captures equipment images of the circumferential sector-shaped collection surface of the substation equipment that can be captured; Determine, according to the mapping relationship, an area in the flight route of the drone in the flight plan of the drone where the photography of the substation equipment and the monitoring of the fixed point camera overlap; The UAV control system controls the UAV to collect equipment images on the remaining circumferential sector-shaped collection surfaces of the substation equipment according to the monitoring overlapping area.

5. According to the method of claim 4, the substation equipment identification method for image recognition UAV route planning is characterized in that: Identify the substation equipment from the substation equipment image and build a substation equipment identification model; The device image is input into the substation device recognition model, and the type and fault of the substation device in the device image are output.

6. According to the method of claim 5, the substation equipment identification method for image recognition drone route planning is characterized in that: Construct a substation equipment identification model. The specific steps are as follows: Use the RetinaNet model to build a substation equipment identification model; Install multi-sensor equipment on drones and fixed-point cameras, and set the same recording parameters for drones and fixed-point cameras; The recording parameters include the same timestamp and the status of the substation equipment at different time points; Collecting a number of multi-angle sample equipment images, annotating the multi-angle sample equipment images with equipment types and fault types, and performing preprocessing to establish a sample data set; Dividing the sample data set into a sample training set and a sample test set; The minimum limit value FJ of the circumferential target coverage is preset; the circumferential target coverage F is introduced to further screen the sample training set; Extracting visual features of fixed-point cameras and visual features of drone images from the filtered sample data set; The visual features of the fixed point camera are integrated with the visual features of the machine set image and the surrounding target coverage F to form a joint feature vector; A joint loss function is defined for the joint feature vector, and the FocalLoss loss function is used to optimize the sample training set for the equipment type and fault type of the joint feature vector. The definition formula is: FL(p t )=-a t ·(1-p t ) γ ·log(p t ); Where FL is the Focal Loss function that adjusts the equipment type and fault type in the sample training set; pt is the probability predicted by the RetinaNet model; αt is a balancing factor, which is used to control the weight of samples with high coverage of circumferential targets in the sample training set; γ is a regulation factor. By adjusting the value of γ, the RetinaNet model can reduce the focus on easy-to-classify samples. Use the optimized sample training set to train the RetinaNet model, and calculate the accuracy, recall rate, and F1-score for the device types and fault types in the optimized sample training set; According to the accuracy, recall and calculated F1-score, the hyperparameters of the RetinaNet model are adjusted to obtain a preliminary substation equipment recognition model; The preliminary substation equipment identification model is evaluated using a sample test set, and the accuracy of the preliminary substation equipment identification model in identifying equipment types and fault types for the sample test set is recorded to obtain the substation equipment identification model after the final test is completed.

7. According to the method of claim 6, the substation equipment identification method for image recognition drone route planning is characterized by: According to the accuracy, recall rate and calculated F1-score, the hyperparameters of the RetinaNet model are adjusted to obtain a preliminary substation equipment recognition model. The specific operation steps are as follows: Using grid search to construct a grid search object for the hyperparameters, searching the hyperparameters to obtain a hyperparameter grid and a scoring index; using a sample training set for fitting to obtain a hyperparameter grid combination; evaluating the hyperparameter grid combination to obtain an optimal hyperparameter combination; The RetinaNet model is retrained according to the optimal hyperparameter combination to obtain the preliminary substation equipment identification model.

8. According to the method of claim 7, the substation equipment identification method for image recognition drone route planning is characterized in that: The introduction of the circumferential target coverage F to further screen the sample training set specifically includes the following steps: Determine, for the multi-angle sample device images in the sample data set, whether the circumferential target coverage F of the target to be identified in the fixed point camera is greater than the minimum limit value FJ of the circumferential target coverage; If yes, the sample data set with a coverage greater than the minimum limit value FJ of the circumferential target coverage is determined to be a circumferential target high coverage sample, and the sample data set part of the circumferential target high coverage sample is used as a sample training set; If not, the sample data set that is less than or equal to the minimum limit value FJ of the circumferential target coverage is determined to be a circumferential target low coverage sample, and the sample data set part of the circumferential target low coverage sample is used as the sample test set.

9. A substation equipment identification system for image recognition drone route planning, characterized in that: include: Generate route module; Identify the module; The route generation module is used to obtain a layout plan of the substation, and construct a three-dimensional plan of the substation equipment based on the layout plan of the substation; Generate a flight route for the drone according to the three-dimensional plan; The recognition module is used to generate a UAV flight plan for the UAV flight route; when the UAV control system of the UAV flies according to the UAV flight plan, the substation equipment image is collected according to the position of the substation equipment in the three-dimensional plane map; The substation equipment image is used to identify the substation equipment.

Images