A drone inspection target recognition algorithm integrating graph neural network and spatial semantics

Abstract

The present invention discloses a drone inspection target recognition algorithm that integrates graph neural networks and spatial semantics, and relates to the field of drone inspection technology. The key points of its technical solution include the following steps: collecting the target inspection area of ​​the drone during the inspection process, dividing and combining the obstacle area, obstacle-free area and identification area in the target inspection area to obtain a first combined area set and a second combined area set; judging the image features of the obstacle area to obtain an obstacle image feature judgment result, and judging the image features of the obstacle-free area to obtain an obstacle-free image feature judgment result; based on the obstacle image feature judgment result and the obstacle-free image feature judgment result, processing and analyzing the target recognition feature quantity and comparison feature quantity of the first combined area set to obtain a first recognition quantity result set and a first comparison quantity result set; the effect is to greatly improve the automation level of inspection and the abnormality recognition capability.

Description

Technical Field

[0001] The present invention relates to the field of drone inspection technology, and more specifically, to a drone inspection target recognition algorithm that integrates graph neural networks and spatial semantics. Background Art

[0002] The widespread application of drone technology in inspection scenarios across sectors such as power, transportation, and security has created an urgent need for accurate target recognition in complex environments. In terms of adaptability to complex scenarios, inspection areas often face obstacles, multi-angle viewing, and drastic lighting changes. Existing graph neural network (GNN) applications are often limited to modeling static regional relationships. For example, graph convolutional networks (GCNs) use image segmentation regions as nodes and spatial distances as edge weights. However, these methods lack explicit encoding of the spatial semantic relationships between obstacles and signs (such as occlusion logic and functional relevance), making it difficult to effectively address spatial associations between obstacles and signs. This can occur, for example, when branches obscure tower signs during power inspections or billboards interfere with signposts during traffic inspections. This leads to incomplete target feature extraction and a significant decrease in recognition accuracy. Existing spatial semantic analysis technologies often rely on handcrafted rules or predefined knowledge bases, but lack dynamic fusion mechanisms with visual features. Furthermore, existing technologies often focus on analyzing single-region features, making it difficult to capture feature dependencies arising from regional interactions, such as the impact of obstacle shadows on background textures or the semantic associations between signs and their surroundings, which in turn affects target recognition accuracy. Summary of the Invention

[0003] In view of the shortcomings of the existing technology, the purpose of the present invention is to provide a drone inspection target recognition algorithm that integrates graph neural networks and spatial semantics.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] A drone inspection target recognition algorithm that integrates graph neural networks and spatial semantics includes the following steps:

[0006] Collect the target inspection area of ​​the drone during the inspection process, and divide and combine the obstacle area, obstacle-free area and marked area in the target inspection area to obtain a first combined area set and a second combined area set;

[0007] The image features of the obstacle area are judged to obtain the obstacle image feature judgment result, and the image features of the obstacle-free area are judged to obtain the obstacle-free image feature judgment result;

[0008] According to the obstacle image feature judgment results and the obstacle-free image feature judgment results, the target recognition feature quantities of the first combined area set are processed and analyzed to obtain a first recognition quantity result set, and the comparison feature quantities of the first combined area set are extracted to obtain a first comparison quantity result set;

[0009] Determine the positions of the obstacle area and the marking area in the second combined area set to obtain a first area position determination result and a second area position determination result;

[0010] Processing and analyzing the target recognition feature quantity and the comparison feature quantity of the second combined area set according to the first area position judgment result and the second area position judgment result to obtain a second recognition quantity result set, a second comparison quantity result set, a third recognition quantity result set, and a third comparison quantity result set;

[0011] According to the first recognition quantity result set, the second recognition quantity result set, the third recognition quantity result set, the first comparison quantity result set, the second comparison quantity result set and the third comparison quantity result set, the target recognition result of the target inspection area is judged to obtain a target recognition warning result.

[0012] Preferably, processing and analyzing the target recognition feature quantity and the comparison feature quantity of the second combined area set according to the first area position judgment result and the second area position judgment result to obtain a second recognition quantity result set, a second comparison quantity result set, a third recognition quantity result set, and a third comparison quantity result set specifically includes the following steps:

[0013] Processing and analyzing the target recognition feature quantity of the second combined area set according to the first area position judgment result to obtain a second recognition quantity result set, and processing and analyzing the comparison feature quantity of the second combined area set according to the first area position judgment result to obtain a second comparison quantity result set;

[0014] The target recognition feature of the second combined area set is extracted according to the second area position judgment result to obtain a third recognition feature result set, and the comparison feature of the second combined area set is extracted according to the second area position judgment result to obtain a third comparison feature result set.

[0015] Preferably, the obstacle area, obstacle-free area and marking area in the target inspection area are divided and combined to obtain a first combined area set and a second combined area set, which specifically includes the following steps:

[0016] Divide the target inspection area into multiple inspection areas;

[0017] Detect the presence of obstacles and signage in the inspection area to obtain an inspection result set;

[0018] Extract obstacle areas, obstacle-free areas and marked areas from the inspection result set;

[0019] Matching and combining adjacent areas of the obstacle area and the obstacle-free area to obtain a first combined area set;

[0020] The obstacle area and the marking area are matched and combined with each other to obtain a second combined area set.

[0021] Preferably, judging the image features of the obstacle area to obtain the obstacle image feature judgment result, and judging the image features of the obstacle-free area to obtain the obstacle-free image feature judgment result, specifically includes the following steps:

[0022] Obtaining first obstacle feature information by counting the coverage area, height, and shape information of obstacles in the obstacle area;

[0023] After collecting texture features, color feature types, and quantity of obstacles in the obstacle area, second obstacle feature information is obtained; wherein the first obstacle feature information and the second obstacle feature information are combined into an obstacle feature information set;

[0024] The image feature status in the obstacle area is judged based on the obstacle feature information set to obtain an obstacle image feature judgment result;

[0025] Detecting background texture features in the obstacle-free area to obtain an obstacle-free background feature information set;

[0026] The image feature status of the obstacle-free area is judged according to the obstacle-free background feature information set to obtain the obstacle-free image feature judgment result.

[0027] Preferably, according to the obstacle image feature judgment result and the obstacle-free image feature judgment result, the target recognition feature quantity of the first combined area set is processed and analyzed to obtain a first recognition quantity result set, which specifically includes the following steps:

[0028] detecting the image clarity and shooting angle of each combined area in the first combined area set to obtain a first image status information set;

[0029] Judging the target recognition feature status of the obstacle-free area in the first combined area set based on the first image status information set and the obstacle-free image feature judgment result to obtain a status judgment result set;

[0030] Extracting target recognition feature quantities from the first combined area set according to the first image status information set and the status judgment result set to obtain a first recognition quantity result set;

[0031] Preferably, processing and analyzing the target recognition feature quantity of the second combined area set according to the first area position judgment result to obtain a second recognition quantity result set, and processing and analyzing the comparison feature quantity of the second combined area set according to the first area position judgment result to obtain a second comparison quantity result set specifically includes the following steps:

[0032] Determine the image feature status of the marked area to obtain a determination result of the marked image feature;

[0033] detecting the image clarity and shooting angle of each combined area in the second combined area set to obtain a second image status information set;

[0034] Determining the positions of the obstacle area and the marking area in the second combined area set according to the shooting angle in the second image status information set to obtain a first area position determination result;

[0035] Based on the first area position judgment result, the marker image feature judgment result, the obstacle image feature judgment result and the situation judgment result set, the target recognition feature quantity in the second combined area set is predicted to obtain a second recognition quantity result set;

[0036] Based on the second recognition quantity result set, the marker image feature judgment result and the obstacle image feature judgment result, the contrast feature quantity in the second combined area set is extracted to obtain a second contrast quantity result set.

[0037] Preferably, judging the positions of the obstacle area and the marking area in the second combined area set according to the shooting angle in the second image status information set to obtain the first area position judgment result specifically includes the following steps:

[0038] If the obstacle area is located in front of the marked area, and the marked area is larger than or equal to the obstacle area, then the first area position determination result is output;

[0039] If the obstacle area is located behind the marked area and the marked area is smaller than the obstacle area, the second area position determination result is output.

[0040] Preferably, judging the target recognition result of the target inspection area based on the first recognition quantity result set, the second recognition quantity result set, the third recognition quantity result set, the first comparison quantity result set, the second comparison quantity result set, and the third comparison quantity result set to obtain the target recognition warning result specifically includes the following steps:

[0041] The first recognition quantity result set, the second recognition quantity result set and the third recognition quantity result set are combined into a target recognition status result set;

[0042] The first comparison amount result set, the second comparison amount result set and the third comparison amount result set are combined into a feature comparison condition result set;

[0043] Based on the target recognition status result set and the feature comparison status result set, the target recognition results of the inspection area are extracted to obtain the target inspection area recognition result set; the target inspection area recognition result set is identified and counted to obtain the inspection area target recognition result;

[0044] Calculate the difference between the inspection area target recognition result and the preset standard recognition result to obtain the recognition difference;

[0045] When the recognition difference is greater than or equal to the preset recognition warning threshold, the target recognition warning result is obtained.

[0046] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, an algorithm for identifying unmanned aerial vehicle inspection targets that integrates graph neural networks and spatial semantics is implemented.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The present invention uses the UAV inspection target recognition algorithm that integrates graph neural networks and spatial semantics to scientifically divide the inspection area into different combination area sets. By using geometric analysis of shooting angles and semantic rule matching, it accurately distinguishes the different positional relationships between obstacles and signs, and adopts differentiated feature processing strategies for different scenarios such as obstacles blocking signs and obstacles serving as backgrounds. This refined position judgment and processing method can more completely extract identification features in complex scenarios. Through the hierarchical combination and collaborative analysis of multi-level result sets, an intelligent recognition and early warning system covering the entire process is constructed, realizing closed-loop processing from feature extraction to abnormal early warning. This mechanism can conduct a comprehensive and systematic analysis of the inspection area, promptly discover potential abnormal conditions and trigger early warnings, greatly improving the automation level and abnormal recognition capabilities of inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This paper proposes a schematic diagram of the steps of a drone inspection target recognition algorithm that integrates graph neural networks and spatial semantics;

[0050] Figure 2 A schematic diagram of the steps for obtaining the first combined area set and the second combined area set in a drone inspection target recognition algorithm that integrates graph neural networks and spatial semantics is proposed in the present invention;

[0051] Figure 3 It is a structural diagram of an electronic device provided by an embodiment of the present invention.

[0052] 610 , processor; 620 , communication interface; 630 , memory; 640 , communication bus. DETAILED DESCRIPTION

[0053] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0054] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0055] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it constitute a separate or selective embodiment that is mutually exclusive with other embodiments.

[0056] Reference Figure 1-Figure 3 shown.

[0057] The embodiment further illustrates the drone inspection target recognition algorithm that integrates graph neural network and spatial semantics proposed in the present invention.

[0058] A drone inspection target recognition algorithm that integrates graph neural networks and spatial semantics includes the following steps:

[0059] Collect the target inspection area of ​​the drone during the inspection process, and divide and combine the obstacle area, obstacle-free area and marked area in the target inspection area to obtain a first combined area set and a second combined area set;

[0060] The image features of the obstacle area are judged to obtain the obstacle image feature judgment result, and the image features of the obstacle-free area are judged to obtain the obstacle-free image feature judgment result;

[0061] According to the obstacle image feature judgment results and the obstacle-free image feature judgment results, the target recognition feature quantities of the first combined area set are processed and analyzed to obtain a first recognition quantity result set, and the comparison feature quantities of the first combined area set are extracted to obtain a first comparison quantity result set;

[0062] Determine the positions of the obstacle area and the marking area in the second combined area set to obtain a first area position determination result and a second area position determination result;

[0063] Processing and analyzing the target recognition feature quantity and the comparison feature quantity of the second combined area set according to the first area position judgment result and the second area position judgment result to obtain a second recognition quantity result set, a second comparison quantity result set, a third recognition quantity result set, and a third comparison quantity result set;

[0064] According to the first recognition quantity result set, the second recognition quantity result set, the third recognition quantity result set, the first comparison quantity result set, the second comparison quantity result set and the third comparison quantity result set, the target recognition result of the target inspection area is judged to obtain a target recognition warning result.

[0065] After the drone of this application collects the target inspection area during the inspection process, the area needs to be finely processed. First, the obstacle area, obstacle-free area and marking area in the target inspection area are identified. The reason for distinguishing these areas is that areas of different properties play different roles in target recognition. Then, through the segmentation and combination rules, the obstacle area and the obstacle-free area are adjacently matched to form a first combination area set, which is used for subsequent analysis of the relationship between obstacles and background environment; the obstacle area and the marking area are adjacently matched to obtain the second combination area set.

[0066] For areas with obstacles, features are extracted from multiple dimensions. Basic spatial information such as the area, height, and shape of the obstacles is collected. Furthermore, visual information such as texture features (such as surface roughness or smoothness) and color features, including type and quantity, is collected. This information together forms an obstacle feature information set. Based on this information, the image characteristics of the obstacle-prone areas are determined to obtain the obstacle image feature judgment result. For areas without obstacles, background texture features are detected to form an obstacle-free background feature information set. This information is then used to determine the image characteristics of the obstacle-free areas and obtain the obstacle-free image feature judgment result.

[0067] After obtaining the image feature determination results for obstacle and obstacle-free areas, the image clarity and shooting angle of each combined area in the first combined area set (combination of obstacle areas and obstacle-free areas) are first tested, as these factors affect feature extraction and recognition. The target recognition characteristics of the obstacle-free areas are then determined based on the obstacle-free image feature determination results. Based on this information, target recognition feature quantities are extracted from the first combined area set to form a first recognition quantity result set. Simultaneously, the obstacle feature information set and the obstacle image feature determination results are used to extract comparative feature quantities for the first combined area set, forming the first comparative quantity result set.

[0068] For the second combined area set (obstacle area + marker area combination), the positional relationship between the obstacle area and the marker area is determined. For example, the obstacle area may be in front of the marker area with a different range, or behind the marker area with a different range, etc., to obtain the position determination results of the first and second areas.

[0069] Finally, the first, second, and third identification result sets, as well as the first, second, and third comparison result sets, are integrated. These result sets reflect the identification of the target inspection area from different area combinations and feature dimensions. These results are then combined to extract the identification results of the target inspection area and statistically analyzed. The identification difference is then compared with the standard recognition results to calculate the identification difference. If the difference is greater than or equal to the preset identification warning threshold, an identification anomaly is determined and a target identification warning result is output. This effectively identifies and warns of anomalies in drone inspection targets, ensuring timely detection of target-related issues during inspections.

[0070] The target recognition feature quantity and the comparison feature quantity of the second combined area set are processed and analyzed according to the first area position judgment result and the second area position judgment result to obtain a second recognition quantity result set, a second comparison quantity result set, a third recognition quantity result set, and a third comparison quantity result set, specifically including the following steps:

[0071] Processing and analyzing the target recognition feature quantity of the second combined area set according to the first area position judgment result to obtain a second recognition quantity result set, and processing and analyzing the comparison feature quantity of the second combined area set according to the first area position judgment result to obtain a second comparison quantity result set;

[0072] The target recognition feature of the second combined area set is extracted according to the second area position judgment result to obtain a third recognition feature result set, and the comparison feature of the second combined area set is extracted according to the second area position judgment result to obtain a third comparison feature result set.

[0073] The first and second region position determination results of this application reflect the spatial relationship between the obstruction area and the marker area. For example, the first region position determination result is output when the obstruction area is located in front of the marker area and the marker range has a specific relationship with the obstacle range (e.g., the marker range is greater than or equal to the obstacle area); the second region position determination result is output when the obstruction area is located behind the marker area and the marker range is smaller than the obstacle area.

[0074] After obtaining the first area location determination results, the target recognition feature quantities of the second combined area set are first processed and analyzed. Specifically, the features of the sign area are corrected and enhanced by combining image condition information such as image clarity and shooting angle, as well as the obstacle image feature determination results and the obstacle-free area condition determination result set. For example, if the determination result shows that an obstacle partially obscures the sign, the features of the obscured portion are predicted using a graph neural network model and combined with the features of the visible portion to form a complete target recognition feature quantity, ultimately generating the second recognition quantity result set.

[0075] When generating the second comparison result set, the results of the marker image feature determination and the obstacle image feature determination are combined to extract feature quantities for comparison with the standard feature library. For example, the degree of interference of the obstacle on the marker features (such as color mixing and texture deformation) is analyzed. The interfered portions are separated from the original features, while retaining the stable features of the marker itself. This second comparison result set is then used for subsequent comparison with the standard marker features to assess recognition reliability.

[0076] When the second area position determination result is obtained (the obstacle is located behind the sign and the sign range is smaller than the obstacle), the focus is on extracting the independent features of the sign area against the complex background. In this case, the obstacle may interfere with the sign features as part of the background, so spatial semantic analysis is required to separate the sign from the background. Specifically, a background model is constructed using the shooting angle information and the texture and color characteristics of the obstacle. A differential algorithm is then used to extract the unique features of the sign to generate the third set of recognition results. This process effectively suppresses background interference and highlights the key features of the sign.

[0077] The third comparison result set is generated by analyzing the differences between the sign features and the standard features based on the third recognition result set. Taking into account factors such as lighting variations and shadows caused by obstacles behind the sign, the extracted features are normalized and matched against feature templates from the standard feature library under multiple angles and lighting conditions. The most representative comparison features are extracted to form the third comparison result set. This result set is used to evaluate the recognition accuracy of signs in complex backgrounds.

[0078] The second recognition metric result set, the second comparison metric result set, the third recognition metric result set, and the third comparison metric result set reflect the characteristic conditions of the sign area from different perspectives. The second set of results (second recognition metric and second comparison metric) focuses on handling occlusion scenarios when obstacles are in front of the sign, improving recognition accuracy through prediction and correction. The third set of results (third recognition metric and third comparison metric) focuses on background interference scenarios when obstacles are behind the sign, improving recognition robustness through background separation and feature normalization. These four result sets complement each other and provide comprehensive and reliable feature basis for the final target recognition and warning, enabling the algorithm to accurately identify targets and assess risk levels in complex spatial relationships.

[0079] The obstacle area, obstacle-free area, and marking area in the target inspection area are divided and combined to obtain a first combined area set and a second combined area set, specifically comprising the following steps:

[0080] Divide the target inspection area into multiple inspection areas;

[0081] The target inspection area is divided into multiple inspection areas based on terrain boundaries and facility boundaries. Relying on geographic information system data, terrain boundaries such as mountains and rivers, as well as facility boundaries such as roads, buildings, and fences, geographically independent inspection areas are segmented to ensure the spatial semantic integrity of inspection targets (such as power towers and markers) within the area, facilitating subsequent feature extraction.

[0082] Detect the presence of obstacles and signage in the inspection area to obtain an inspection result set;

[0083] Extract obstacle areas, obstacle-free areas and marked areas from the inspection result set;

[0084] Matching and combining adjacent areas of the obstacle area and the obstacle-free area to obtain a first combined area set;

[0085] The obstacle area and the marking area are matched and combined with each other to obtain a second combined area set.

[0086] Based on the drone inspection's coverage and identification requirements, the large inspection area is broken down into smaller, more easily analyzed sub-areas. Image recognition technology (such as deep learning models) is used to analyze the image content of each inspection area. The presence of obstacles (such as buildings, trees, and foreign objects) and specific markings (such as tower markings for power inspections and road signs for traffic inspections) are identified. The inspection results for each area, including the presence of obstacles and markings, are aggregated to form a set of inspection results.

[0087] Based on the previously obtained inspection result set, we filter by regional attributes and group areas with the same characteristics. We then determine the spatial relationship between the areas and identify adjacent areas with and without obstacles for combination. In actual inspection scenarios, the interaction between the features of obstacles and the surrounding obstacle-free background (such as shadows and environmental contrast) is crucial for target recognition. Through adjacent matching, we associate these two types of areas to form the first combined area set. Subsequently, we use graph neural networks to mine the feature associations between obstacles and the background environment, thereby assisting in identifying obstructed targets.

[0088] Adjacent obstacle and sign areas are matched based on spatial location. During inspections, obstacles may block and affect sign recognition, such as a power sign obscured by trees. Combining these two adjacent areas creates a second combined area set. This can then be combined with spatial semantics to determine the spatial interference relationship between the obstacle and the sign (e.g., occlusion range and positional impact), helping to accurately identify the sign target and providing a basis for determining whether the obstacle poses a threat to the sign's functionality.

[0089] The image features of the obstacle area are judged to obtain the obstacle image feature judgment result, and the image features of the obstacle-free area are judged to obtain the obstacle-free image feature judgment result, which specifically includes the following steps:

[0090] Obtaining first obstacle feature information by counting the coverage area, height, and shape information of obstacles in the obstacle area;

[0091] After collecting texture features, color features, types, and quantities of obstacles in the obstacle area, second obstacle feature information is obtained; wherein the first obstacle feature information and the second obstacle feature information are combined into an obstacle feature information set;

[0092] The image feature status in the obstacle area is judged based on the obstacle feature information set to obtain an obstacle image feature judgment result;

[0093] Detecting background texture features in the obstacle-free area to obtain an obstacle-free background feature information set;

[0094] The image feature status of the obstacle-free area is judged according to the obstacle-free background feature information set to obtain the obstacle-free image feature judgment result.

[0095] The images obtained by drone inspections use image segmentation, contour extraction and other technologies to locate the scope of obstacles in the area, calculate their coverage area (such as by converting pixel ratio), and combine the spatial scale of the image to obtain the obstacle height. At the same time, the shape is identified (whether it is a regular rectangle, circle, or irregular shape). This information constitutes the first obstacle feature information, which depicts the basic form of the obstacle from a macroscopic spatial dimension.

[0096] Use feature extraction in computer vision to determine whether the obstacle surface is rough or smooth; through color space conversion (such as from RGB to HSV), count the types (how many main tones) and quantity (the proportion of each tones) of color features, and supplement the feature description of the obstacle from a microscopic visual dimension.

[0097] The collected multi-dimensional features are matched against a pre-set obstacle feature library (constructed through extensive sample training and containing characteristic patterns for different obstacle types). For example, if a region's feature information is concentrated across a large area, has an irregular shape, a rough texture, and a single color, the obstacle can be identified by comparison with the library as a tree. If the area is small, the shape is regular, and the color matches the properties of metal, the obstacle can be identified as a small device component. This feature matching and logical analysis outputs the obstacle image feature judgment result, clarifying the attributes of the target within the obstacle area.

[0098] Although there are no obvious physical obstacles in the obstacle-free area, the background texture (such as the texture of the ground, walls, and sky) contains scene information. Texture detection (such as Gabor filtering and wavelet transform) is used to extract background texture patterns, such as the texture of floor tiles and wall paint, to construct an obstacle-free background feature information set and capture the visual patterns of the background.

[0099] The detected background texture features are also matched against a pre-set background feature library (covering background texture patterns in different scenarios, such as urban roads, open spaces, and building rooftops). For example, if the background texture feature set exhibits a regular grid pattern with uniform colors, the library comparison will identify it as an obstacle-free area in an artificial scene, such as a road or building wall. If the texture is cluttered and the colors change naturally, it may be identified as an obstacle-free area in a natural scene, such as a grassland or woodland. This helps clarify the scene attributes of the obstacle-free area.

[0100] According to the obstacle image feature judgment results and the obstacle-free image feature judgment results, the target recognition feature quantities of the first combined area set are processed and analyzed to obtain a first recognition quantity result set, which specifically includes the following steps:

[0101] detecting the image clarity and shooting angle of each combined area in the first combined area set to obtain a first image status information set;

[0102] Judging the target recognition feature status of the obstacle-free area in the first combined area set based on the first image status information set and the obstacle-free image feature judgment result to obtain a status judgment result set;

[0103] Extracting target recognition feature quantities from the first combined area set according to the first image status information set and the status judgment result set to obtain a first recognition quantity result set;

[0104] UAV inspection imaging is affected by flight conditions and ambient lighting. Image clarity (measured by edge sharpness and noise level) and shooting angle (such as overhead, side, and oblique angles) can directly interfere with feature extraction and recognition. Image quality assessment algorithms are used to measure clarity, and shooting angles are determined using camera parameters and attitude data (or through image perspective transformation analysis). These fundamental image condition information that influences recognition is aggregated to form the first image condition information set.

[0105] The background features of the obstacle-free area are significantly affected by image clarity and shooting angle. For example, a blurred image can make it difficult to accurately extract background texture features, and a side-on shot can cause perspective distortion of the background. The obstacle-free image feature judgment results (e.g., whether the background scene attributes are road, grass, etc.) are corrected and verified in conjunction with the first image condition information set. For example, if the shooting angle causes the background texture to stretch and distort, the features need to be restored based on the principle of perspective. If the clarity is low, the background features need to be enhanced or denoised. The target recognition features of the obstacle-free area (e.g., whether the background is pure and whether there are weak interference features) are usable and need further optimization. This ultimately forms a condition judgment result set.

[0106] The first combined region set focuses on the relationship between areas with obstacles and areas without obstacles. The target recognition feature quantity needs to integrate the characteristics of both and the image conditions. Based on the first image condition information set (for example, high definition and orthogonal angles prioritize complete feature extraction; low definition and oblique angles prioritize robust features), combined with the condition judgment result set (whether the features of the obstacle-free area are stable), features that can represent the essence of the target are extracted from the combined region. For example, if the obstacle is a foreign object on the power transmission line and the obstacle-free area is the sky background, if the image is clear and the shooting angle is appropriate, the shape, color features, and edge contrast features with the sky background of the foreign object are extracted; if the image is blurred, the focus is on extracting anti-interference features such as the texture and contour of the foreign object. These features are integrated into the first recognition quantity result set for subsequent target recognition and matching.

[0107] Comparison features are used to compare against a standard feature library (such as obstacle and background features in normal inspection scenarios) to determine if the recognition result is abnormal. The obstacle feature information set (covering the obstacle's spatial form and visual attributes) and the obstacle image feature judgment results (obstacle attribute categories) are used to extract features from the first combined area set for comparison. For example, if the obstacle is a bird (based on feature judgment), its coverage area and color features are extracted and compared with the standard bird feature library; if the obstacle is a hanging object, its texture and shape features are extracted and compared with the standard hanging object feature library.

[0108] The target recognition feature quantity of the second combined area set is processed and analyzed according to the first area position judgment result to obtain a second recognition quantity result set, and the comparison feature quantity of the second combined area set is processed and analyzed according to the first area position judgment result to obtain a second comparison quantity result set, specifically comprising the following steps:

[0109] Determine the image feature status of the marked area to obtain a determination result of the marked image feature;

[0110] detecting the image clarity and shooting angle of each combined area in the second combined area set to obtain a second image status information set;

[0111] Determining the positions of the obstacle area and the marking area in the second combined area set according to the shooting angle in the second image status information set to obtain a first area position determination result;

[0112] Based on the first area position judgment result, the marker image feature judgment result, the obstacle image feature judgment result and the situation judgment result set, the target recognition feature quantity in the second combined area set is predicted to obtain a second recognition quantity result set;

[0113] Based on the second recognition quantity result set, the marker image feature judgment result and the obstacle image feature judgment result, the contrast feature quantity in the second combined area set is extracted to obtain a second contrast quantity result set.

[0114] This application determines the image feature status of the identification area, and determines the integrity, clarity and feature stability of the identification through texture analysis, color clustering, shape matching and other technologies, and generates the identification image feature judgment result. For example, if the identification is partially blocked, the blocked feature area and its impact degree will be identified. At the same time, the image clarity and shooting angle of each combined area in the second combined area set are detected to obtain the second image status information set. In this step, the image clarity is quantified by methods such as the Laplace operator variance, and the shooting angle is combined with the drone posture data and the image perspective relationship to solve. The two together provide basic constraints for subsequent position judgment.

[0115] Based on the shooting angle in the second image status information set, the system performs geometric reasoning on the spatial relationship between the obstruction area and the sign area. When shooting from an overhead angle, the system calculates the projection ratio and relative position of the two areas in the image. This is combined with pre-set spatial semantic rules (e.g., "If the obstacle area's projection covers at least 30% of the sign area's projection and is located in front of it, it is considered partially blocked") to output the position determination result for the first area. For example, if a bird's nest (an obstacle) on a power line is detected in front of a tower sign and partially blocks the text, this spatial blockage relationship can be identified.

[0116] In scenarios where obstacles block the view, a graph neural network is used to simulate the effect of occlusion on sign features, and the features of the unobstructed portion are predicted based on spatial position relationships. For example, if the first region position determination result indicates that the lower portion of the sign is blocked by an obstacle, the obscured characters or patterns are determined based on the known shape characteristics of the sign and the occlusion ratio, thereby generating a second recognition result set containing the complete sign features.

[0117] When generating the second comparison result set, the second recognition result set is used as a foundation, combined with the image feature judgment results of the sign and obstacle to extract key features for comparison with the standard feature library. Stable features such as the sign's color distribution and texture pattern are extracted, while also considering the degree of interference from obstacles on these features (such as color mixing and texture deformation). The disturbed features are separated from the original features, while retaining the sign's stable features. These comparison features are then matched against feature templates from the standard feature library under multiple angles and lighting conditions to calculate similarity scores, ultimately forming the second comparison result set.

[0118] Determining the positions of the obstacle area and the marking area in the second combined area set according to the shooting angle in the second image status information set to obtain a first area position determination result specifically includes the following steps:

[0119] If the obstacle area is located in front of the marked area, and the marked area is larger than or equal to the obstacle area, then the first area position determination result is output;

[0120] If the obstacle area is located behind the marked area and the marked area is smaller than the obstacle area, the second area position determination result is output.

[0121] In this application, the drone's shooting angle (e.g., overhead, sideways, vertical, etc.) directly affects the projection of an area within the image. First, the shooting angle is converted into a projection transformation matrix using the camera's intrinsic parameters (focal length, optical center coordinates) and extrinsic parameters (drone attitude angle, position coordinates). For example, when the drone shoots at a 30° angle, a rectangular sign on the ground will be projected as a trapezoid in the image, while the projected position and size of obstacles ahead will vary with distance.

[0122] When an obstacle is detected in front of the marker area, the two areas are compared (pixel area or actual physical size). If the marker area is larger than or equal to the obstacle area, it indicates that there is partial spatial overlap or possible occlusion between the two. For example, in a power inspection image, the projection of a bird's nest (obstacle) covers the upper left portion of the tower marker (marker area), and the marker's pixel area is larger than the bird's nest. In this case, based on the "front + range difference" rule, the first area position judgment result is output, indicating "The obstacle is in front of the marker, which may cause partial occlusion." This judgment logic combines the spatial front-to-back relationship with size comparison to avoid misjudging small obstacles at a distance as occlusion sources.

[0123] If the obstacle area is located behind the sign area, pay attention to the size relationship between the sign area and the obstacle area. If the sign area is smaller than the obstacle area, it indicates that the obstacle may be present as background and its actual size is larger than the sign. For example, during a road inspection, a large billboard (obstacle) in the distance is located behind the traffic sign (sign area), and the actual area of ​​the billboard is much larger than the sign. In this case, based on the "behind + smaller sign area" rule, the second area position judgment result is output, indicating "the obstacle is located behind the sign and may cause background interference."

[0124] The shooting angle can cause perspective distortion in the regional projection, which requires geometric correction. For example, when shooting from the side, the projection sizes of an obstacle and a sign at the same distance in the image may be significantly different. The image pixel coordinates are converted into world coordinates using a perspective transformation formula (such as the pinhole imaging model) to calculate the actual spatial distance and size of the two. If, after correction, it is found that the actual position of the obstacle is indeed in front of the sign and there is spatial overlap between the two, the first regional position judgment result is confirmed; if the obstacle is actually behind the sign and the physical size of the sign is smaller than the obstacle, the second regional position judgment result is triggered.

[0125] According to the first recognition quantity result set, the second recognition quantity result set, the third recognition quantity result set, the first comparison quantity result set, the second comparison quantity result set, and the third comparison quantity result set, the target recognition result of the target inspection area is judged to obtain a target recognition warning result, which specifically includes the following steps:

[0126] The first recognition quantity result set, the second recognition quantity result set, and the third recognition quantity result set are combined into a target recognition status result set;

[0127] The first comparison amount result set, the second comparison amount result set, and the third comparison amount result set are combined into a feature comparison condition result set;

[0128] Based on the target recognition status result set and the feature comparison status result set, the target recognition results of the inspection area are extracted to obtain the target inspection area recognition result set; the target inspection area recognition result set is identified and counted to obtain the inspection area target recognition result;

[0129] Calculate the difference between the inspection area target recognition result and the preset standard recognition result to obtain the recognition difference;

[0130] When the recognition difference is greater than or equal to the preset recognition warning threshold, the target recognition warning result is obtained.

[0131] This application combines the first, second, and third recognition result sets to form a target recognition status result set. These three result sets correspond to the feature extraction results of different regional combination scenes. The first recognition quantity focuses on the associated features of obstacles and backgrounds, the second recognition quantity focuses on the feature prediction of obstacles blocking signs, and the third recognition quantity focuses on the independent features of signs under background interference.

[0132] The first, second, and third comparison result sets are combined to form the feature comparison result set. Each set of comparison values ​​is differentially matched against the standard feature library. The first comparison value is based on basic obstacle feature comparisons, the second comparison value addresses differences in signature features under occlusion, and the third comparison value focuses on feature deviations under background interference. This combined result set forms a feature difference map covering the entire scene, providing a quantitative basis for anomaly detection.

[0133] Based on these two sets of comprehensive results, the algorithm extracts target recognition results for each inspection area through a weighted fusion strategy. For example, for obstacle areas, the algorithm combines the shape and texture features in the target recognition results set with the deviation of the standard obstacle features in the feature comparison results set to determine whether they are abnormal objects. For identification areas, the algorithm determines whether they are blocked or damaged based on the matching degree between the integrity features in the identification results set and the identification features in the comparison results set.

[0134] The recognition results for all inspection areas are statistically analyzed to generate target recognition results for the inspection area. The first metric is the overall scene recognition accuracy (e.g., number of correctly identified obstacles divided by total number of detections), and the second metric is the key target anomaly rate (e.g., percentage of obstructed signs). For example, during a power line inspection, this step can count the number of identified foreign obstacles and the percentage of tower signs obscured by trees to provide a comprehensive understanding of the inspection area.

[0135] The target recognition results for the inspection area are compared with the preset standard recognition results. These standard recognition results are based on historical normal inspection data and include feature baseline values ​​for various scenarios (e.g., feature templates for unobstructed signs and obstacle thresholds for normal routes). The recognition difference is calculated by calculating the difference between the two (e.g., feature matching difference, number of abnormal targets difference).

[0136] When the recognition difference is greater than or equal to the preset recognition warning threshold, a target recognition warning result is output. Threshold settings are scenario-specific. For example, in traffic inspections, the sign recognition difference threshold is set at 15% (i.e., an alert is triggered when the sign feature match falls below 85%); in power inspections, the foreign object obstacle number difference threshold is set at 3 (i.e., an alert is triggered when three or more new foreign objects are detected). This differentiated threshold mechanism ensures warning accuracy and adaptability to specific scenarios.

[0137] An electronic device includes a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, it implements an unmanned aerial vehicle inspection target recognition algorithm that integrates graph neural networks and spatial semantics.

[0138] like Figure 3 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute a drone inspection target recognition algorithm that integrates graph neural networks and spatial semantics.

[0139] Furthermore, the logic instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as a standalone product, stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for causing a computer device (such as a personal computer, server, or network device) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0140] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute a drone inspection target recognition algorithm that integrates graph neural networks and spatial semantics.

[0141] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute a drone inspection target recognition algorithm that integrates graph neural networks and spatial semantics.

[0142] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0143] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A drone inspection target recognition algorithm that integrates graph neural networks and spatial semantics, characterized by: The following steps are involved: Collect the target inspection area of ​​the drone during the inspection process, and divide and combine the obstacle area, obstacle-free area and marked area in the target inspection area to obtain a first combined area set and a second combined area set; The image features of the obstacle area are judged to obtain the obstacle image feature judgment result, and the image features of the obstacle-free area are judged to obtain the obstacle-free image feature judgment result; According to the obstacle image feature judgment results and the obstacle-free image feature judgment results, the target recognition feature quantities of the first combined area set are processed and analyzed to obtain a first recognition quantity result set, and the comparison feature quantities of the first combined area set are extracted to obtain a first comparison quantity result set; Determine the positions of the obstacle area and the marking area in the second combined area set to obtain a first area position determination result and a second area position determination result; Processing and analyzing the target recognition feature quantity and the comparison feature quantity of the second combined area set according to the first area position judgment result and the second area position judgment result to obtain a second recognition quantity result set, a second comparison quantity result set, a third recognition quantity result set, and a third comparison quantity result set; According to the first recognition quantity result set, the second recognition quantity result set, the third recognition quantity result set, the first comparison quantity result set, the second comparison quantity result set and the third comparison quantity result set, the target recognition result of the target inspection area is judged to obtain a target recognition warning result.

2. The UAV inspection target recognition algorithm integrating graph neural network and spatial semantics according to claim 1 is characterized in that: The target recognition feature quantity and the comparison feature quantity of the second combined area set are processed and analyzed according to the first area position judgment result and the second area position judgment result to obtain a second recognition quantity result set, a second comparison quantity result set, a third recognition quantity result set, and a third comparison quantity result set, specifically including the following steps: Processing and analyzing the target recognition feature quantity of the second combined area set according to the first area position judgment result to obtain a second recognition quantity result set, and processing and analyzing the comparison feature quantity of the second combined area set according to the first area position judgment result to obtain a second comparison quantity result set; The target recognition feature of the second combined area set is extracted according to the second area position judgment result to obtain a third recognition feature result set, and the comparison feature of the second combined area set is extracted according to the second area position judgment result to obtain a third comparison feature result set.

3. The UAV inspection target recognition algorithm integrating graph neural network and spatial semantics according to claim 2 is characterized in that: The obstacle area, obstacle-free area, and marking area in the target inspection area are divided and combined to obtain a first combined area set and a second combined area set, specifically comprising the following steps: Divide the target inspection area into multiple inspection areas; Detect the presence of obstacles and signage in the inspection area to obtain an inspection result set; Extract obstacle areas, obstacle-free areas and marked areas from the inspection result set; Matching and combining adjacent areas of the obstacle area and the obstacle-free area to obtain a first combined area set; The obstacle area and the marking area are matched and combined with each other to obtain a second combined area set.

4. The UAV inspection target recognition algorithm integrating graph neural network and spatial semantics according to claim 3 is characterized in that: The image features of the obstacle area are judged to obtain the obstacle image feature judgment result, and the image features of the obstacle-free area are judged to obtain the obstacle-free image feature judgment result, which specifically includes the following steps: Obtaining first obstacle feature information by counting the coverage area, height, and shape information of obstacles in the obstacle area; After collecting texture features, color feature types, and quantity of obstacles in the obstacle area, second obstacle feature information is obtained; wherein the first obstacle feature information and the second obstacle feature information are combined into an obstacle feature information set; The image feature status in the obstacle area is judged based on the obstacle feature information set to obtain an obstacle image feature judgment result; Detecting background texture features in the obstacle-free area to obtain an obstacle-free background feature information set; The image feature status of the obstacle-free area is judged according to the obstacle-free background feature information set to obtain the obstacle-free image feature judgment result.

5. The UAV inspection target recognition algorithm integrating graph neural network and spatial semantics according to claim 4 is characterized in that: According to the obstacle image feature judgment results and the obstacle-free image feature judgment results, the target recognition feature quantities of the first combined area set are processed and analyzed to obtain a first recognition quantity result set, which specifically includes the following steps: detecting the image clarity and shooting angle of each combined area in the first combined area set to obtain a first image status information set; Judging the target recognition feature status of the obstacle-free area in the first combined area set based on the first image status information set and the obstacle-free image feature judgment result to obtain a status judgment result set; According to the first image status information set and the status judgment result set, the target recognition feature quantity in the first combined area set is extracted to obtain a first recognition quantity result set.

6. The UAV inspection target recognition algorithm integrating graph neural network and spatial semantics according to claim 5 is characterized in that: The target recognition feature quantity of the second combined area set is processed and analyzed according to the first area position judgment result to obtain a second recognition quantity result set, and the comparison feature quantity of the second combined area set is processed and analyzed according to the first area position judgment result to obtain a second comparison quantity result set, specifically comprising the following steps: Determine the image feature status of the marked area to obtain a determination result of the marked image feature; detecting the image clarity and shooting angle of each combined area in the second combined area set to obtain a second image status information set; Determining the positions of the obstacle area and the marking area in the second combined area set according to the shooting angle in the second image status information set to obtain a first area position determination result; Based on the first area position judgment result, the marker image feature judgment result, the obstacle image feature judgment result and the situation judgment result set, the target recognition feature quantity in the second combined area set is predicted to obtain a second recognition quantity result set; Based on the second recognition quantity result set, the marker image feature judgment result and the obstacle image feature judgment result, the contrast feature quantity in the second combined area set is extracted to obtain a second contrast quantity result set.

7. The UAV inspection target recognition algorithm integrating graph neural network and spatial semantics according to claim 6 is characterized in that: Determining the positions of the obstacle area and the marking area in the second combined area set according to the shooting angle in the second image status information set to obtain a first area position determination result specifically includes the following steps: If the obstacle area is located in front of the marked area, and the marked area is larger than or equal to the obstacle area, then the first area position determination result is output; If the obstacle area is located behind the marked area and the marked area is smaller than the obstacle area, the second area position determination result is output.

8. The UAV inspection target recognition algorithm integrating graph neural network and spatial semantics according to claim 7 is characterized in that: According to the first recognition quantity result set, the second recognition quantity result set, the third recognition quantity result set, the first comparison quantity result set, the second comparison quantity result set, and the third comparison quantity result set, the target recognition result of the target inspection area is judged to obtain a target recognition warning result, which specifically includes the following steps: The first recognition quantity result set, the second recognition quantity result set and the third recognition quantity result set are combined into a target recognition status result set; The first comparison amount result set, the second comparison amount result set and the third comparison amount result set are combined into a feature comparison condition result set; Based on the target recognition status result set and the feature comparison status result set, the target recognition results of the inspection area are extracted to obtain the target inspection area recognition result set; the target inspection area recognition result set is identified and counted to obtain the inspection area target recognition result; Calculate the difference between the inspection area target recognition result and the preset standard recognition result to obtain the recognition difference; When the recognition difference is greater than or equal to the preset recognition warning threshold, the target recognition warning result is obtained.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements a drone inspection target recognition algorithm that integrates graph neural networks and spatial semantics as described in any one of claims 1 to 8.

Images