Visual detection method, system and equipment based on unmanned aerial vehicle

Through the drone shooting initial and detection images, and using visual detection neural network to compare the difference characteristics, the problem of low efficiency of traditional illegal building supervision methods is solved, and efficient and accurate detection and labeling of illegal building is achieved.

CN119580131AInactive Publication Date: 2025-03-07CHINA TELECOM UNMANNED TECHNOLOGY (JIANGSU) CO LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202411628727.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-14
Publication Date
2025-03-07
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional illegal building supervision methods are inefficient, costly and timely. Especially in complex and changeable urban environments, illegal buildings with strong concealment and fast construction speed are difficult to be discovered and effectively dealt with in a timely manner.

Method used

Using a drone-based vision detection method, by obtaining the initial image and detection image taken by the drone at a preset position, performing slice processing and inputting a pre-trained visual detection neural network, directly comparing the differential features between the detection image and the initial image to generate a detection result picture.

Benefits of technology

It improves the efficiency and accuracy of illegal building inspections, can detect and mark illegal building areas faster and more accurately, and reduces the cost and time of manual inspections.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119580131A_ABST
    Figure CN119580131A_ABST
Patent Text Reader

Abstract

The invention provides a visual detection method, system and device based on an unmanned aerial vehicle, and the method comprises the steps: obtaining a first frame image shot by the unmanned aerial vehicle at a preset position as an initial image, and obtaining a frame image shot by the unmanned aerial vehicle at the preset position after the initial image as a detection image; respectively segmenting the initial image and the detection image to obtain an initial image slice and a detection image slice, and sequentially labeling the initial image slice and the detection image slice according to positions; the initial image slices and the detection image slices are sequentially input into a visual detection neural network obtained through pre-training, the visual detection neural network compares difference features between each initial image slice and each detection image slice, and the detection image slices with difference feature labels are output; and combining all the detection image slices with the difference feature marks according to the mark number sequence of the detection image slices to generate a detection result picture.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to a visual detection method, system and device based on drones. Background Art

[0002] Today, as the urbanization process continues to accelerate, urban planning and management face unprecedented challenges. With the advancement of construction technology and the growth of residents' demand for living space, illegal construction has become increasingly prominent, affecting the overall appearance and planning order of the city and posing a potential threat to public safety. Traditional methods of illegal construction supervision mainly rely on manual inspections and satellite remote sensing technology, but these methods have limitations such as low efficiency, high cost, and poor timeliness. Especially in complex and changing urban environments, illegal buildings with strong concealment and fast construction speed are often difficult to be discovered and effectively handled in a timely manner. The rapid development of drone technology has brought new opportunities for urban management. With its advantages of high flexibility, strong maneuverability, and wide field of view, drones have shown great potential in environmental monitoring, disaster assessment, traffic monitoring and other fields. Especially in terms of visual recognition and processing technology, with the integrated application of artificial intelligence technologies such as deep learning and computer vision, drones have been able to perform more complex and high-precision image acquisition and analysis tasks.

[0003] In recent years, some scholars have applied drones to the task of comparing and detecting illegal buildings, which are specifically divided into semantic segmentation methods and direct comparison methods. The Chinese patent with publication number CN115100531A discloses the use of a semantic segmentation model to extract the segmentation highlights of illegal buildings, and compare the two segmentation results before and after to obtain the illegal construction area. The Chinese patent with publication number CN116883843A discloses the construction of an attention neural network to compare whether there are differences between inspection images and historical images. The effect of the illegal construction detection method based on semantic segmentation in the related art depends on the accuracy of the segmentation, and it takes longer than the direct comparison method. The illegal construction detection method based on direct comparison is easily interfered by non-building changes, such as moving vehicles, and cannot directly return to the specific illegal construction area. Summary of the invention

[0004] In view of the above problems, the present invention proposes a visual inspection method, system and equipment based on drones. In a first aspect, an embodiment of the present application provides a visual inspection method based on a drone, the method comprising obtaining a first frame image taken by the drone at a preset position as an initial image, and obtaining a frame image taken at the preset position after the initial image as a detection image; segmenting the initial image and the detection image to obtain initial image slices and detection image slices, respectively, and labeling the initial image slices and the detection image slices in sequence according to their positions; inputting the initial image slices and the detection image slices in sequence into a pre-trained visual inspection neural network, the visual inspection neural network compares the difference features between each initial image slice and the detection image slice, and outputs detection image slices with difference feature annotations; merging all detection image slices with difference feature annotations in the order of the detection image slice labels to generate a detection result picture.

[0005] In some embodiments, before obtaining the first image taken by the drone at a preset position as the initial image, the method also includes: obtaining the original image collected by the drone at the preset position, inputting the original image into an image filtering model, and performing abnormal image filtering on the original image to obtain a primary image; inputting the primary image into an angle detection model, and performing angle filtering on the primary image to obtain the initial image and the detection image.

[0006] In some embodiments, the visual detection neural network includes a backbone network, a detail capturer and a decoder, and the training process of the visual detection neural network includes: the backbone network captures the first feature vector in the initial image slice and the detection image slice; the detail capturer obtains the second feature vector in the initial image slice and the detection image slice; the backbone network also includes a feature injector, and the feature injector combines the second feature vector with the first feature vector through a cross-attention mechanism to obtain a feature vector; the decoder captures the feature vector difference between the initial image slice and the detection image slice, and obtains the difference feature through cascaded convolutional layers and upsampling operations and outputs it.

[0007] In some embodiments, the detail capturer includes a first CBS module, a second CBS module, a first CPPCSPC module, a first CBAM module, a first ELAN module, an MP module, a second ELAN module, a second CPPCSPC module, a second CBAM module and a CNN convolutional layer connected in series.

[0008] In some embodiments, the detail capturer obtains second feature information in the initial image slice and the detection image slice, including: obtaining second feature vectors of 1 / 2, 1 / 4, and 1 / 8 scales from the initial image slice and the detection image slice, respectively.

[0009] In some embodiments, the feature injector includes three cross-attention blocks. Correspondingly, the feature injector combines the second feature vector with the first feature vector through the cross-attention mechanism to obtain a feature vector, including: combining the second feature vector of scales of 1 / 2, 1 / 4, and 1 / 8 with the first feature vector to obtain a feature vector.

[0010] In some embodiments, the first feature vector is used as the query vector, and the dot product is performed on the second feature vectors of 1 / 2, 1 / 4, and 1 / 8 scales of the key and value vectors through a cross-attention mechanism to obtain a feature vector; or, the first feature vector is used as the key and value vector, and the dot product is performed on the second feature vector of 1 / 2, 1 / 4, and 1 / 8 scales of the query vector through a cross-attention mechanism to obtain a feature vector.

[0011] In a second aspect, an embodiment of the present application provides a visual inspection system based on a drone, the system comprising: an acquisition module, used to acquire a first frame image taken by the drone at a preset position as an initial image, and to acquire a frame image taken at the preset position after the initial image as a detection image; a slicing processing module, used to slice the initial image and the detection image respectively to obtain initial image slices and detection image slices, and to number the initial image slices and the detection image slices in sequence according to their positions; a visual inspection neural network, used to compare the difference features between each initial image slice and the detection image slice, and output a detection image slice with the difference feature annotations; a merging module, used to merge all the detection image slices with the difference feature annotations in the order of the detection image slice labels to generate a detection result picture.

[0012] The present invention proposes a visual inspection method based on a drone, which directly compares the difference between the detection image and the initial image through a visual recognition neural network to improve the visual inspection efficiency. The present invention optimizes the visual recognition neural network through a detail capturer and a feature injector to improve the accuracy of the detected detail features and outputs the results visually. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic diagram of the flow of the drone-based visual inspection method provided in Example 1 of the present application; Figure 2 A schematic diagram of a visual detection neural network provided in Example 1 of the present application; Figure 3 A structural diagram of a CBS module provided in an embodiment of the present application; Figure 4 A structural diagram of the SPC module provided in the embodiment of the present application; FIG5 is a structural diagram of a CBAM module provided in an embodiment of the present application; Figure 6A structural diagram of an ELAN module provided in an embodiment of the present application; Figure 7 A structural diagram of the MP module provided in an embodiment of the present application; FIG8 is a schematic diagram of a feature injector provided in Example 1 of the present application; Fig. 9 This is a picture of the test results provided in Example 2 of the present application. DETAILED DESCRIPTION

[0014] The specific implementation of the present invention will be described in more detail below in conjunction with the schematic diagram. The advantages and features of the present invention will become clearer based on the following description. It should be noted that the drawings are all in a very simplified form and are not in exact proportions, and are only used to facilitate and clearly assist in explaining the purpose of the embodiments of the present invention.

[0015] Terminology Note: CBS: Cross Stage Binary, mainly used to enhance the feature expression ability of convolutional neural networks.

[0016] SPPCSPC: Spatial Pyramid Pooling, Cross Stage Partial Channel, which consists of two parts: Spatial Pyramid Pooling (SPP) and Cross Stage Partial Channel (CSPC).

[0017] CBAM: Convolutional Block Attention Module, which represents the attention mechanism module of the convolutional module, is an attention mechanism module that combines spatial and channel.

[0018] ELAN: Extended efficient layer aggregation networks.

[0019] MP: Max Pooling, its function is to perform maximum pooling operation.

[0020] CNN: Convolutional Neural Networks, convolutional neural network.

[0021] Cat: concatenation layer, which means concatenating the feature maps processed by multiple modules in the channel dimension.

[0022] Embodiment 1 Embodiment 1 of the present application provides a visual detection method based on a drone, Figure 1 The flowchart of the visual inspection method based on drone provided in Example 1 of the present application is shown in FIG. Figure 1 , the method comprising: Step S101: obtaining a first frame of image taken by the drone at a preset position as an initial image, and obtaining a frame of image taken at the preset position after the initial image as a detection image; In an embodiment of the present application, multiple drones are used to perform aerial photography to collect ground data, such as taking images, or drones take ground videos and extract them into frames of images. The drone takes images at a preset position that needs to be detected, such as an area where it is necessary to detect whether there are illegal buildings. Two frames of images taken successively at the same preset position are used as the initial image and the detection image, respectively. The format of the initial image and the detection image can be TIF format. By directly comparing the difference between the initial image and the detection image, it is determined whether there are illegal buildings in the image.

[0023] In the embodiment of the present application, before step S101, the method further includes: Step S110: Obtain the original image collected by the drone at a preset position, input the original image into the image filtering model, and filter the original image for abnormal images to obtain a primary image; the image filtering model here is a binary classifier based on yolov5. The original image is input into the image filtering model to remove blurred and abnormal images, thereby improving the accuracy of subsequent algorithms.

[0024] Step S120: Input the primary image into the angle detection model, and perform angle filtering on the primary image to obtain the initial image and the detection image. The drone is equipped with sensors and gyroscopes, so as to obtain the tilt angle of the image taken by the drone in each azimuth. The angle detection model performs angle filtering by judging whether the tilt angle of each azimuth meets the preset conditions. For example, only orthophoto images are retained, and images with a tilt angle greater than 5° in a certain azimuth are deleted to reduce subsequent judgment errors.

[0025] Step S201: Slice the initial image and the detection image to obtain initial image slices and detection image slices respectively, and label the initial image slices and the detection image slices in sequence according to their positions.

[0026] In an embodiment of the present application, the initial image slices and the detection image slices are first segmented to obtain I×J initial image slices and I×J detection image slices, respectively, where I and J are arbitrary positive integers. Here, the initial image and the detection image are segmented, and the image can be segmented into any number of slices, and the large image can be divided into multiple smaller image blocks, each of which retains part of the original image, but the volume is significantly reduced, so that it is easy to process separately. It not only reduces the computational burden of a single processing, but also provides the possibility for subsequent parallel processing of neural networks. Through the parallel computing framework, multiple image blocks can be processed on different processors or computing nodes at the same time, greatly shortening the overall processing time.

[0027] After the segmentation process is completed, the initial image slices and the detection image slices are numbered in sequence according to their positions. Here, the numbering is performed from left to right and from top to bottom. For example, the number of the first image slice in the upper left corner is (1, 1), and the number of the last image block in the lower right corner is (i, j), where i and j are the number of slices of the image block in the horizontal (width) and vertical (height) directions, respectively.

[0028] Step S301: inputting the initial image slice and the detection image slice into a pre-trained visual detection neural network in sequence, the visual detection neural network compares the difference features between each initial image slice and the detection image slice, and outputs the detection image slice with the difference feature annotation.

[0029] Figure 2 For a schematic diagram of a visual detection neural network provided in an embodiment of the present application, see Figure 2 The visual detection neural network 200 includes a backbone network 201, a detail capturer 202 and a decoder 203. The training process of the visual detection neural network includes: Step S310: The backbone network captures the first feature vector in the initial image slice and the detection image slice. Here, the initial image slice and the detection image slice obtained after the slicing process are divided into a training set according to a certain ratio, and the training process of the visual detection neural network is carried out. The backbone network 201 in the embodiment of the present application is a ViT backbone network. The ViT backbone network performs vector mapping on the input initial image slice and the detection image slice, and reduces the feature resolution of each initial image slice and the detection image slice to 1 / 16 of the original image. The first feature vector F v It is a large-scale spatial feature used to capture the initial image slice and detect large-scale changes in image slices.

[0030] Step S320: The detail capturer 202 obtains the second feature vector in the initial image slice and the detection image slice. Here, the detail capturer is introduced to obtain the second feature vector F c, the second eigenvector F c It is a detailed spatial feature, that is, the detail variation feature that the ViT backbone network cannot capture.

[0031] In the embodiment of the present application, the detail capturer 202 includes a first CBS module, a second CBS module, a first CPPCSPC module, a first CBAM module, a first ELAN module, an MP module, a second ELAN module, a second CPPCSPC module, a second CBAM module and a CNN convolutional layer connected in series.

[0032] Figure 3 For the structural diagram of the CBS module provided in the embodiment of the present application, see Figure 3 ,The CBS module includes a convolution layer (CNN), a normalization layer (BN) and an activation function sigmoid. The function of the CBS module is to perform feature extraction.

[0033] Figure 4 For the structural diagram of the SPC module provided in the embodiment of the present application, see Figure 4 The SPPCSPC module consists of two branches. The first branch passes through a CBS module with a convolution kernel of 1x1. The second branch first passes through three CBS modules with convolution kernels of 1×1, 3×3, and 1×1, and then passes through the maximum pooling layers of scales 5, 9, and 13. The results of different pooling layers are spliced ​​and passed through two CBS modules with convolution kernels of 1×1 and 3×3, and then spliced ​​with the CBS module of the first branch. The function of the SPPCSPC module is to increase the sensitivity of the network so that the network can adapt to pictures of different scales.

[0034] FIG5 is a structural diagram of a CBAM module provided in an embodiment of the present application, see Figure 5a ,The CBAM module consists of channel attention and spatial attention. Its network structure is connected in series with these two modules, aiming to enhance the feature expression ability of convolutional neural networks. Figure 5b , the channel attention module captures the dependencies between channels through global average pooling (AvgPool) and maximum pooling (MaxPoor) operations to generate two channel descriptors. These two descriptors are then transformed through a shared fully connected layer MLP and ReLU activation function, and then through a Sigmoid function to obtain channel attention weights. These weights are multiplied with the original feature map to achieve feature recalibration in the channel dimension. See Figure 5c The spatial attention module focuses on the spatial position information of the feature map. It first performs average pooling and maximum pooling operations on the feature map in the channel dimension to generate two spatial descriptors. These two descriptors are fused through a convolution layer and then the spatial attention weights are obtained through the Sigmoid function. These weights are multiplied with the original feature map to achieve feature recalibration of the spatial position.

[0035] Figure 6 For a structural diagram of the ELAN module provided in the embodiment of the present application, see Figure 6 The ELAN module consists of two branches. The first branch passes through a CBS module with a convolution kernel of 1×1. The second branch first passes through a CBS module with a convolution kernel of 1×1, and then passes through four CBS modules with convolution kernels of 3×3. Then the first branch and the second branch are spliced. The role of the ELAN module is to enable the network to learn more features and introduce a residual network structure to improve the robustness of the network.

[0036] Figure 7 For the structural diagram of the MP module provided in the embodiment of the present application, see Figure 7 The MP module has two branches. The first branch first passes through the maximum pooling layer (MaxPool), and then passes through a CBS module with a convolution kernel of 1×1. The second branch first passes through a CBS module with a convolution kernel of 1×1, and then passes through a CBS module with a convolution kernel of 1×1 and a sliding step of 2. Then the first branch and the second branch are spliced. The function of the MP module is to downsample the features.

[0037] Combination Figure 2 In an embodiment of the present application, the detail capturer takes the detection image slice and the initial image slice as input, first passes through the first CBS module and the second CBS module to extract the second feature information, and then passes through the first SPPCSPC module, the first CBAM module, the first ELAN module, the MP module, the second ELAN module, the second SPPCSPC module, and the second CBAM module in sequence to realize the second feature information labeling, and finally adjusts the latitude of the second feature information outputted finally through a 1×1 CNN convolutional layer and outputs it.

[0038] The detail capturer obtains the second feature information in the initial image slice and the detection image slice, including: obtaining the second feature vectors of 1 / 2, 1 / 4, and 1 / 8 scales from the initial image slice and the detection image slice, respectively, represented as F c1 、F c2 、F c3 .

[0039] Step S330: The backbone network 201 further includes a feature injector 204, which combines the second feature vector with the first feature vector through a cross attention mechanism to obtain a feature vector. The feature injector includes three cross attention blocks, which respectively inject the second feature vector F at 1 / 2, 1 / 4, and 1 / 8 scales. c1 、F c2 、F c3 With the first eigenvector F vCombined to get the eigenvector F vE Here, the feature injector is used to efficiently integrate fine-grained spatial information into high-level semantic learning, combining the high-level features of ViT with the detailed features through a cross-attention mechanism.

[0040] FIG8 is a schematic diagram of a feature injector provided in an embodiment of the present application. Step S330 is implemented in the following two ways: Figure 8a , Method 1: The first eigenvector F v As the query vector, the dot product of the key and value vectors is used as the second feature vector (i.e., F) at 1 / 2, 1 / 4, and 1 / 8 scales through the cross attention mechanism. c1 、F c2 、F c3 ). See Figure 8b , Method 2: The first eigenvector F v As the key and value vectors, the dot product is used as the second feature vector (i.e., F) of the query vector at 1 / 2, 1 / 4, and 1 / 8 scales through the cross attention mechanism. c1 、F c2 、F c3 )F vE Then, after a 2D depth convolution layer FC with a kernel size of 1×1, the feature vector F is obtained. vE Here, the input vector is multiplied by the parameter weight matrices Wq, Wk, and Wv to obtain three matrices Q, V, and K, and finally the final eigenvector F is obtained by calculation. vE .

[0041] It can be understood that in the embodiment of the present application, in the training of the visual detection neural network, it also includes applying a multi-scale feature fusion decoder to predict the probability map of changes, and calculating the loss function, so as to improve the training process of the visual detection neural network.

[0042] Step S340: The decoder captures the feature vector difference between the initial image slice and the detection image slice, obtains the difference feature through cascaded convolution layers and upsampling, and outputs it. Here, the decoder gradually aggregates the difference features through cascaded convolution layers and upsampling operations, and finally restores it to the original resolution.

[0043] Step S401: merge all detection image slices with difference feature annotations in the order of the detection image slice labels to generate a detection result image.

[0044] Here, the detection result image slices are placed one by one on a new blank image in the order of the numbers in step S201. For example, when placing, the detection result of the (1, 1) image slice is placed in the upper left corner. After the merging is completed, the detection result image is combined. In an application scenario, the illegal construction information in the detection image is marked. Fig. 9 The detection result picture provided in the embodiment of the present application is obtained based on the above-mentioned detection picture and the initial picture, and the specific location, shape and scope of the illegal building are clearly marked in the picture.

[0045] The drone-based visual inspection method provided in the embodiment of the present application directly compares the difference between the detection image and the initial image through a visual recognition neural network, thereby improving the efficiency of visual inspection. The present invention optimizes the visual recognition neural network through a detail capturer and a feature injector, thereby improving the accuracy of the detected detail features and visually outputting the results.

[0046] Example 2 Example 2 of the present application provides a visual inspection system based on a drone, and the system includes: an acquisition module, used to acquire a first frame image taken by the drone at a preset position as an initial image, and acquire a frame image taken at the preset position after the initial image as a detection image; a slice processing module, used to slice the initial image and the detection image respectively to obtain initial image slices and detection image slices, and number the initial image slices and the detection image slices in sequence according to their positions; a visual inspection neural network, used to compare the difference features between each initial image slice and the detection image slice, and output the detection image slice with the difference feature annotations; a merging module, used to merge all the detection image slices with the difference feature annotations in the order of the detection image slice labels to generate a detection result picture.

[0047] In some embodiments, the system also includes an image filtering model and an angle detection model, which are respectively used to perform abnormal image filtering and angle filtering on the original image collected by the drone at a preset position to obtain the initial image and the detection image.

[0048] In some embodiments, the visual detection neural network includes a backbone network, a detail capturer and a decoder, wherein the backbone network is used to capture a first feature vector in an initial image slice and a detection image slice; the detail capturer is used to obtain a second feature vector in an initial image slice and a detection image slice; the backbone network also includes a feature injector, which is used to combine the second feature vector with the first feature vector through a cross-attention mechanism to obtain a feature vector; the decoder is used to capture the feature vector difference between the initial image slice and the detection image slice, and obtain and output the difference feature through cascaded convolutional layers and upsampling operations.

[0049] Embodiment 3 Embodiment 3 of the present application provides a computer device, the computer device includes a memory and a processor, the memory stores a computer program, the memory may include, for example, a flash memory, a hard disk memory, a multimedia card micro memory, a card memory (for example, an SD card or an XD memory), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a PROM, a magnetic memory, a magnetic disk, and an optical disk. When the processor executes the computer program, the above-mentioned visual detection method based on a drone is implemented.

[0050] Embodiment 4 Embodiment 4 of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the above-mentioned drone-based visual detection method can be implemented.

[0051] The above description is only the preferred embodiment of the present invention and is not intended to limit the present invention. Any technician familiar with the profession may make various changes, modifications, substitutions and variations to these embodiments without departing from the principle and purpose of the present invention. The scope of the present invention is defined by the claims and their equivalents.

Claims

1. A visual inspection method based on drone, characterized in that: The method comprises: Acquire a first frame of image taken by the drone at a preset position as an initial image, and acquire a frame of image taken at the preset position after the initial image as a detection image; The initial image and the detection image are respectively segmented to obtain initial image slices and detection image slices, and the initial image slices and the detection image slices are numbered in sequence according to their positions; The initial image slice and the detection image slice are sequentially input into a pre-trained visual detection neural network, the visual detection neural network compares the difference features between each initial image slice and the detection image slice, and outputs the detection image slice with the difference feature annotation; All detection image slices with difference feature annotations are merged in the order of their label numbers to generate a detection result image.

2. The method according to claim 1, characterized in that Before obtaining a first image taken by the drone at a preset position as an initial image, the method further includes: Obtaining an original image collected by the drone at a preset position, inputting the original image into an image filtering model, and performing abnormal image filtering on the original image to obtain a primary image; The primary image is input into an angle detection model, and angle filtering is performed on the primary image to obtain the initial image and the detection image.

3. The method according to claim 2, characterized in that The visual detection neural network includes a backbone network, a detail capturer and a decoder. The training process of the visual detection neural network includes: The backbone network captures the initial image slice and detects the first feature vector in the image slice; The detail capturer acquires the initial image slice and detects the second feature vector in the image slice; The backbone network also includes a feature injector, which combines the second feature vector with the first feature vector through a cross attention mechanism to obtain a feature vector; The decoder captures the feature vector difference between the initial image slice and the detection image slice, obtains the difference feature through cascaded convolutional layers and upsampling operations, and outputs it.

4. The method according to claim 3, characterized in that The detail capturer includes a first CBS module, a second CBS module, a first CPPCSPC module, a first CBAM module, a first ELAN module, an MP module, a second ELAN module, a second CPPCSPC module, a second CBAM module and a CNN convolutional layer which are connected in series in sequence.

5. The method according to claim 4, characterized in that The detail capturer obtains the initial image slice and detects the second feature information in the image slice, including: The second eigenvectors of 1 / 2, 1 / 4, and 1 / 8 scales are obtained from the initial image slice and the detection image slice, respectively.

6. The method according to claim 5, characterized in that The feature injector includes three cross-attention blocks. Correspondingly, the feature injector combines the second feature vector with the first feature vector through the cross-attention mechanism to obtain a feature vector, including: combining the second feature vector of scales of 1 / 2, 1 / 4, and 1 / 8 with the first feature vector to obtain a feature vector.

7. The method according to claim 6, characterized in that The first feature vector is used as the query vector, and the feature vector is obtained by using the cross attention mechanism to obtain the second feature vector with a scale of 1 / 2, 1 / 4, and 1 / 8 of the key and value vectors respectively; Alternatively, the first feature vector is used as the key and value vector, and the feature vector is obtained by taking the dot product as the second feature vector of scale 1 / 2, 1 / 4, and 1 / 8 of the query vector through the cross attention mechanism.

8. A visual inspection system based on drones, characterized in that: The system comprises: An acquisition module is used to acquire a first frame of image taken by the drone at a preset position as an initial image, and acquire a frame of image taken at the preset position after the initial image as a detection image; A slicing processing module, used to slice the initial image and the detection image to obtain initial image slices and detection image slices, and to label the initial image slices and the detection image slices in sequence according to their positions; A visual inspection neural network is used to compare the difference features between each initial image slice and the inspection image slice, and output the inspection image slice with the difference feature annotations; The merging module merges all the detection image slices with difference feature annotations in the order of the detection image slice labels to generate a detection result image.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the drone-based visual detection method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the drone-based visual detection method according to any one of claims 1 to 7 can be implemented.

Citation Information

Patent Citations

  • Attention mechanism-based river channel illegal building detection method

    CN116883843A

  • Automatic illegal building monitoring method based on image and realization system thereof

    CN105893972A

  • Unmanned-aerial-vehicle identification system used for city illegal-building supervision

    CN108009507A

  • Unmanned aerial vehicle small target detection method and system and storable medium

    CN114067225A

  • Illegal building inspection and measurement method and system based on unmanned aerial vehicle vision

    CN115100531A