Small target detection data processing method and device based on unmanned aerial vehicle inspection scene

Abstract

The invention provides a small target detection data processing method and device based on an unmanned aerial vehicle inspection scene, and the method comprises the steps: extracting a target object material under the view angle and height of an unmanned aerial vehicle from an unmanned aerial vehicle image, and the target object material comprises the segmentation regions of at least two to-be-detected targets; for each to-be-detected target, performing data enhancement processing on the to-be-detected target according to the attitude feature and the body shape feature of the to-be-detected target and the environment feature in the unmanned aerial vehicle image to obtain a new target object material; synthesizing the new target object material with the unmanned aerial vehicle image according to a target labeling mode to obtain small target detection data, so that a target detection model executes a small target detection task; wherein the target labeling mode comprises at least one of random position labeling and fixed interval labeling. According to the method, the number and quality of small target samples under the view angle of the unmanned aerial vehicle are improved, and then the small target detection accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a method and device for processing small target detection data in an unmanned aerial vehicle inspection scenario. Background Art

[0002] In recent years, with the introduction of the concept of low-altitude economy, low-altitude UAVs have ushered in a broad space for development and played an important role in many application scenarios, such as wetland patrol, forest fire prevention, security patrol, etc. UAVs can effectively replace manual inspections in complex environments. The cameras and intelligent systems they carry can realize real-time processing of captured images and videos, enabling them to have functions such as target detection, recognition, positioning and tracking, which can reduce manpower input and improve operational efficiency.

[0003] Target detection is an important technology in the field of drone applications. Due to the wider field of view, drone images may contain more target objects, and the size of objects is much smaller than that in natural images. Small target objects usually occupy a larger area in the image. For example, the pixel area of ​​a small target is less than 32*32. Since the images in the drone target detection application dataset are mostly collected from videos, and continuous frames with small feature differences are filtered, the data set is small in size, lacks diversity, has a small number of targets, and has poor background diversity. In order to improve the detection accuracy of the target detection model and enhance the generalization ability of the target detection model, various methods need to be applied to increase the data volume.

[0004] In related technologies, traditional data enhancement increases the number and diversity of data samples by randomly transforming the data, thereby improving the generalization ability and robustness of the model. However, due to the small number of actual application scenarios of drones and the limited data that can be actually collected, the image data after data enhancement has little effective information. In addition, in real-time detection application scenarios, drones need to transmit the captured video stream back. Due to the limitation of communication bandwidth, the actual returned image clarity is limited. When there is a lack of sufficient small target data sets that match common applications for drone-based target detection, the model has low detection accuracy for small targets due to the lack of basic data sets. Summary of the invention

[0005] The present invention provides a method and device for processing small target detection data in a drone inspection scenario, so as to solve the problem that the prior art is limited by the small number of drone application scenarios and the limited drone data collected. The sample quality obtained by random transformation data enhancement is low, resulting in low small target detection accuracy. The method and device improve the number and quality of small target samples from the drone's perspective.

[0006] The present invention provides a method for processing small target detection data in a drone inspection scenario, comprising: Extract target object materials from the UAV image at the UAV's perspective and height, where the target object materials include segmentation regions of at least two targets to be detected; For each target to be detected, perform data augmentation processing on the target to be detected according to the pose characteristics, body shape characteristics of the target to be detected, and environmental characteristics in the UAV image to obtain new target object materials; Synthesize the new target object materials with the UAV image according to the target annotation method to obtain small target detection data for the target detection model to perform small target detection tasks; where the target annotation method includes at least one of random position annotation and fixed interval annotation.

[0007] According to a method for processing small target detection data in a UAV inspection scenario provided by the present invention, the performing data augmentation processing on the target to be detected according to the pose characteristics, body shape characteristics of the target to be detected, and environmental characteristics in the UAV image to obtain new target object materials includes: Rotate the target to be detected according to the pose characteristics to obtain the rotated target to be detected; Adjust the image saturation of the rotated target to be detected according to the environmental characteristics to obtain the adjusted target to be detected; Scale the width and height of the adjusted target to be detected respectively according to the body shape characteristics to obtain the new target object materials.

[0008] According to a method for processing small target detection data in a UAV inspection scenario provided by the present invention, the extracting target object materials from the UAV image at the UAV's perspective and height includes: Perform image segmentation on the enhanced region in the UAV image based on an instance segmentation model to obtain the target object materials, and the instance segmentation model is trained based on the YOLOv8-seg network.

[0009] According to a method for processing small target detection data in a UAV inspection scenario provided by the present invention, the synthesizing the new target object materials with the UAV image according to the target annotation method to obtain small target detection data includes: Take the new target object materials as the first layer and the UAV image as the second layer, and superimpose the first layer on the second layer according to the target annotation method to obtain the small target detection data.

[0010] According to a method for processing small target detection data in a UAV inspection scenario provided by the present invention, after obtaining the small target detection data, the method further includes: Using the small target detection data as training samples and the joint loss function as the training loss, iteratively train the SPD-YOLOv8 network to obtain a target detection model for performing small target detection tasks; wherein, the joint loss function is determined based on location loss, classification loss, and confidence loss.

[0011] A method for processing small target detection data based on an unmanned aerial vehicle (UAV) inspection scenario provided by the present invention includes: A material acquisition module for extracting target object materials from UAV images at the UAV's perspective and altitude, where the target object materials include segmentation regions of at least two targets to be detected. An image enhancement module for, for each target to be detected, performing data enhancement processing on the target to be detected according to the pose characteristics, body type characteristics of the target to be detected, and environmental characteristics in the UAV image to obtain new target object materials. An image synthesis module for synthesizing the new target object materials with the UAV image according to the target annotation method to obtain small target detection data for the target detection model to perform small target detection tasks; wherein, the target annotation method includes at least one of random position annotation and fixed interval annotation.

[0012] A device for processing small target detection data based on an unmanned aerial vehicle (UAV) inspection scenario provided by the present invention further includes: A detection module for, after obtaining the small target detection data, using the small target detection data as training samples and the joint loss function as the training loss to iteratively train the SPD-YOLOv8 network to obtain a target detection model for performing small target detection tasks; wherein, the joint loss function is determined based on location loss, classification loss, and confidence loss.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the method for processing small target detection data based on an unmanned aerial vehicle (UAV) inspection scenario as described in any one of the above when executing the computer program.

[0014] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and the computer program implements the method for processing small target detection data based on an unmanned aerial vehicle (UAV) inspection scenario as described in any one of the above when executed by a processor.

[0015] The present invention also provides a computer program product, including a computer program, and the computer program implements the method for processing small target detection data based on an unmanned aerial vehicle (UAV) inspection scenario as described in any one of the above when executed by a processor.

[0016] The small target detection data processing method and device based on the UAV inspection scenario provided by the present invention extract the target object materials from the UAV images at the UAV's perspective and altitude. For each target to be detected, data enhancement processing is performed on the target to be detected according to the attitude characteristics, body type characteristics of the target to be detected, and environmental characteristics in the UAV images, obtaining new target object materials. Finally, the new target object materials are synthesized with the UAV images according to the target annotation method to obtain small target detection data for the target detection model to perform small target detection tasks, improving the quantity and quality of small target samples from the UAV's perspective. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 FIG. 1 is one of the schematic flowcharts of the small target detection data processing method based on the UAV inspection scenario provided by the present invention.

[0019] Figure 2 FIG. 2 is another schematic flowchart of the small target detection data processing method based on the UAV inspection scenario provided by the present invention.

[0020] Figure 3 FIG. 3 is yet another schematic flowchart of the small target detection data processing method based on the UAV inspection scenario provided by the present invention.

[0021] Figure 4 FIG. 4 is still another schematic flowchart of the small target detection data processing method based on the UAV inspection scenario provided by the present invention.

[0022] Figure 5 FIG. 5 is the schematic diagram of the enhanced region segmentation provided by the present invention.

[0023] Figure 6 FIG. 6 is the schematic structural diagram of the small target detection data processing device based on the UAV inspection scenario provided by the present invention.

[0024] Figure 7 FIG. 7 is the schematic structural diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] To make the objectives, technical solutions and advantages of the present invention more clear, the following will, in conjunction with the accompanying drawings in the present invention, clearly and completely describe the technical solutions in the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts belong to the scope of protection of the present invention.

[0026] The following will, in conjunction with Figures 1-6 describe the method and device for processing small target detection data based on the UAV inspection scenario of the present invention.

[0027] Figure 1 is one of the flow schematic diagrams of the method for processing small target detection data based on the UAV inspection scenario provided by the present invention. As Figure 1 shown, the method for processing small target detection data based on the UAV inspection scenario includes the following steps: Step 110: Extract the target object materials from the UAV images at the UAV's perspective and height. The target object materials include the segmentation regions of at least two targets to be detected.

[0028] In this step, the UAV images include the images found in the public dataset with similar perspectives and containing the targets to be detected. The target object features in such images are sufficient, but the perspectives and contrasts may be different from the actual ones, and some processing needs to be carried out during generation.

[0029] For example, due to the differences in the shooting angles and heights of the UAV, there may be distortions in the images. Through geometric correction, these distortions can be eliminated to ensure the accuracy of the image data.

[0030] In this step, the UAV images also include the targets to be detected collected at a long focal length in the actual environment using the shooting equipment. Such images have a high degree of fit with the actual situation, but the data volume is limited.

[0031] In this step, the targets to be detected include people and animals; the animals include but are not limited to elk, ostriches, cheetahs, etc.

[0032] In this embodiment, the UAV images can be segmented by threshold segmentation, edge detection, region growing, texture analysis or a method based on deep learning to obtain the corresponding target object materials; the size of the target object materials is the same as the length and width of the original image.

[0033] Step 120: For each target to be detected, perform data augmentation processing on the target to be detected according to the pose features, body shape features of the target to be detected and the environmental features in the UAV images to obtain new target object materials.

[0034] In this step, in order to make the target object material conform to the proportion in the original image and improve the richness of the samples, it is necessary to randomly select material objects from the material library before each round of texturing. For different material objects, the preprocessing process is different.

[0035] In this embodiment, data augmentation processing is performed on the target to be detected according to the pose characteristics, body shape characteristics of the target to be detected, and environmental characteristics in the UAV image, and the new target object materials obtained include: (1) Rotate the target to be detected according to the pose characteristics to obtain the rotated target to be detected.

[0036] Specifically, when the target to be detected is a person, since a person mainly presents a standing state as a whole at different pitch angles of the UAV, and the deflection angle relative to the center line of the picture will not be greater than 20°, the target object material corresponding to the person can be flipped within the range of 1° to 20°; for a target to be detected such as an elk, its posture and overall angle can be rotated almost 360° from the UAV perspective, and the target object material corresponding to the elk can be flipped within the range of 1° to 360°; different rotation strategies can be adopted for different material objects.

[0037] (2) Adjust the image saturation of the rotated target to be detected according to the environmental characteristics to obtain the adjusted target to be detected.

[0038] Specifically, after obtaining the rotated target to be detected, in order to make the material object more realistic, it is necessary to adjust the saturation of the current material according to the lighting conditions and environmental state of the original image (such as the UAV image) corresponding to each material, so that it is not too obtrusive in the background image; it should be noted that the darker the lighting, the smaller the saturation needs to be adjusted.

[0039] (3) Scale the width and height of the adjusted target to be detected respectively according to the body shape characteristics to obtain the new target object material.

[0040] Specifically, after obtaining the adjusted target to be detected, the size of the material object is adjusted according to the size of the target object captured by the UAV at different flight heights; it should be noted that if the height data of the UAV is missing, the material object needs to be scaled according to the size of the existing target object in the background image to make it have reasonable width and height in the background image.

[0041] Figure 2 is the second flow chart of the small target detection data processing method based on the UAV inspection scenario provided by the present invention. In Figure 2In the illustrated embodiment, a target object (material) is randomly selected from the material library. When it is confirmed that the target object is a person, the object angle is randomly rotated by -30° to 30° with a 20% probability; otherwise, the object angle is randomly rotated by -180° to 180° with a 50% probability. Then, the saturation of the rotated target object is decreased, and the adjusted target object is scaled according to the expected width and height of the object to obtain a new target object material.

[0042] Step 130: Synthesize the new target object material with the UAV image according to the target annotation method to obtain small target detection data for the target detection model to perform the small target detection task. The target annotation method includes at least one of random position annotation and fixed interval annotation.

[0043] In this step, for the background image containing the target object material, the expected width and height of the material can be calculated separately according to different categories of the material. For example, for the target object of a person, the average height is calculated, while for the target object of a deer, the average maximum width and height are calculated. If the background image does not contain the target object, the material is scaled according to the default width and height.

[0044] In this embodiment, after the preprocessing of the material is completed, the generation position is generated in the enhanced area division. The generation position can adopt a fixed interval or a random position.

[0045] Specifically, synthesizing the new target object material with the UAV image according to the target annotation method to obtain the small target detection data includes: using the new target object material as the first layer and the UAV image as the second layer, and overlaying the first layer with the second layer according to the target annotation method to obtain the small target detection data.

[0046] In this embodiment, overlaying the first layer in the target area of the second layer in a fixed interval manner can make the new target object material traverse and generate on the enhanced areas in different background environments; the first layer can also be overlaid in the target area of the second layer in a random position manner to generate a new background image.

[0047] In this embodiment, an appropriate material annotation method can be selected according to the dataset size and hardware capabilities. After the material annotation is completed, annotation boxes are generated according to the size of the material and the annotation file is exported, thereby obtaining the small target detection data.

[0048] In this embodiment, the small target detection data can be used as a training sample for the supervised training of the small target detection model to obtain a target detection model with better detection performance and robustness; the small target detection data can be used as a test sample to verify the small target detection effect of the trained target detection model.

[0049] Figure 3It is the third flowchart of the small target detection data processing method based on the UAV inspection scenario provided by the present invention. In Figure 3 In the shown embodiment, when it is confirmed that the background image contains the target (object material), the average height of all personnel objects is taken as the expected width and height of the personnel material, and the average maximum width and height of all elk objects are taken as the expected width and height of the material to preprocess the material. Otherwise, the material is directly preprocessed according to the preset fixed width and height; the enhanced area is identified from the UAV image, and the personnel material is merged into the personnel area (person_area) in the enhanced area to generate personnel; the elk material is merged into the elk area in the enhanced area to generate elk, and the corresponding annotation boxes are generated, and the annotation file is exported to obtain the corresponding small target detection data.

[0050] The small target detection data processing method based on the UAV inspection scenario provided by the embodiment of the present invention extracts the target object material at the UAV perspective and height from the UAV image. For each target to be detected, data enhancement processing is performed on the target to be detected according to the pose characteristics, body shape characteristics of the target to be detected and the environmental characteristics in the UAV image to obtain a new target object material. Finally, the new target object material is synthesized with the UAV image according to the target annotation method to obtain the small target detection data for the target detection model to perform the small target detection task, improving the quantity and quality of the small target samples under the UAV perspective.

[0051] Figure 4 It is the fourth flowchart of the small target detection data processing method based on the UAV inspection scenario provided by the present invention. In Figure 4 In the shown embodiment, a small target detection data processing method based on the UAV inspection scenario further includes: annotating the enhanced area on the existing dataset, collecting and extracting the target object material at the corresponding UAV perspective and height, randomly generating the adjusted target detection objects within the range of the dataset enhanced area to obtain the enhanced target detection dataset, improving the generalization ability of small targets.

[0052] In some embodiments, extracting the target object material at the UAV perspective and height from the UAV image includes: performing image segmentation on the enhanced area in the UAV image based on an instance segmentation model to obtain the target object material, and the instance segmentation model is trained based on the YOLOv8-seg network.

[0053] In this embodiment, the enhanced area is a closed area composed of a series of coordinate points in the UAV image. It is inefficient and non-expandable to manually annotate the enhanced area. In order to realize the automatic segmentation of the enhanced area for a large amount of image data, an instance segmentation model can be pre-trained to automatically segment the enhanced area according to the scene requirements.

[0054] In this embodiment, after the YOLOv8-seg model is trained, preprocessing operations of resizing and normalizing the above-mentioned enhanced region are performed, and the preprocessed image is input into the YOLOv8-seg model for instance segmentation inference. Finally, the segmentation mask and bounding box information are extracted from the model output to obtain the segmentation result of the target object.

[0055] Figure 5 is a schematic diagram of enhanced region segmentation provided by the present invention. In Figure 5 the illustrated embodiment, the current object detection task is to identify the people appearing in the park and the elk inside the fence. The possible activity areas of the two types of detection objects can be segmented from the enhanced region. Specifically, the areas where people may appear, such as squares and greenways, are labeled as person_area (the dark areas with outlines in the left figure), and the areas where elk may appear, such as fences, are labeled as elk_area (the dark areas with oval outlines in the right figure). After manually annotating about hundreds of pieces of data, an instance segmentation model can be trained based on YOLOv8-seg, and the automatic segmentation of the enhanced region can be achieved through the model to obtain the corresponding target object materials; among them, the segmented target object is a single individual, the background is transparent, and the length and width of the image are the same as the size of the target object materials.

[0056] The method for processing small target detection data in the UAV inspection scenario provided by the embodiment of the present invention improves the efficiency and quality of obtaining target object materials by performing image segmentation on the enhanced region in the UAV image through an instance segmentation model trained based on the YOLOv8-seg network.

[0057] In some embodiments, after obtaining the small target detection data, the method for processing small target detection data in the UAV inspection scenario further includes: using the small target detection data as training samples and using the joint loss function as the training loss to perform iterative training on the SPD-YOLOv8 network to obtain a target detection model for performing small target detection tasks; among them, the joint loss function is determined based on the location loss, classification loss, and confidence loss.

[0058] In this embodiment, the location loss (Localization Loss) includes Smooth L1 loss, Intersection over Union Loss, and EIoU loss, etc.; the classification loss (Classification Loss) includes Cross-Entropy Loss and Focal Loss.

[0059] In this embodiment, in the UAV small target detection task, the joint loss function is usually a weighted sum of the above multiple loss functions, that is, the joint loss function L is expressed as follows: L = λ 1 ⋅ Localization Loss + λ 2 ⋅ Classification Loss + λ 3 ⋅ Confidence Loss; where λ 1 、λ 2 and λ 3 are the weights of each part of the loss function, which usually need to be adjusted according to the specific task; specifically, these weights can be dynamically adjusted according to the characteristics of small target detection. For example, in order to detect small targets more accurately, the weights of the location loss or confidence loss can be increased.

[0060] In this embodiment, the method for obtaining the sample object material is as described in the corresponding embodiments of the above steps 110 - 130, and this embodiment will not be elaborated.

[0061] In this embodiment, after obtaining the target detection model, the drone image to be detected is input into the target detection model for detection to obtain the small target detection result, so as to perform subsequent image analysis tasks.

[0062] The method for processing small target detection data based on the drone patrol scenario provided by the embodiment of the present invention iteratively trains the SPD - YOLOv8 network with the small target detection data as the training sample and the joint loss function as the training loss to obtain the target detection model, improving the detection accuracy and robustness of the target detection model.

[0063] Next, the device for processing small target detection data based on the drone patrol scenario provided by the present invention will be described. The device for processing small target detection data based on the drone patrol scenario described below can be mutually referred to the method for processing small target detection data based on the drone patrol scenario described above.

[0064] Figure 6 is the structural schematic diagram of the device for processing small target detection data based on the drone patrol scenario provided by the present invention, as Figure 6 shown, the method for processing small target detection data based on the drone patrol scenario includes: a material acquisition module 610, an image enhancement module 620, and an image synthesis module 630.

[0065] The material acquisition module 610 is used to extract the target object material from the drone image at the drone's perspective and height, and the target object material includes at least two segmentation regions of the targets to be detected; An image enhancement module 620, configured to perform data enhancement processing on each target to be detected according to the pose features, body type features of the target to be detected, and environmental features in the UAV image, so as to obtain new target object materials; An image synthesis module 630, configured to synthesize the new target object materials with the UAV image according to the target annotation method to obtain small target detection data for the target detection model to perform the small target detection task; wherein, the target annotation method includes at least one of random position annotation and fixed interval annotation.

[0066] The small target detection data processing device based on the UAV inspection scenario provided by the embodiment of the present invention extracts target object materials from the UAV image from the perspective and height of the UAV. For each target to be detected, data enhancement processing is performed on the target to be detected according to the pose features, body type features of the target to be detected, and environmental features in the UAV image to obtain new target object materials. Finally, the new target object materials are synthesized with the UAV image according to the target annotation method to obtain small target detection data for the target detection model to perform the small target detection task, improving the quantity and quality of small target samples from the UAV perspective.

[0067] In some embodiments, the small target detection data processing device based on the UAV inspection scenario further includes: a detection module 640.

[0068] The detection module 640 is configured to, after obtaining the small target detection data, use the small target detection data as training samples and use the joint loss function as the training loss to iteratively train the SPD-YOLOv8 network to obtain a target detection model to perform the small target detection task; wherein, the joint loss function is determined based on the position loss, classification loss, and confidence loss.

[0069] The small target detection data processing device based on the UAV inspection scenario provided by the embodiment of the present invention iteratively trains the SPD-YOLOv8 network with the small target detection data as training samples and the joint loss function as the training loss to obtain a target detection model, improving the detection accuracy of the target detection model and the robustness of the model.

[0070] Figure 7 is a schematic structural diagram of an electronic device provided by the present invention, as Figure 7As shown in the figure, the electronic device may include: a processor 710, a communications interface 720, a memory 730, and a communication bus 740. Among them, the processor 710, the communications interface 720, and the memory 730 complete communication with each other through the communication bus 740. The processor 710 may call the logical instructions in the memory 730 to execute a method for processing small target detection data in the scenario of drone inspection. The method includes: extracting target object materials at the perspective and height of the drone from the drone image, where the target object materials include segmentation regions of at least two targets to be detected; for each target to be detected, performing data augmentation processing on the target to be detected according to the pose characteristics, body shape characteristics of the target to be detected, and environmental characteristics in the drone image to obtain new target object materials; synthesizing the new target object materials with the drone image according to the target annotation method to obtain small target detection data for the target detection model to perform the small target detection task; where the target annotation method includes at least one of random position annotation and fixed interval annotation.

[0071] In addition, when the logical instructions in the above-mentioned memory 730 are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0072] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the small target detection data processing method provided by the above-mentioned various methods. The method includes: extracting target object materials from the drone image at the perspective and height of the drone, where the target object materials include segmentation regions of at least two targets to be detected; for each target to be detected, performing data augmentation processing on the target to be detected according to the pose characteristics, body shape characteristics of the target to be detected, and environmental characteristics in the drone image to obtain new target object materials; synthesizing the new target object materials with the drone image according to the target annotation method to obtain small target detection data for the target detection model to perform the small target detection task; where the target annotation method includes at least one of random position annotation and fixed interval annotation.

[0073] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the small target detection data processing method provided by the above-mentioned various methods. The method includes: extracting target object materials from the drone image at the perspective and height of the drone, where the target object materials include segmentation regions of at least two targets to be detected; for each target to be detected, performing data augmentation processing on the target to be detected according to the pose characteristics, body shape characteristics of the target to be detected, and environmental characteristics in the drone image to obtain new target object materials; synthesizing the new target object materials with the drone image according to the target annotation method to obtain small target detection data for the target detection model to perform the small target detection task; where the target annotation method includes at least one of random position annotation and fixed interval annotation.

[0074] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0075] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing small target detection data in a drone inspection scenario, characterized in that: include: Extracting target object material at the perspective and height of the drone from the drone image, wherein the target object material includes at least two segmented regions of the target to be detected; For each target to be detected, data enhancement processing is performed on the target to be detected according to the posture characteristics, body shape characteristics and environmental characteristics in the drone image to obtain a new target object material; The new target object material is synthesized with the drone image according to the target annotation method to obtain small target detection data for the target detection model to perform the small target detection task; wherein the target annotation method includes at least one of random position annotation and fixed interval annotation.

2. The method for processing small target detection data in a drone inspection scenario according to claim 1 is characterized in that: The step of performing data enhancement processing on the target to be detected according to the posture features, body shape features and environmental features in the drone image to obtain new target object material includes: Rotating the target to be detected according to the posture feature to obtain a rotated target to be detected; Adjusting the image saturation of the rotated target to be detected according to the environmental characteristics to obtain an adjusted target to be detected; The adjusted width and height of the target to be detected are scaled according to the body shape features to obtain the new target object material.

3. The method for processing small target detection data in a drone inspection scenario according to claim 1 is characterized in that: The method of extracting target object material under the perspective and height of the drone from the drone image includes: The enhanced area in the drone image is segmented based on an instance segmentation model to obtain the target object material, and the instance segmentation model is obtained based on YOLOv8-seg network training.

4. The method for processing small target detection data in a drone inspection scenario according to claim 1 is characterized in that: The synthesizing the new target object material with the drone image according to the target annotation method to obtain small target detection data includes: The new target object material is used as a first layer, the drone image is used as a second layer, and the first layer is superimposed on the second layer according to the target annotation method to obtain the small target detection data.

5. The method for processing small target detection data in a drone inspection scenario according to claim 1 is characterized in that: After obtaining the small target detection data, the method further includes: The small target detection data is used as a training sample, and the SPD-YOLOv8 network is iteratively trained with a joint loss function as the training loss to obtain a target detection model to perform a small target detection task; wherein the joint loss function is determined based on position loss, classification loss, and confidence loss.

6. A small target detection data processing device based on drone inspection scenarios, characterized in that: include: A material acquisition module is used to extract target object materials under the perspective and height of the drone from the drone image, wherein the target object materials include at least two segmented areas of the target to be detected; An image enhancement module is used to perform data enhancement processing on each target to be detected according to the posture characteristics, body shape characteristics and environmental characteristics of the target to be detected in the drone image to obtain a new target object material; An image synthesis module is used to synthesize the new target object material with the drone image according to the target annotation method to obtain small target detection data for the target detection model to perform the small target detection task; wherein the target annotation method includes at least one of random position annotation and fixed interval annotation.

7. The small target detection data processing device based on drone inspection scenario according to claim 6 is characterized in that: The device also includes: The detection module is used to iteratively train the SPD-YOLOv8 network with the small target detection data as training samples and the joint loss function as training loss after obtaining the small target detection data, so as to obtain a target detection model to perform the small target detection task; wherein the joint loss function is determined based on position loss, classification loss and confidence loss.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the method for processing small target detection data in a drone inspection scenario is implemented as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the method for processing small target detection data in a drone inspection scenario as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by the processor, the method for processing small target detection data in a drone inspection scenario as described in any one of claims 1 to 6 is implemented.

Images