An aerial image target detection method, device, equipment and medium
This aerial image target detection method, which combines pixel statistical feature analysis and dynamic preprocessing strategies with textual prompts, solves the problem of environmental factors affecting aerial images and improves the accuracy and precision of detection.
Patent Information
- Application Number
- CN202610472943.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-04-10
- Publication Date
- 2026-07-07
Smart Images
Figure CN122347672A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of target detection technology, and more specifically, to a target detection method, apparatus, device, and medium for aerial images. Background Technology
[0002] With the rapid development of deep learning-based object detection technology, significant progress has been made in the field of aerial object detection. Most current aerial object detection methods are based on general object detection methods, such as Fast R-CNN, Faster R-CNN, YOLO series, and SSD. Compared to general object detection, aerial object detection faces more complex challenges. First, due to the influence of natural environmental factors such as weather and lighting, the appearance of objects in aerial images varies greatly in different scenes. Furthermore, aerial images are usually taken from higher altitudes, resulting in a higher proportion of small objects in the images. These factors exacerbate the scale imbalance challenge prevalent in aerial object detection. Second, the bird's-eye view (BEV) of aerial images causes significant changes in object orientation. This results in horizontal bounding boxes containing more background information, leading to an imbalance between foreground and background categories. These factors contribute to the lower accuracy of object detection in aerial images. Summary of the Invention
[0003] The purpose of this application is to provide a target detection method, apparatus, device and medium for aerial images, so as to solve the above-mentioned problems existing in the prior art and achieve efficient and accurate aerial image target detection.
[0004] Firstly, a target detection method for aerial images is provided, which may include: Obtain the aerial image to be identified and the corresponding text recognition prompt information for the aerial image; Pixel statistical feature analysis is performed on the aerial image to be identified to obtain the pixel scene features of the aerial image to be identified; and based on the pixel scene features, the target recognition scene corresponding to the aerial image to be identified is determined. Match the target preprocessing strategy corresponding to the target recognition scenario from the comparison table of different recognition scenarios and different preprocessing strategies; The target preprocessing strategy is used to preprocess the aerial image to be identified to obtain the preprocessed aerial image to be identified; The preprocessed aerial image to be identified and the text recognition prompt information are input into a pre-trained target detection model to obtain the target detection result corresponding to the aerial image to be identified.
[0005] In an optional implementation, the target recognition scene includes a foggy scene, an illuminated scene, a scale scene, a blurred scene, a noisy scene, a background scene, a tone scene, and an edge scene.
[0006] In an optional implementation, the target detection model includes: The input layer is used to input the preprocessed aerial image to be recognized and the text recognition prompt information; The coarse detection network branch is used to extract features from the preprocessed aerial image to be identified to obtain a feature vector; based on the feature vector, multiple initial candidate regions are generated through preset anchor boxes. The MCNN density map generation branch is used to generate a target density distribution feature map based on the preprocessed aerial image to be identified. The foreground information packaging module is used to fuse multiple generated initial candidate regions and target density distribution feature maps to generate comprehensive foreground information; The CLIP encoder is used to perform feature encoding on the integrated foreground information and the text recognition prompt information respectively, and generate visual feature embedding vectors and text feature embedding vectors. The feature enhancement module is used to fuse the visual feature embedding vector and the text feature embedding vector to obtain a multimodal feature vector; and to perform generative feature enhancement on the multimodal feature vector to obtain an enhanced multimodal feature vector. The YOLO detector is used to generate target detection results based on the enhanced multimodal feature vector.
[0007] In an optional implementation, the MCNN density map generation branch is specifically used for: Density features are extracted from the preprocessed aerial image to be identified to obtain a multi-scale density feature vector. The multi-scale density feature vector is subjected to pixel-by-pixel weighted fusion processing to obtain the global density feature vector; The global density feature vector is mapped to a density map to obtain the target density distribution feature map.
[0008] In an optional implementation, the CLIP encoder is specifically used for: The integrated foreground information is mapped into a visual feature vector through a linear projection layer, and the visual feature vector is input into the CLIP visual encoder to obtain a visual feature embedding vector. The text recognition prompt information is lexicalized, and after adding positional encoding, it is input into the pre-trained CLIP text encoder. After multi-layer multi-head self-attention calculation, the text feature embedding vector is output. The visual feature embedding vector and the text feature embedding vector are mapped to a common semantic space of the same dimension to obtain the visual feature embedding vector and the text feature embedding vector, respectively.
[0009] In an optional implementation, the YOLO detector is specifically used for: The enhanced multimodal feature vectors are reshaped into spatial feature maps; The spatial feature map is input into the YOLOv8 detector head to obtain prediction tensors at different scales; For any prediction tensor, the prediction tensor is decoded to obtain bounding box parameters, target confidence scores, and probabilities of each class at each anchor point. Based on the bounding box parameters, target confidence scores, and class probabilities corresponding to prediction tensors at different scales, target detection results are generated.
[0010] In an optional implementation, the pixel scene features include: dark channel mean, average brightness value, proportion of dark pixels, edge response value, high frequency intensity value, background pixel proportion, RGB channel intensity value, and edge sharpness value.
[0011] Secondly, a target detection device for aerial images is provided, the device comprising: The acquisition unit is used to acquire the aerial image to be identified and the text recognition prompt information corresponding to the aerial image to be identified; The determining unit is used to perform pixel statistical feature analysis on the aerial image to be identified to obtain the pixel scene features of the aerial image to be identified; and based on the pixel scene features, determine the target recognition scene corresponding to the aerial image to be identified. The matching unit is used to match the target preprocessing strategy corresponding to the target recognition scenario from a reference table of different recognition scenarios and different preprocessing strategies. The preprocessing unit is used to preprocess the aerial image to be identified using the target preprocessing strategy to obtain the preprocessed aerial image to be identified. The detection unit is used to input the preprocessed aerial image to be identified and the text recognition prompt information into a pre-trained target detection model to obtain the target detection result corresponding to the aerial image to be identified.
[0012] Thirdly, an electronic device is provided, which includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements any of the steps described in the first aspect above.
[0013] Fourthly, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements the steps of any of the methods described in the first aspect above.
[0014] This application uses pixel statistical feature analysis to perceive the current scene state of the image and dynamically matches a targeted preprocessing strategy accordingly. This effectively solves the problem that traditional fixed preprocessing processes cannot adapt to the changing aviation environment and significantly reduces the interference of severe weather or special lighting on the subsequent detection accuracy. The high-quality image optimized by scene adaptation is input into the model together with text recognition prompts. By utilizing the complementary advantages of visual features and semantic prior knowledge, the false detection rate and false negative rate caused by complex backgrounds and small targets in aviation images are significantly reduced, and the model's localization accuracy and classification confidence for specific targets are improved. Attached Figure Description
[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 An architecture diagram of an aerial image target detection system provided in this application embodiment; Figure 2 A schematic flowchart illustrating a target detection method for aerial images provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a target detection device for aerial images provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise defined, the technical or scientific terms used in this application should have the ordinary meaning understood by those skilled in the art. The words "first," "second," and similar terms used in this application do not indicate any order, quantity, or importance, but are only used to distinguish different components. The words "comprising" or "including," etc., mean that the element or object preceding the word covers the element or object listed after the word and its equivalents, but do not exclude other elements or objects. The words "connected," "coupled," or "connected," etc., are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up," "down," "left," "right," etc., are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0018] The target detection method for aerial images provided in this application embodiment can be applied to... Figure 1 In the system architecture shown, such as Figure 1 As shown, the system may include: a server, an input device, and a drone. The drone is used to acquire aerial images to be identified; the input device is used for a user to input text recognition prompts corresponding to the aerial images to be identified; the server is used to execute the target detection method for aerial images provided in this application embodiment; the server may be a physical server, a server cluster composed of multiple physical servers, or a distributed system, or it may be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.
[0019] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application. Furthermore, the embodiments and features in the embodiments of this application can be combined with each other without conflict.
[0020] Figure 2 This is a flowchart illustrating a target detection method for aerial images provided in an embodiment of this application. Figure 2 As shown, the method may include: Step S210: Obtain the aerial image to be recognized and the corresponding text recognition prompt information.
[0021] The aerial images to be identified are raw RGB images captured using drones or other aviation equipment.
[0022] Step S220: Perform pixel statistical feature analysis on the aerial image to be identified to obtain the pixel scene features of the aerial image to be identified; and determine the target recognition scene corresponding to the aerial image to be identified based on the pixel scene features.
[0023] The pixel scene features can include: dark channel mean, average brightness value, proportion of dark pixels, edge response value, high frequency intensity value, background pixel proportion, RGB channel intensity value, and edge sharpness value; the target recognition scene can include foggy day recognition scene, lighting scene, scale scene, blurred scene, noisy scene, background scene, tone scene, and edge scene; the foggy day recognition scene can be a foggy day scene or a non-foggy day scene; the lighting scene can be a low light scene or a normal light scene; the scale scene can be a high-altitude small target scene or a non-high-altitude small target scene; the blurred scene can be a motion blurred scene or a non-motion blurred scene; the noisy scene can be a high noise scene or a low noise scene; the background scene can be a high background proportion scene or a low background proportion scene; the tone scene can be a color-biased scene or a non-color-biased scene; and the edge scene can be a blurred edge scene or a sharp edge scene.
[0024] In practice The minimum values of the RGB three channels of all pixels within a preset local neighborhood of each pixel in the aerial image to be identified are calculated to obtain the dark channel intensity value within the preset local neighborhood of each pixel. Based on the dark channel intensity value within the preset local neighborhood of each pixel, the average dark channel value of the aerial image to be identified is obtained. If the average dark channel value of the aerial image to be identified is greater than the configured fog concentration threshold, the target recognition scene of the aerial image to be identified is a foggy scene. If the average dark channel value of the aerial image to be identified is not greater than the configured fog concentration threshold, the target recognition scene of the aerial image to be identified is a non-foggy scene. Calculate the grayscale value of each pixel in the aerial image to be identified; based on the grayscale intensity value of each pixel, obtain the average brightness value and the proportion of dark pixels in the aerial image to be identified, where dark pixels are pixels whose grayscale intensity value is less than the configured grayscale threshold; if the average brightness value of the aerial image to be identified is less than the configured brightness threshold and the proportion of dark pixels is greater than the configured proportion threshold, then the target recognition scene of the aerial image to be identified is a low-light scene; if the average brightness value of the aerial image to be identified is not less than the configured brightness threshold or the proportion of dark pixels is not greater than the configured proportion threshold, then the target recognition scene of the aerial image to be identified is a normal-light scene. The edge response value of each pixel in the aerial image to be identified is calculated, which can be obtained by convolution operation of the image using the Sobel operator. Based on the edge intensity value of each pixel, pixels with edge intensity values greater than a configured edge threshold are selected as target candidate pixels. The proportion of target candidate pixels to the total number of pixels in the aerial image to be identified is calculated to obtain the target pixel ratio. If the target pixel ratio of the aerial image to be identified is less than the configured scale threshold, the target recognition scene of the aerial image to be identified is a high-altitude small target scene. If the target pixel ratio of the aerial image to be identified is not less than the configured scale threshold, the target recognition scene of the aerial image to be identified is a non-high-altitude small target scene. A Fast Fourier Transform (FFT) is performed on the aerial image to be identified to obtain its frequency domain spectrum. The amplitude at each frequency point in the frequency domain spectrum is calculated to obtain the frequency domain intensity value at each frequency point. Based on the frequency domain intensity value at each frequency point, bright lines in the frequency domain spectrum with brightness higher than the configured spectral threshold are identified, and the distribution angle and length of the bright lines are calculated. If there are bright lines in the frequency domain spectrum that meet the configured size requirements, the target recognition scene of the aerial image to be identified is a motion-blurred scene. If there are no bright lines in the frequency domain spectrum that meet the configured size requirements, the target recognition scene of the aerial image to be identified is a non-motion-blurred scene. Wavelet transform is performed on the aerial image to be identified, decomposing it into high-frequency and low-frequency components. The amplitude of each pixel in the high-frequency component is calculated to obtain the high-frequency intensity value of each pixel. Based on the high-frequency intensity value of each pixel, the proportion of the high-frequency component in the aerial image to be identified is obtained. If the proportion of the high-frequency component in the aerial image to be identified is greater than the configured noise threshold, the target recognition scene of the aerial image to be identified is a high-noise scene; if the proportion of the high-frequency component in the aerial image to be identified is not greater than the configured noise threshold, the target recognition scene of the aerial image to be identified is a low-noise scene. Lightweight saliency detection is performed on the aerial image to be identified, and the saliency intensity value of each pixel is calculated. Based on the saliency intensity values of each pixel, pixels with saliency intensity values less than a configured saliency threshold are selected as background pixels. The proportion of background pixels to the total pixels of the aerial image to be identified is calculated to obtain the background pixel ratio. If the background pixel ratio of the aerial image to be identified is greater than the configured background threshold, the target recognition scene of the aerial image to be identified is a high background ratio scene; if the background pixel ratio of the aerial image to be identified is not greater than the configured background threshold, the target recognition scene of the aerial image to be identified is a low background ratio scene. Calculate the pixel value of each pixel in the aerial image to be identified using the RGB three channels to obtain the RGB channel intensity value of each pixel; based on the RGB channel intensity value of each pixel, calculate the global mean of the RGB three channels of the aerial image to be identified, and then calculate the deviation value of each channel mean from the average of the three channel means (grayscale mean); if the deviation value of any channel is greater than the configured color deviation threshold, the target recognition scene of the aerial image to be identified is a color-biased scene; if the deviation values of all channels are not greater than the configured color deviation threshold, the target recognition scene of the aerial image to be identified is a non-color-biased scene. The Sobel operator is used to perform convolution on the aerial image to be identified, and the gradient magnitude of each pixel is calculated to obtain the edge sharpness value of each pixel. Based on the edge sharpness values of each pixel, the global mean of the gradient magnitude of the aerial image to be identified is obtained as the edge sharpness index. If the global mean of the gradient magnitude of the aerial image to be identified is less than the configured sharpness threshold, the target recognition scene of the aerial image to be identified is a blurred edge scene; if the global mean of the gradient magnitude of the aerial image to be identified is not less than the configured sharpness threshold, the target recognition scene of the aerial image to be identified is a sharp edge scene.
[0025] Step S230: Match the target preprocessing strategy corresponding to the target recognition scenario from the comparison table of different recognition scenarios and different preprocessing strategies.
[0026] The target preprocessing strategy may include: preprocessing steps, preprocessing execution order, and preprocessing parameters for each preprocessing step. In practical applications, the target recognition scenario includes 8 dimensions, and the preprocessing execution order is used to characterize the execution order of the preprocessing strategies corresponding to the 8 dimensions.
[0027] Step S240: Use the target preprocessing strategy to preprocess the aerial image to be identified, and obtain the preprocessed aerial image to be identified.
[0028] In practice, if the target scene of the aerial image to be identified is a foggy scene, the matched foggy preprocessing strategy is executed. The dark channel of the aerial image to be identified is calculated sequentially to obtain a dark channel image; atmospheric light value is estimated based on the dark channel image; the transmittance value of each pixel in the aerial image to be identified is calculated in combination with the atmospheric light value to obtain an initial transmittance map; a guided filtering algorithm is used to smooth the initial transmittance map and refine the transmittance value to obtain a refined transmittance map; image restoration is performed based on the inverse transformation of the atmospheric scattering model, combined with the aerial image to be identified, atmospheric light value, and refined transmittance map, to obtain a defogging aerial image; if artifacts exist in the defogging aerial image, the guided filtering parameters are adjusted and reprocessed until a defogging image without obvious artifacts is output. If the target scene of the aerial image to be identified is a motion-blurred scene, the matched motion blur preprocessing strategy is executed. The aerial image to be identified is sequentially converted into a frequency domain spectrum by Fast Fourier Transform; the blur kernel parameters in the frequency domain spectrum are estimated, and the Wiener filtering algorithm is used to optimize the frequency domain spectrum to suppress noise amplification and obtain an optimized frequency domain spectrum; the optimized frequency domain spectrum is then converted back into a spatial domain image by Inverse Fast Fourier Transform (IFFT) to obtain the deblurred aerial image. If the target scene in the aerial image to be identified is a low-light scene, the matched low-light preprocessing strategy is executed. Logarithmic transformation is applied to the aerial image to convert the multiplicative illumination model into an additive illumination model, resulting in a logarithmically transformed image. Gaussian surround function convolution is then applied to the logarithmically transformed image to calculate the low-frequency illumination component. The low-frequency illumination component is subtracted from the logarithmically transformed image to extract the reflection component, which reflects the essential properties of the object. Dynamic range compression is applied to the extracted reflection component to enhance details in the dark areas of the image. Finally, exponential operations are performed on the dynamically compressed reflection component to restore it to a normal dynamic range image, resulting in an illuminated aerial image. If the target recognition scene of the aerial image to be recognized is a high-noise scene, the matched high-noise preprocessing strategy is executed; bilateral filtering is performed on the aerial image to be recognized, and smoothing is performed on the aerial image to be recognized in the high-noise scene to obtain the image after bilateral filtering; non-local mean filtering is performed on the image after bilateral filtering; edge enhancement is performed on the image after non-local mean filtering to obtain the low-noise aerial image.
[0029] If the target recognition scene of the aerial image to be identified is a high-altitude small target scene, the matching high-altitude small target preprocessing strategy is executed; lightweight super-resolution reconstruction processing is performed on the aerial image to be identified; edge sharpening processing is performed on the image after super-resolution reconstruction processing; size normalization processing is performed on the image after edge sharpening processing to obtain an aerial image adapted for high-altitude small target detection.
[0030] When the target recognition scene is a non-foggy scene, a normal lighting scene, a non-high-altitude small target scene, a non-motion-blurred scene, a low-noise scene, a non-high background ratio scene, a non-color-biased scene, or a clear scene, a general preprocessing strategy is executed: the aerial image to be recognized is resized to a preset standard size; the pixel value of the aerial image to be recognized is normalized, linearly mapping the image pixel values from the integer range [0,255] to the floating-point range [0.0,1.0]. A slight edge enhancement process is then applied to the normalized image to improve the distinction between the target and the background, resulting in a general preprocessed aerial image that meets the input requirements of subsequent core modules. Simultaneously, the aerial images to be identified, processed using preprocessing strategies corresponding to other dimensions, will undergo size adjustment and normalization.
[0031] In one embodiment of this application, the target recognition scene includes 8 dimensions, each dimension corresponds to a preprocessing strategy, and the image processed by the preprocessing strategy corresponding to the previous dimension is used as the input of the preprocessing strategy corresponding to the next dimension according to the preprocessing execution order.
[0032] Step S250: Input the preprocessed aerial image to be identified and the text recognition prompt information into the pre-trained target detection model to obtain the target detection result corresponding to the aerial image to be identified.
[0033] The target detection model may include: The input layer is used to input the preprocessed aerial image to be recognized and the text recognition prompts; The coarse detection network branch is used to extract features from the preprocessed aerial image to be identified, obtaining feature vectors. Based on the feature vectors, multiple initial candidate regions are generated by pre-defined anchor boxes. Specifically, a lightweight MobileNet convolutional neural network is used to perform layer-by-layer convolution, batch normalization, and activation function operations on the preprocessed aerial image to be identified to extract multi-scale feature maps. Anchor boxes of different scales and aspect ratios are pre-defined at each position of the multi-scale feature maps. The offset of each anchor box relative to the ground truth bounding box is regressed through a 1×1 convolutional layer, and another 1×1 convolutional layer outputs the confidence score of each anchor box containing the target. According to the confidence scores, the top K anchor boxes with the highest scores are retained. After removing redundant boxes through non-maximum suppression, the position coordinates, width, height, and corresponding confidence scores of multiple initial candidate regions are obtained. The MCNN density map generation branch is used to generate a target density distribution feature map based on the preprocessed aerial image to be identified. Specifically, it performs density feature extraction processing based on multi-scale convolution kernels on the preprocessed aerial image to be identified to obtain a multi-scale density feature vector; it performs pixel-wise weighted fusion processing on the multi-scale density feature vector to obtain a global density feature vector; and it performs density map mapping processing on the global density feature vector to obtain the target density distribution feature map. The foreground information packaging module is used to fuse multiple generated initial candidate regions and target density distribution feature maps to generate comprehensive foreground information. Specifically, for any initial candidate region, the position coordinates of the initial candidate region are mapped to the same spatial scale as the target density distribution feature map to obtain the corresponding region of the initial candidate region on the target density distribution feature map; the sum of the density values of the corresponding regions of each initial candidate region on the target density distribution feature map is calculated to obtain the density score of the initial candidate region; the density score of each initial candidate region is multiplied by its corresponding confidence score to obtain the fused comprehensive confidence score; and comprehensive foreground information is generated based on the position coordinates of the initial candidate regions, the original confidence scores, the density scores, and the fused comprehensive confidence score. The CLIP encoder is used to encode features of the integrated foreground information and text recognition prompts separately, generating visual feature embedding vectors and text feature embedding vectors. Specifically, the integrated foreground information is mapped to a visual feature vector of a preset dimension through a linear projection layer, and this visual feature vector is input into the CLIP visual encoder. After multi-layer Transformer encoding, the visual feature embedding vector is output. The text recognition prompts are lexicalized, positional encoding is added, and then input into the pre-trained CLIP text encoder (based on Transformer architecture). After multi-layer multi-head self-attention calculation, the text feature embedding vector is output. The visual feature embedding vector and the text feature embedding vector are respectively mapped to a common semantic space of the same dimension through independent linear projection layers to obtain the visual feature embedding vector and the text feature embedding vector. The feature enhancement module fuses visual feature embedding vectors and text feature embedding vectors to obtain a multimodal feature vector. It then performs feature enhancement on the multimodal feature vector to obtain an enhanced multimodal feature vector. Specifically, it concatenates the visual feature embedding vector and the text feature embedding vector along the channel dimension to obtain the multimodal feature vector. The fused multimodal feature vector is then input into an enhancement network composed of multiple fully connected layers to obtain the enhanced multimodal feature vector. Each fully connected layer in the enhancement network undergoes batch normalization and ReLU activation operations. A gating mechanism is introduced into the enhancement network: the weight coefficients of each feature channel are calculated using the Sigmoid activation function, and these weight coefficients are multiplied element-wise with the current layer's output to achieve dynamic enhancement and suppression of feature channels. The final fully connected layer maps the feature vector to an output space with the same dimension as the visual feature embedding, resulting in the enhanced multimodal features. A residual connection is introduced: the enhanced multimodal features are added to the original visual feature embedding, preserving the original visual information. The YOLO detector is used to generate target detection results based on enhanced multimodal feature vectors. Specifically, the enhanced multimodal feature vectors are reshaped into spatial feature maps through fully connected layers. The spatial feature maps are then input into the YOLOv8 detector head, which contains three branches at different scales. Each branch consists of multiple convolutional layers, outputting prediction tensors at the corresponding scales. Each prediction tensor is decoded to obtain bounding box parameters (center coordinates x, y, width w, height h), target confidence scores, and probabilities of each class at each anchor point. Bounding boxes at all scales are merged, sorted in descending order of confidence scores, and non-maximum suppression (IoU threshold set to 0.5) is applied to remove redundant boxes, resulting in the target detection results. The target detection results can include the class label, confidence score, and location coordinates of each target.
[0034] Corresponding to the above method, embodiments of this application also provide a target detection device for aerial images, such as... Figure 3 As shown, the device includes: The acquisition unit 310 is used to acquire the aerial image to be recognized and the text recognition prompt information corresponding to the aerial image to be recognized; The determining unit 320 is used to perform pixel statistical feature analysis on the aerial image to be identified, obtain the pixel scene features of the aerial image to be identified, and determine the target recognition scene corresponding to the aerial image to be identified based on the pixel scene features. The matching unit 330 is used to match the target preprocessing strategy corresponding to the target recognition scenario from a reference table of different recognition scenarios and different preprocessing strategies. Preprocessing unit 340 is used to preprocess the aerial image to be identified using a target preprocessing strategy to obtain a preprocessed aerial image to be identified. The detection unit 350 is used to input the preprocessed aerial image to be identified and the text recognition prompt information into the pre-trained target detection model to obtain the target detection result corresponding to the aerial image to be identified.
[0035] The functions of each functional unit of the aerial image target detection device provided in the above embodiments of this application can be implemented through the above method steps. Therefore, the specific working process and beneficial effects of each unit in the aerial image target detection device provided in the embodiments of this application will not be repeated here.
[0036] This application also provides an electronic device, such as... Figure 4 As shown, it includes a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440.
[0037] Memory 430 is used to store computer programs; When the processor 410 executes the program stored in the memory 430, it performs the following steps: Obtain the aerial image to be recognized and the corresponding text recognition prompts for the aerial image; Pixel statistical feature analysis is performed on the aerial image to be identified to obtain the pixel scene features of the aerial image to be identified; and based on the pixel scene features, the target recognition scene corresponding to the aerial image to be identified is determined. Match the target preprocessing strategy corresponding to the target recognition scenario from the comparison table of different recognition scenarios and different preprocessing strategies; The aerial image to be identified is preprocessed using a target preprocessing strategy to obtain the preprocessed aerial image to be identified. The preprocessed aerial image to be identified and the text recognition prompts are input into a pre-trained target detection model to obtain the target detection results corresponding to the aerial image to be identified.
[0038] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.
[0039] The communication interface is used for communication between the aforementioned electronic devices and other devices.
[0040] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.
[0041] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0042] The implementation methods and beneficial effects of the various components of the electronic device in the above embodiments for solving the problem can be found in [reference needed]. Figure 2 The steps in the illustrated embodiments are used to implement the electronic device. Therefore, the specific working process and beneficial effects of the electronic device provided in this application will not be repeated here.
[0043] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform any of the target detection methods for aerial images described in the above embodiments.
[0044] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the target detection method for aerial images described in any of the above embodiments.
[0045] Those skilled in the art will understand that the embodiments in this application can be provided as methods, systems, or computer program products. Therefore, the embodiments in this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments in this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0046] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0047] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0048] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0049] Although preferred embodiments have been described in this application, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of this application.
[0050] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of this application and its equivalents, then these modifications and variations are also intended to be included in the embodiments of this application.
Claims
1. A target detection method for aerial images, characterized in that, The method includes: Obtain the aerial image to be identified and the corresponding text recognition prompt information for the aerial image; Pixel statistical feature analysis is performed on the aerial image to be identified to obtain the pixel scene features of the aerial image to be identified; and based on the pixel scene features, the target recognition scene corresponding to the aerial image to be identified is determined. Match the target preprocessing strategy corresponding to the target recognition scenario from the comparison table of different recognition scenarios and different preprocessing strategies; The target preprocessing strategy is used to preprocess the aerial image to be identified to obtain the preprocessed aerial image to be identified; The preprocessed aerial image to be identified and the text recognition prompt information are input into a pre-trained target detection model to obtain the target detection result corresponding to the aerial image to be identified.
2. The method as described in claim 1, characterized in that, The target recognition scenarios include foggy weather scenarios, illuminated scenarios, scale scenarios, blurred scenarios, noisy scenarios, background scenarios, tone scenarios, and edge scenarios.
3. The method as described in claim 1, characterized in that, The target detection model includes: The input layer is used to input the preprocessed aerial image to be recognized and the text recognition prompt information; The coarse detection network branch is used to extract features from the preprocessed aerial image to be identified to obtain a feature vector; based on the feature vector, multiple initial candidate regions are generated through preset anchor boxes. The MCNN density map generation branch is used to generate a target density distribution feature map based on the preprocessed aerial image to be identified. The foreground information packaging module is used to fuse multiple generated initial candidate regions and target density distribution feature maps to generate comprehensive foreground information; The CLIP encoder is used to perform feature encoding on the integrated foreground information and the text recognition prompt information respectively, and generate visual feature embedding vectors and text feature embedding vectors. The feature enhancement module is used to fuse the visual feature embedding vector and the text feature embedding vector to obtain a multimodal feature vector; and to perform generative feature enhancement on the multimodal feature vector to obtain an enhanced multimodal feature vector. The YOLO detector is used to generate target detection results based on the enhanced multimodal feature vector.
4. The method as described in claim 3, characterized in that, The MCNN density map generation branch is specifically used for: Density features are extracted from the preprocessed aerial image to be identified to obtain a multi-scale density feature vector. The multi-scale density feature vector is subjected to pixel-by-pixel weighted fusion processing to obtain the global density feature vector; The global density feature vector is mapped to a density map to obtain the target density distribution feature map.
5. The method as described in claim 3, characterized in that, The CLIP encoder is specifically used for: The integrated foreground information is mapped into a visual feature vector through a linear projection layer, and the visual feature vector is input into the CLIP visual encoder to obtain a visual feature embedding vector. The text recognition prompt information is lexicalized, and after adding positional encoding, it is input into the pre-trained CLIP text encoder. After multi-layer multi-head self-attention calculation, the text feature embedding vector is output. The visual feature embedding vector and the text feature embedding vector are mapped to a common semantic space of the same dimension to obtain the visual feature embedding vector and the text feature embedding vector, respectively.
6. The method as described in claim 3, characterized in that, The YOLO detector is specifically used for: The enhanced multimodal feature vectors are reshaped into spatial feature maps; The spatial feature map is input into the YOLOv8 detector head to obtain prediction tensors at different scales; For any prediction tensor, the prediction tensor is decoded to obtain bounding box parameters, target confidence scores, and probabilities of each class at each anchor point. Based on the bounding box parameters, target confidence scores, and class probabilities corresponding to prediction tensors at different scales, target detection results are generated.
7. The method as described in claim 1, characterized in that, The pixel scene features include: dark channel mean, average brightness value, proportion of dark pixels, edge response value, high frequency intensity value, background pixel proportion, RGB channel intensity value, and edge sharpness value.
8. A target detection device for aerial images, characterized in that, The device includes: The acquisition unit is used to acquire the aerial image to be identified and the text recognition prompt information corresponding to the aerial image to be identified; The determining unit is used to perform pixel statistical feature analysis on the aerial image to be identified to obtain the pixel scene features of the aerial image to be identified; and based on the pixel scene features, determine the target recognition scene corresponding to the aerial image to be identified. The matching unit is used to match the target preprocessing strategy corresponding to the target recognition scenario from a reference table of different recognition scenarios and different preprocessing strategies. The preprocessing unit is used to preprocess the aerial image to be identified using the target preprocessing strategy to obtain the preprocessed aerial image to be identified. The detection unit is used to input the preprocessed aerial image to be identified and the text recognition prompt information into a pre-trained target detection model to obtain the target detection result corresponding to the aerial image to be identified.
9. An electronic device, characterized in that, The electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method of any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.