A method for detecting occluded targets in polar environments

By combining multimodal data acquisition of visible light and infrared images and improved YOLOV8 network detection, the problem of accuracy and computing resource limitation of penguin occlusion target recognition in polar environments is solved, and efficient and accurate target detection in extreme environments is achieved .

CN119229296BActive Publication Date: 2025-02-14POLAR RES INST OF CHINA
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411710124.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-11-27
Publication Date
2025-02-14
Estimated Expiration
2044-11-27

AI Technical Summary

Technical Problem

Existing object detection algorithms are difficult to accurately identify and locate penguin individuals in polar environments, mainly due to extreme lighting conditions, complex background environments and limitations of computing resources.

Method used

A multimodal data acquisition method combining visible light images and infrared images is adopted to generate images to be detected through target subject area recognition and image fusion, and an improved YOLOV8 network is input for occlusion target detection.

Benefits of technology

Improve the accuracy of penguin occlusion target recognition in polar environments, overcome the problem of limited computing resources, and maintain image clarity and real-time in severe weather conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119229296B_ABST
    Figure CN119229296B_ABST
Patent Text Reader

Abstract

The present invention provides a method for detecting occluded targets in polar environments, and relates to the field of image processing technology. The method for detecting targets in the present invention specifically includes the following steps: collecting visible light images and infrared images of polar regions; respectively identifying the target main area of ​​the visible light images and the infrared images; based on the target main area identification result, fusing the visible light image and the infrared image to obtain an image to be detected; and inputting the image to be detected into an improved YOLOV8 network for occluded target detection. Through the above steps, the present invention can improve the accuracy of penguin occluded target recognition in extreme polar environments, and overcome the problem of limited computing resources in polar environments.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a method for detecting occluded targets in a polar environment, which is used for group monitoring and protection of Antarctic penguins. Background Art

[0002] The Antarctic ecosystem is one of the most fragile and unique environments on Earth. Penguins are important indicator species in the region, and changes in their populations directly reflect the health of the Antarctic ecosystem and the impact of climate change. With the rise in global temperatures and the increase in human activities, the ecological balance of Antarctica is facing unprecedented challenges. In this context, monitoring the population size and distribution of penguins has become a key means of assessing and protecting the Antarctic ecological environment. Accurate data can not only help scientists understand the health of penguin populations, but also serve as a scientific basis for formulating protection policies and management measures, thereby ensuring the continued stability of the ecosystem in this region.

[0003] In recent years, the rapid development of computer vision technology, especially deep learning in the field of target detection, has provided effective tools for automated monitoring and data collection. By using advanced algorithms such as convolutional neural networks (CNN) and regional convolutional neural networks (R-CNN), researchers are able to automatically identify and classify objects in images. These technologies have been widely used in monitoring systems ranging from medical imaging diagnosis to autonomous vehicles. For example, in wildlife research, by combining high-resolution cameras installed on drones with target detection algorithms, scientists can count animals and conduct behavioral studies without disturbing the natural behavior of animals, greatly improving the efficiency and accuracy of data collection.

[0004] Although the development of computer vision technology has brought revolutionary progress to penguin monitoring, existing target detection algorithms still face many challenges in practical applications. First, the complex background environment in the Antarctic region, such as the perennial snow cover, makes the contrast between penguins and the environment low and difficult to distinguish. In addition, the living habits of penguin groups lead to mutual occlusion between individuals, which is manifested as the loss of some targets in the image, making it difficult for the algorithm to accurately identify and locate all individuals. Extreme changes in lighting conditions, such as strong reflections and shadows, also seriously affect the performance of the detection algorithm. Finally, the resources of the Antarctic expedition site are limited, such as the lack of high-performance computing resources, requiring the detection algorithm to be not only accurate but also efficient. These challenges make it difficult for existing general target detection algorithms to be directly applied to penguin detection in polar environments. Summary of the invention

[0005] The present invention provides a method for detecting occluded targets in a polar environment, the method specifically comprising the following steps:

[0006] S1: Collect visible light and infrared images of polar regions;

[0007] S2: Recognize the target subject area on the visible light image and the infrared image respectively;

[0008] S3: Based on the target body area recognition result, the visible light image and the infrared image are fused to obtain an image to be detected;

[0009] S4: Input the image to be detected into the improved YOLOV8 network for occluded target detection.

[0010] The present invention provides a system for detecting obstructed targets in a polar environment, the system comprising:

[0011] Image acquisition equipment: The image acquisition equipment acquires visible light images and infrared images of polar regions;

[0012] Target subject detection module: The target subject detection module performs target subject area recognition on the visible light image and the infrared image respectively;

[0013] Occluded target detection: The occluded target detection module is used to input the image to be detected into the improved YOLOV8 network for occluded target detection module.

[0014] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for detecting obstructed targets in polar environments when executing the computer program.

[0015] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for detecting an obscured target in a polar environment is implemented.

[0016] Compared with the prior art, the present invention proposes a method for detecting occluded targets in polar environments. In view of the special conditions of polar environments, the present invention combines the features of visible light images and infrared images for information complementarity. Based on the limited computing resources in the polar regions, the main target area is detected before occlusion recognition, which reduces the computational complexity of occluded target recognition. At the same time, the local features of penguins are detected as attention to guide penguin occlusion detection. In addition, the present invention uses Resnet-50 multi-scale features to extract detailed features in images for small target recognition. At the same time, IoU is introduced in the loss function to focus the model on occluded target detection. Through the above method, the present invention can improve the accuracy of penguin occluded target recognition in extreme polar environments and overcome the problem of limited computing resources in polar environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 It is a flow chart of the present invention for detecting occluded targets in polar environments. DETAILED DESCRIPTION

[0019] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0020] The following describes the implementation methods of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, in the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.

[0021] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present application, it should be understood by those skilled in the art that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number and aspect described herein can be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this device and / or practice this method.

[0022] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the examples can be practiced without these specific details.

[0023] The embodiment of this specification proposes a method for detecting occluded targets in a polar environment, and the method specifically includes the following steps:

[0024] S1: Collect visible light and infrared images of polar regions;

[0025] S2: Recognize the target subject area on the visible light image and the infrared image respectively;

[0026] S3: Based on the target body area recognition result, the visible light image and the infrared image are fused to obtain an image to be detected;

[0027] S4: Input the image to be detected into the improved YOLOV8 network for occluded target detection.

[0028] Specifically, in one embodiment, the visible light image and the infrared image acquisition device are drone devices, and the drone device needs to be able to simultaneously acquire infrared spectrum images and visible light images. At the same time, for extreme polar environments, the drone device can operate safely in low temperature environments. Preferably, the drone device may be equipped with a self-heating battery (such as a TB60 battery), which can ensure that the drone device is continuously powered in a low temperature environment, such as the temperature in polar conditions can reach minus thirty degrees Celsius. Preferably, the drone device is a DJI Matrice300 RTK, and the wind resistance design of the device enables it to fly stably under level six wind conditions, making it suitable for operations in windy environments.

[0029] At the same time, the camera of the drone equipment should be equipped with a large-size (45 million pixels or more) full-frame sensor, which can capture very detailed visible light images and is suitable for capturing high-resolution images of vast snowfields and penguin colonies. In addition to the visible light camera, the camera should also include an infrared thermal imaging system that can capture thermal images of penguins at night or in low light conditions, thereby performing 24-hour monitoring without light restrictions. Preferably, the camera of the drone equipment can use the Zenmuse H20T multi-sensor device.

[0030] During the collection process, the drone was set to operate along a predetermined flight route in automatic cruise mode. Through the DJI Pilot application, the flight path can be preset to ensure coverage of all key penguin habitats. The flight altitude was maintained at 100 meters, which can obtain a sufficiently wide perspective while ensuring image clarity.

[0031] During the flight, the drone transmits the collected images back to the ground station in real time through the OcuSync 3.0 image transmission system. The transmission system should support long-distance, high-bandwidth data transmission to ensure that the image clarity and real-time performance can be maintained even in adverse weather conditions. The collected data will be stored in the drone's internal memory card and downloaded and backed up after each flight, and the collected visible light and infrared images will be subsequently analyzed and processed.

[0032] In the process of polar penguin colony occlusion detection, due to the particularity of the polar environment, severe weather such as wind and snow often occurs, and the lighting conditions are extremely unstable. These factors greatly reduce the recognition rate of target detection relying solely on visible light. In order to overcome these challenges, the present invention not only uses visible light image acquisition, but also specially introduces infrared imaging technology. The introduction of infrared imaging is based on a key natural property: there is a significant difference in temperature between the penguin colony and the ice and snow environment in which it lives. Taking advantage of this temperature difference, it can work effectively even under conditions of insufficient light or interference from snow and wind. This multimodal data acquisition method, which combines both visible light and infrared image information, greatly enriches the understanding of the scene and the acquisition of information, thereby effectively improving the accuracy of target detection.

[0033] The target subject area recognition of the visible light image specifically includes:

[0034] S2-1-1: grayscale the visible light image and perform Gaussian blur smoothing on the image;

[0035] S2-1-2: Set a sliding window, which slides from left to right and from top to bottom on the visible light image to detect the penguin area in the image, W Slide ×H Slide is the size of the sliding window, W Slide is the width of the window, H Slide is the height of the window;

[0036] S2-1-3: For each window position, a scoring function is calculated, which determines whether the content in the window contains a penguin. The scoring function is defined as:

[0037] ;

[0038] in, is the grayscale value of the visible light image at position (x+i, y+j), x and y are the horizontal and vertical coordinates of the lower left corner of the sliding window, i and j are iteration parameters, is an indicator function, which takes the value 1 when the condition is met, otherwise it takes the value 0. is a preset grayscale threshold;

[0039] S2-1-4: Apply a sliding window to the entire visible light image. For each window position where the Score is higher than the preset score, mark the window area as containing the penguin and calculate the confidence score for marking the window. :

[0040] ;

[0041] in, , and is the weight parameter, is the standard deviation of the grayscale value in the window, The maximum standard deviation in the image, is the edge density function;

[0042] S2-1-5: Mark the area consisting of the window containing the penguin as the target main area, and obtain the visible light target main area image .

[0043] The target main area recognition of the infrared image specifically includes:

[0044] S2-2-1: Use Sobel operator to calculate the infrared image The Edge ;

[0045] The Sobel operator is defined as: ;

[0046] S2-2-2: Calculate the infrared image Frequency domain image of ;

[0047] ;Fre and denote Fourier transform and inverse transform respectively;

[0048] S2-2-3: On the edge Perform edge refinement to obtain a fine edge image ;

[0049] ;

[0050] ;

[0051] in, represents the neighborhood of the structural element B at position (x, y), Represents the intermediate processing feature map;

[0052] S2-2-4: Fusion of frequency domain images and fine edge images , and perform super-pixel segmentation on the fused image;

[0053] S2-2-5: Cluster the superpixels and mark the target area, and calculate the confidence of the superpixels marked as the target area ;

[0054] ;

[0055] , and is the weight parameter, is the average value of the edge strength within the superpixel 𝑘, is the average value of the frequency domain image features within the superpixel 𝑘; in the clustering process, the background part cluster center and the target subject area cluster center As optimization target parameter:

[0056] ;

[0057] is the kth superpixel in the nth superpixel;

[0058] In the severe polar environment, penguins usually resist low temperatures by gathering together. This behavioral characteristic is manifested as obvious gathering areas in the images of penguin groups taken by drones. This natural phenomenon provides convenience for monitoring and studying penguins. In addition, due to the harsh polar environment, deploying large-scale computing equipment for data processing is not only costly but also difficult to operate. Therefore, the present invention first detects the target main area based on traditional image processing technology. By applying image grayscale difference and edge feature analysis to visible light images and infrared images, the present invention can quickly identify the main area in the image, that is, the area where penguins gather. This method is fast and is particularly suitable for use in situations where computing resources are limited. After determining the main distribution areas of penguin groups, the present invention further calculates the probability of containing penguin groups in these areas. This step is key in identification and counting because it is directly related to the accuracy and efficiency of subsequent precise identification of penguins.

[0059] Based on the target main area recognition result, fusing the visible light image and the infrared image to obtain the image to be detected specifically includes:

[0060] S3-1: Confidence score based on labeled window and the confidence of the superpixel Generate a composite confidence map ;

[0061] ;

[0062] in, is the weight parameter, and They represent the window confidence score and superpixel confidence score of the image position (x, y). If the position (x, y) is not marked as the target body area by the visible light image or infrared image, the window confidence score or superpixel confidence score is 0.

[0063] S3-2: synthesize a multi-channel image as an image to be detected I(x, y);

[0064] The synthesized multi-channel image is supplemented by complementary information to obtain the image to be detected, specifically:

[0065] S4-1: Input the visible light image and infrared image symmetry network into the twin symmetry feature extraction network;

[0066] S4-2: Construct a cross-information module to extract the difference information between visible light images and infrared images;

[0067] S4-3: Fusing the difference information extracted in step S4-2 into the twin symmetry feature extraction feature map;

[0068] S4-4: Input the feature map fused with the difference information into the decoding network to obtain the enhanced visible light image and the enhanced infrared image, and synthesize the multi-channel image as the image to be detected;

[0069] Each branch of the twin symmetric feature extraction network consists of five convolution modules. The cross information module is set in the 2nd to 4th layers of the twin symmetric feature extraction network, and extracts difference information and fuses it to the current layer output. The cross information module is defined as:

[0070] ;

[0071] ;

[0072] in, , They represent the feature maps extracted from the i-th layer of the twin symmetric feature extraction network for visible light images and infrared images, respectively. and They represent the i-th layer output feature maps after the visible light image and the infrared image are fused with the difference information, and Sigmoid are average pooling and activation functions respectively, represents channel multiplication, Indicates pixel-by-pixel addition;

[0073] The decoding network uses 5 convolution modules to integrate global information and local information. Except for the last convolution module which uses a 1*1 convolution kernel, the other first four convolution modules use a 3*3 convolution kernel, and finally output an enhanced visible light image. and enhanced infrared images Synthesize the image to be detected I(x,y);

[0074] .

[0075] In the process of image fusion, the present invention utilizes the difference between the visible light image and the infrared image, utilizes the cross information module to extract the difference information, and simultaneously supplements it into the visible light image and the infrared image to enrich the image features for subsequent occluded target detection.

[0076] Before inputting the image to be detected into the improved YOLOV8 network for occluded target detection, it is also necessary to obtain a local attention feature map of the image to be detected. The local attention feature map acquisition includes:

[0077] S5-1: Visible light image Convert to HSV color space;

[0078] S5-2: Setting the local feature color HSV threshold: , , , , , ;

[0079] S5-3: For the area that has been determined to contain the target body, apply the local feature color HSV threshold to obtain a local attention feature map;

[0080] Mask=(H low ≤H≤H high )∧(S low ≤S≤S high )∧(V low ≤V≤V high );

[0081] .

[0082] In the polar cold environment, penguins often resist the cold by gathering closely together. Although this behavior ensures their survival, it introduces certain difficulties when observing with drones or other monitoring equipment, especially when conducting penguin counting and detection research. Due to the close arrangement of penguins' bodies, individuals often block each other, making accurate identification and counting complicated. However, the head and neck of penguins are usually less blocked, and adult penguins have unique light yellow patterns in these areas, which provides key clues for accurate identification.

[0083] The present invention uses these natural features to guide the identification of occluded penguins by focusing on these areas with light yellow patterns. First, the system uses color recognition technology to accurately extract the light yellow areas in the image, which are highly likely to correspond to the head and neck of the penguin. Then, the morphology and position of these specific areas are further analyzed to construct a local attention feature map to guide subsequent enterprise individual identification.

[0084] The improved YOLOV8 network includes a feature extraction network, a neck network and a detection head. The feature extraction network is based on a comprehensive confidence map. The input of the feature extraction network is a comprehensive confidence map Adjusted image to be detected :

[0085] ;

[0086] The Resnet-50 consists of five residual modules Res1-Res5, where the input of the residual module Res2 is the output of the residual module Res1 after adjustment of the local attention feature map:

[0087] ;

[0088] in, Represents the scaled local attention feature map f1(x,y) and They are the output feature map of Res1 and the input feature map of Res2 respectively;

[0089] The output features of the residual modules Res3-Res5 are input to the neck network, which is defined as:

[0090] ;

[0091] ;

[0092] ;

[0093] in, , and They are the output features of the residual modules Res3-Res5, , , and are the feature maps of the middle layer of the neck network respectively; the inputs of the detection head are , and , the detection head adopts the Cascade RCNN network structure.

[0094] The improved YOLOV8 network introduces Occlusion-awareLoss based on classification loss and regression loss, and its calculation method is as follows:

[0095] ;

[0096] in, It represents the predicted probability of the target, and IoU represents the intersection over union of the predicted box and the true box.

[0097] The present invention introduces ResNet50 as the backbone network in the traditional YOLO v8 algorithm framework, and adds Occlusion-aware Loss for occluded targets. By replacing the original CSPDarknet53 network with ResNet50, the present invention utilizes the powerful feature extraction capability of its deep residual network to greatly enhance the model's learning ability for subtle and complex visual features. Because the visible parts of these targets are often very limited, the model needs to be able to extract sufficient features from these limited information for accurate recognition. Furthermore, by enhancing the feature extraction capability for small targets in the multi-scale feature fusion module of ResNet50 and adopting the Cascade RCNN structure in the detection head, the present invention can more effectively handle occluded targets of various sizes. The hierarchical structure of Cascade RCNN provides a mechanism for gradually refining the detection frame, so that the model can gradually adjust and optimize the prediction results after preliminary recognition, thereby improving the accuracy and robustness of detection.

[0098] In addition, the introduced Occlusion-aware Loss adds a greater training focus for objects with a high degree of occlusion by adding an IoU-related weight adjustment term to the loss function. Specifically, the loss weight is increased in the form of (1-IoU)², forcing the network to pay more attention to those objects that are difficult to detect. This approach effectively guides the model to learn more deeply about occluded objects during training, so that these objects can be more accurately identified and processed in practical applications.

[0099] The present invention provides a system for detecting obstructed targets in a polar environment, the system comprising:

[0100] Image acquisition equipment: The image acquisition equipment acquires visible light images and infrared images of polar regions;

[0101] Target subject detection module: The target subject detection module performs target subject area recognition on the visible light image and the infrared image respectively;

[0102] Occluded target detection: The occluded target detection module is used to input the image to be detected into the improved YOLOV8 network for occluded target detection module.

[0103] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for detecting obstructed targets in polar environments when executing the computer program.

[0104] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for detecting an obscured target in a polar environment is implemented.

[0105] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0106] In this specification, the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the embodiments described later, the description is relatively simple, and the relevant parts can be referred to the partial description of the previous embodiments.

[0107] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for detecting occluded targets in polar environments, characterized in that: The target detection method specifically comprises the following steps: S1: Collect visible light and infrared images of polar regions; S2: Recognize the target main area of ​​the visible light image and the infrared image respectively; the target main area recognition of the visible light image specifically includes: S2-1-1: grayscale the visible light image and perform Gaussian blur smoothing on the image; S2-1-2: Set a sliding window, which slides from left to right and from top to bottom on the visible light image to detect the penguin area in the image, W Slide ×H Slide is the size of the sliding window, W Slide is the width of the window, H Slide is the height of the window; S2-1-3: For each window position, a scoring function Score is calculated. The scoring function determines whether the content in the window contains a penguin. The scoring function is defined as: Among them, I img (x+i, y+j) is the grayscale value of the visible light image at position (x+i, y+j), x and y are the horizontal and vertical coordinates of the lower left corner of the sliding window, i and j are iteration parameters, 1(·) is an indicative function, which takes the value 1 when the condition is met and 0 otherwise. g is a preset grayscale threshold; S2-1-4: Apply a sliding window to the entire visible light image. For each window position where the Score is higher than the preset score, mark the window area as containing the penguin, and calculate the confidence score Cnf for marking the window. img : Among them, α, β and γ are weight parameters, σ(x, y) is the standard deviation of the gray value in the window, and σ max The maximum standard deviation in the image, Edg is the edge density function; S2-1-5: Mark the area consisting of the window containing the penguin as the target main area, and obtain the visible light target main area image The target main area recognition of the infrared image specifically includes: S2-2-1: Use Sobel operator to calculate infrared image I ifr Edge of I G (x,y); S2-2-2: Calculate the infrared image I ifr The frequency domain image I F (x,y); I F (x,y)=Fre -1 (log(|Fre(I ifr )|)-Fre(I ifr )); Fre and Fre -1 denote Fourier transform and inverse transform respectively; S2-2-3: For the edge I G (x, y) is used to refine the edge to obtain a fine edge image I GR (x,y); I M (x,y)=max(min(I G (x,y),(x,y)∈N x,y (B))); I GR (x,y)=min(max(I M (x,y),(x,y)∈N x,y (B))); Among them, N x,y (B) represents the neighborhood of the structural element B at position (x, y), I M (x,y) represents the intermediate processing feature map; S2-2-4: Fusion of frequency domain images I F (x,y) and fine edge image I GR (x,y), and perform superpixel segmentation on the fused image; S2-2-5: Cluster the superpixels and mark the target area, and calculate the confidence Cnf of the superpixels marked as the target area ifr ; η,λ and is the weight parameter, is the average edge strength within superpixel k, is the average value of the frequency domain image features within superpixel k; in the clustering process, the background cluster center μ1 and the target main area cluster center μ2 are used as optimization target parameters: H k is the kth superpixel in the nth superpixel; S3: Based on the target body area recognition result, the visible light image and the infrared image are fused to obtain an image to be detected; S4: Input the image to be detected into the improved YOLOV8 network for occluded target detection.

2. The method for detecting occluded targets in polar environments according to claim 1, characterized in that: Based on the target main area recognition result, fusing the visible light image and the infrared image to obtain the image to be detected specifically includes: S3-1: Based on the label window confidence score Cnf img And the confidence of superpixel Cnf ifr Generate a comprehensive confidence map Cnf cmb (x,y); Cnf cmb (x,y)=θ·Cnf img (x,y)+(1−θ)·Cnf ifr (x,y)? Among them, θ is the weight parameter, CNf img (x,y) and Cnf ifr (x, y) represents the window confidence score and superpixel confidence score of the image position (x, y) respectively. If the position (x, y) is not marked as the target body area by the visible light image or infrared image, the window confidence score or superpixel confidence score is 0; S3-2: Synthesize a multi-channel image as the image to be detected I(x, y).

3. The method for detecting occluded targets in polar environments according to claim 2, characterized in that: Before inputting the image to be detected into the improved YOLOV8 network for occluded target detection, it is also necessary to obtain a local attention feature map of the image to be detected. The local attention feature map acquisition includes: S5-1: Visible light image I img Convert to HSV color space; S5-2: Setting the local feature color HSV threshold: H low =50°, H high =60°, S low =0.2, S high =0.5, V low =0.7, V high =1.0; S5-3: For the area that has been determined to contain the target body, apply the local feature color HSV threshold to obtain a local attention feature map; Mask=(H low ≤H≤H high )∧(S low ≤S≤S high )∧(V low ≤V≤V high ); 4. The method for detecting occluded targets in polar environments according to claim 3, characterized in that: The improved YOLOV8 network includes a feature extraction network, a neck network and a detection head. The feature extraction network is composed of a comprehensive confidence map Cnf cmb (x, y) is composed of Resnet-50, and the input of the feature extraction network is the comprehensive confidence map Cnf cmb (x,y) The image to be detected after adjustment I`(x,y): I`(x,y)=Cnf cmb (x,y)⊙I(x,y); The Resnet-50 consists of five residual modules Res1-Res5, where the input of the residual module Res2 is the output of the residual module Res1 after adjusting the local attention feature map. composition: in, Represents the scaled local attention feature map f1(x,y) and They are the output feature map of Res1 and the input feature map of Res2 respectively; The output features of the residual modules Res3-Res5 are input to the neck network, which is defined as: Among them, f3(x,y), f4(x,y) and f5(x,y) are the output features of the residual module Res3-Res5, and are the feature maps of the middle layer of the neck network respectively; the inputs of the detection head are and The detection head adopts the Cascade RCNN network structure.

5. The method for detecting occluded targets in polar environments according to claim 1, characterized in that: The improved YOLOV8 network introduces Occlusion-aware Loss based on classification loss and regression loss, and its calculation method is as follows: L_occl=-log(p_i)*(1-IoU) 2 Among them, p_i represents the predicted probability of the target, and IoU represents the intersection over union ratio between the predicted box and the true box.

6. A system for detecting occluded targets in polar environments, characterized in that: The system is used to execute a method for detecting occluded targets in a polar environment as described in any one of claims 1 to 5, and the system comprises: Image acquisition equipment: The image acquisition equipment acquires visible light images and infrared images of polar regions; Target subject detection module: The target subject detection module performs target subject area recognition on the visible light image and the infrared image respectively; Occluded target detection: The occluded target detection module is used to input the image to be detected into the improved YOLOV8 network for occluded target detection module.

7. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for detecting occluded targets in a polar environment as described in any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements a method for detecting occluded targets in a polar environment as described in any one of claims 1 to 5.

Citation Information

Patent Citations

  • Fire identification method and device for target area, equipment and storage medium

    CN117710874A

  • Unmanned aerial vehicle night vision target recognition system and method based on multi-modal image fusion

    CN117975308A