Target detection method and device, electronic equipment and storage medium

Abstract

According to the target detection method and device, the electronic equipment and the storage medium provided by the embodiment of the invention, local comparison is performed on the current infrared image under different sliding window sizes (i.e., different sizes), so that a target can be enhanced and a background can be suppressed on different scales, local features can be enhanced on different scales, and the target detection efficiency can be improved. And the target detection accuracy is improved. Moreover, the method can adapt to targets of different sizes, effectively separates the targets on different scales, avoids the missing detection and false alarm of the local comparison of a conventional fixed scale, and improves the accuracy of target detection.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to target detection methods, devices, electronic devices and storage media. Background Art

[0002] In real-world scenarios, object detection is often performed by locally comparing the difference between a pixel and its adjacent pixels to enhance local image features. For example, in infrared images captured by image acquisition equipment, the grayscale value difference between a pixel and its adjacent pixels can be used to detect targets such as drones and birds.

[0003] In the above target detection process, the sizes of the regions where multiple adjacent pixels corresponding to different pixels are located are the same and fixed, that is, target detection is performed through local comparison at a fixed size.

[0004] However, in actual detection, the size of the target to be detected is variable. This fixed-size local comparison detection method is difficult to adapt to targets of different sizes and is prone to missed detections and false alarms, affecting the accuracy of target detection. Summary of the Invention

[0005] In view of this, embodiments of the present application provide a target detection method, device, electronic device, and storage medium to improve the accuracy of target detection.

[0006] The present invention provides a method for detecting a target, which includes:

[0007] Obtaining a local contrast value for each pixel in the current infrared image under different sliding window sizes; the local contrast value of any pixel under any sliding window size is used to indicate a comparison between the grayscale value of a central subwindow and the grayscale value of a non-central subwindow within a sliding window of the current infrared image centered on the pixel and having a size of the sliding window; the central subwindow in the sliding window is an area containing the center of the sliding window, and the non-central subwindow is an area in the sliding window adjacent to the central subwindow;

[0008] determining a target local contrast value of each pixel in the current infrared image based on the local contrast value of the pixel under different sliding window sizes;

[0009] Based on the panoramic image of the sky in the current application scenario, the sky area and the non-sky area are distinguished from the current infrared image; target detection is performed based on the target local contrast value of each pixel point in the sky area and in accordance with the target detection method corresponding to the sky area; target detection is performed based on the target local contrast value of each pixel point in the non-sky area and in accordance with the target detection method corresponding to the non-sky area; the target detection method corresponding to the sky area is different from the target detection method corresponding to the non-sky area.

[0010] The present invention further provides a target detection device, which includes:

[0011] An obtaining module is configured to obtain a local contrast value of each pixel in a current infrared image under different sliding window sizes; the local contrast value of any pixel under any sliding window size is configured to indicate a comparison between the grayscale value of a central subwindow and the grayscale value of a non-central subwindow within a sliding window having a size of the sliding window and centered on the pixel in the current infrared image; the central subwindow in the sliding window is an area containing the center of the sliding window, and the non-central subwindow is an area adjacent to the central subwindow in the sliding window;

[0012] a determination module, configured to determine a target local contrast value of each pixel in the current infrared image based on the local contrast value of the pixel under different sliding window sizes;

[0013] A detection module is used to distinguish the sky area and the non-sky area from the current infrared image based on the sky panoramic image in the current application scenario; perform target detection based on the target local contrast value of each pixel point in the sky area and in accordance with the target detection method corresponding to the sky area; perform target detection based on the target local contrast value of each pixel point in the non-sky area and in accordance with the target detection method corresponding to the non-sky area; the target detection method corresponding to the sky area is different from the target detection method corresponding to the non-sky area.

[0014] An embodiment of the present application further provides an electronic device, comprising: a processor and a memory for storing computer program instructions, wherein the computer program instructions, when executed by the processor, enable the processor to execute the steps of the above method.

[0015] An embodiment of the present application further provides a machine-readable storage medium, which stores computer program instructions. When the computer program instructions are executed, the steps of the above method can be implemented.

[0016] As can be seen from the above technical solution, in this embodiment, the local contrast value of each pixel in the current infrared image at different sliding window sizes is obtained. Based on the local contrast value of each pixel in the current infrared image at different sliding window sizes, the target local contrast value of the pixel is determined. Target detection is then performed using different target detection methods based on the target local contrast values ​​of each pixel in the sky area and the non-sky area. In this way, by performing local contrast on the current infrared image at different sliding window sizes (i.e., different sizes), not only can the target be enhanced and the background suppressed at different scales, local features can be enhanced at different scales, and target detection accuracy can be improved, but the method can also adapt to targets of different sizes and effectively separate targets at different scales, avoiding missed detections and false alarms that occur with traditional fixed-scale local contrast, thereby improving target detection accuracy.

[0017] Furthermore, after obtaining the local contrast value of each pixel in the current infrared image at different sliding window sizes, different target detection methods are used for target detection in the sky and non-sky regions based on the local contrast value of each pixel in each region at different sliding window sizes. This approach, using target detection methods that match each region for target detection, fully considers the characteristics of the sky and non-sky regions, effectively removing interference from sources of interference (such as fixed heat sources like buildings) in non-sky regions, and highlighting the features of moving targets (such as drones and birds), thereby further improving target detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0019] Figure 1 A schematic diagram of a target detection method according to an embodiment of the present invention;

[0020] Figure 2 A schematic diagram of a process for obtaining a target local contrast value provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of the structure of the current sliding window provided in an embodiment of the present application;

[0022] Figure 4 A flowchart of obtaining a panoramic sky image according to an embodiment of the present application;

[0023] Figure 5 A schematic diagram of sliding window segmentation of a panoramic image provided in an embodiment of the present application;

[0024] Figure 6A schematic diagram of a process for detecting targets in non-sky areas according to an embodiment of the present application;

[0025] Figure 7 A schematic diagram of a reference area provided for an embodiment of the present application;

[0026] Figure 8 A schematic diagram of the structure of a target detection device provided in an embodiment of the present application;

[0027] Figure 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application are further described in detail below with reference to the accompanying drawings.

[0029] It should be noted that the target detection method provided in the embodiment of the present application is suitable for detecting targets such as drones, birds, and clouds.

[0030] See also Figure 1 , Figure 1 Schematic diagram of the method flow provided in the embodiment of the present application. Optionally, the process can be executed by an embedded digital signal processor (DSP) integrated in the image acquisition device.

[0031] like Figure 1 As shown, the process may include the following steps:

[0032] S101, obtaining the local contrast value of each pixel in the current infrared image under different sliding window sizes; the local contrast value of any pixel under any sliding window size is used to indicate the comparison between the grayscale value of the central subwindow and the grayscale value of the non-central subwindow within a sliding window centered on the pixel and having a size of the sliding window in the current infrared image; the central subwindow in the sliding window is the area containing the center of the sliding window, and the non-central subwindow is the area adjacent to the central subwindow in the sliding window.

[0033] Here, the current infrared image may be acquired by an image acquisition device, such as a thermal imaging panoramic radar.

[0034] The sliding window size is set in advance, which limits the size of the center sub-window and the number and size of each non-center sub-window. The specific method of constructing the sliding window based on the sliding window size will be explained in detail later and will not be described here.

[0035] In infrared images, within a local area (i.e., within a sliding window), if there is a target, there is generally a certain contrast information between the target and the surrounding background, which has isolated saliency. The local contrast method can be used to separate the salient target. Obtaining the local contrast value of each pixel in the current infrared image under different sliding window sizes helps to separate targets at different scales.

[0036] The specific implementation method of obtaining the local contrast value of each pixel in the current infrared image under different sliding window sizes in step S101 will be described in detail later in the form of specific embodiments and will not be described in detail here.

[0037] S102 : determining a target local contrast value of each pixel in the current infrared image based on the local contrast value of the pixel under different sliding window sizes.

[0038] In this step, the local contrast value of each pixel in the current infrared image at different sliding window sizes contains the target separation information at different scales. Therefore, based on the local contrast value of each pixel in the current infrared image at different sliding window sizes, the target local contrast value of the pixel is determined. In this way, the target separation information at different scales can be fully retained, thereby adapting to targets of different sizes and different states (for example, drones hovering or moving), avoiding missed detections and false alarms of traditional fixed-scale local contrast.

[0039] The specific implementation of the above step S102 will be described in detail later in the form of specific embodiments and will not be repeated here.

[0040] S103, based on the sky panoramic image in the current application scenario, distinguish the sky area and the non-sky area from the current infrared image; based on the target local contrast value of each pixel point in the sky area and in accordance with the target detection method corresponding to the sky area, perform target detection; based on the target local contrast value of each pixel point in the non-sky area and in accordance with the target detection method corresponding to the non-sky area, perform target detection; the target detection method corresponding to the sky area is different from the target detection method corresponding to the non-sky area.

[0041] The complexity of the sky area and the non-sky area is different. In the non-sky area, for example, there are interference sources such as fixed heat sources on the ground. Therefore, in order to reduce the impact of the above interference factors, different target detection methods are used for target detection in the sky area and the non-sky area.

[0042] The specific implementation of the above step S103 will be described in detail later in the form of specific embodiments, which will not be described in detail here.

[0043] So far, completed Figure 1 The process shown.

[0044] pass Figure 1 As can be seen from the illustrated process, the local contrast value of each pixel in the current infrared image at different sliding window sizes is obtained. Based on the local contrast value of each pixel in the current infrared image at different sliding window sizes, the target local contrast value of that pixel is determined. Target detection is then performed using different target detection methods based on the target local contrast values ​​of each pixel in the sky area and the non-sky area. In this way, by performing local contrast on the current infrared image at different sliding window sizes (i.e., different scales), not only can the target be enhanced and the background suppressed at different scales, local features are enhanced at different scales, and target detection accuracy is improved, but the method can also adapt to targets of different sizes and effectively separate targets at different scales, avoiding missed detections and false alarms that occur with traditional fixed-scale local contrast, thereby improving target detection accuracy.

[0045] Furthermore, after obtaining the local contrast value of each pixel in the current infrared image at different sliding window sizes, different target detection methods are used for target detection in the sky and non-sky regions based on the local contrast value of each pixel in each region at different sliding window sizes. This approach, using target detection methods that match each region for target detection, fully considers the characteristics of the sky and non-sky regions, effectively removing interference from sources of interference (such as fixed heat sources like buildings) in non-sky regions, and highlighting the features of moving targets (such as drones and birds), thereby further improving target detection accuracy.

[0046] The following is a detailed description of how to obtain the local contrast value of each pixel in the current infrared image under different sliding window sizes:

[0047] See also Figure 2 , Figure 2 A schematic diagram of a process for obtaining local alignment values ​​provided in an embodiment of the present application.

[0048] like Figure 2 As shown, the process may include the following steps:

[0049] S201 , for each sliding window size, first traverse the current infrared image in sequence, and use the traversed current pixel point as the sliding window center to obtain a current sliding window corresponding to the sliding window size.

[0050] In this embodiment, different sliding window sizes may be 3×3, 5×5, 7×7, 9×9, etc. The current infrared image is traversed pixel by pixel from left to right and from top to bottom.

[0051] If the sliding window size is 3×3, the size of the central sub-window and the size of the non-central sub-window are specified to be 3*3=9 pixels, and the number of non-central sub-windows is specified to be 8, which are the sub-windows above the central sub-window along the top, bottom, left, right, upper left, upper right, lower left, and lower right and adjacent to the central sub-window.

[0052] If the sliding window size is 5×5, the size of the central sub-window and the size of the non-central sub-window are specified to be 5*5=25 pixels, and the number of non-central sub-windows is specified to be 8, which are the sub-windows adjacent to the central sub-window along the top, bottom, left, right, upper left, upper right, lower left, and lower right directions. The same applies to other sliding window sizes.

[0053] For example, if Figure 3 As shown, the current pixel point traversed is used as the center of the sliding window, and the specific structure of the current sliding window corresponding to the sliding window size is obtained as follows Figure 3 As shown in FIG5 , with the current pixel as the center, the adjacent pixels in the eight directions of up, down, left, right, upper left, upper right, lower left, and lower right and the pixel constitute a central sub-window T. The central sub-windows above the central sub-window T in the eight directions of up, down, left, right, upper left, upper right, lower left, and lower right and adjacent to the central sub-window, namely B1, B2, B3, B4, B5, B6, and B7, are non-central sub-windows.

[0054] S202, based on the grayscale value of the central sub-window and the grayscale value of the non-central sub-window in the current sliding window, determine the local contrast value of the current pixel point in the sliding window size; the non-central sub-window and the central sub-window have the same size, and the non-central sub-window is a sub-window of the central sub-window along the specified direction and adjacent to the central sub-window.

[0055] A specific implementation method for determining the local contrast value of the current pixel point at the sliding window size based on the grayscale value of the central sub-window and the grayscale value of the non-central sub-window in the current sliding window is as follows: for each non-central sub-window in the current sliding window, if the grayscale value of the central sub-window is less than or equal to the grayscale value of the non-central sub-window, then a first specified value (for example, 0) is determined as the reference local contrast value between the current pixel point and the non-central sub-window at the sliding window size.

[0056] If the grayscale value of the central subwindow is greater than the grayscale value of the non-central subwindow, a first specified operation is performed on the grayscale values ​​of the central subwindow and the non-central subwindow, and the result of the first specified operation is determined as the reference local contrast value between the current pixel and the non-central subwindow for the sliding window size. For example, the first specified operation is to calculate the square of the difference between the grayscale values ​​of the central subwindow and the non-central subwindow, and the resulting square value is used as the reference local contrast value.

[0057] The maximum value of the reference local contrast values ​​between the current pixel and each central sub-window in the sliding window size is determined as the local contrast value of the current pixel in the sliding window size.

[0058] Optionally, the grayscale value of the central sub-window can be represented by the grayscale value of a designated pixel in the central sub-window. For example, the grayscale value of the central sub-window can be the grayscale value of the pixel located at the center of the central sub-window (i.e., the current pixel). The grayscale value of the non-central sub-window can be represented by the grayscale value of a designated pixel in the non-central sub-window. For example, the grayscale value of the non-central sub-window can be the grayscale value of the pixel located at the center of the non-central sub-window.

[0059] Thus, when calculating local contrast values ​​for different sliding window sizes, the calculation is based on the grayscale value of a designated pixel in the center subwindow and the grayscale value of a designated pixel in each non-center subwindow (this can be called a local contrast operation for different sliding window sizes), rather than using the grayscale values ​​of all pixels in the center subwindow domain and each non-center subwindow. This ensures that the computational effort required to calculate local contrast values ​​is the same for different sliding window sizes, effectively reducing unnecessary computational overhead and providing strong real-time responsiveness, thereby ensuring efficient target detection. This is particularly true in embedded DSPs with limited computing power, where real-time requirements can be met.

[0060] For example, continuing with the above example, if the sliding window size is 3×3, the grayscale value of the center sub-window is the grayscale value of the pixel at the center of the center sub-window (that is, the current pixel). The grayscale value of the non-center sub-window is the grayscale value of the pixel at the center of the non-center sub-window.

[0061] The local contrast value of any pixel in the sliding window size is calculated by the following formula:

[0062]

[0063] Where C is the local contrast value of the center sub-window T under the sliding window size;

[0064] Ci is the reference local contrast value between the central sub-window T and the non-central sub-window Bi under the sliding window size;

[0065] c0 is the grayscale value of the pixel at the center of the central sub-window T;

[0066] b i is the grayscale value of the pixel located at the center of the non-center sub-window Bi.

[0067] S203: Determine whether the current pixel is the last pixel. If so, the traversal ends; if not, the traversal continues in order. The process returns to step S201, where the traversed current pixel is used as the sliding window center to obtain a current sliding window of the corresponding sliding window size.

[0068] So far, completed Figure 2 The process shown.

[0069] pass Figure 2 It can be seen from the process shown that determining the local contrast value of each pixel in the current infrared image under different sliding window sizes in the above way can not only adapt to targets of different sizes and provide a basis for effectively separating targets at different scales, but also effectively reduce unnecessary computational overhead and have strong real-time response capabilities.

[0070] The above is a detailed explanation of how to obtain the local contrast value of each pixel in the current infrared image under different sliding window sizes:

[0071] The following is a detailed description of how to determine the target local contrast value of each pixel in the current infrared image based on the local contrast value of the pixel under different sliding window sizes:

[0072] As an embodiment, based on the local contrast value of each pixel point in the current infrared image under different sliding window sizes, a specific implementation method for determining the target local contrast value of the pixel point may be: for each pixel point in the current infrared image, a second specified operation is performed on the local contrast value of the pixel point under different sliding window sizes, such as a weighted average operation, and the result of the second specified operation is determined as the target local contrast value of the pixel point.

[0073] As another embodiment, based on the local contrast value of each pixel point in the current infrared image under different sliding window sizes, a specific implementation method for determining the target local contrast value of the pixel point may also be: determining a specified local contrast value (for example, the maximum local contrast value, or the minimum local contrast value, etc.) among the local contrast values ​​of the pixel point under different sliding window sizes as the target local contrast value of the pixel point.

[0074] In this way, the local contrast value of each pixel in the current infrared image under different sliding window sizes can detect large targets (such as buildings, etc.) at a large scale, and can also detect small targets (such as birds, etc.) at a small scale. In order not to miss information, the maximum local contrast value of the pixel under different sliding window sizes is determined as the target local contrast value of the pixel, which can fully contain the information of each target to be detected, providing a basis for avoiding missed detection in subsequent target detection.

[0075] The above describes in detail how to determine the target local contrast value of each pixel in the current infrared image based on the local contrast value of the pixel under different sliding window sizes.

[0076] The following describes in detail how to distinguish the sky area from the non-sky area, detect targets in the sky area, and detect targets in the non-sky area from the current infrared image:

[0077] First, let’s explain how to distinguish the sky area from the non-sky area in the current infrared image:

[0078] The sky panoramic image in the current application scenario indicates a sky area and a non-sky area. According to the sky area and the non-sky area indicated by the sky panoramic image, the sky area and the non-sky area can be distinguished in the current infrared image.

[0079] For example, in combination Figure 4 As shown, the sky panoramic image is obtained by the following specific implementation method: the image acquisition device (such as a thermal imaging panoramic radar) stitches the infrared images collected by itself in a 360-degree rotation in the current application scenario, and outputs the panoramic image of the current application scenario.

[0080] According to the resolution of the device, the panoramic image is divided into slices, such as Figure 5 As shown, a slice size consistent with the device resolution is selected, and sliding slices are performed to crop the panorama into multiple slices.

[0081] For each slice, semantic segmentation is performed on it. For example, the slice is input into the obtained Transformer semantic segmentation network to obtain the probability value of each pixel in the slice belonging to the sky area. The probability value is compared with the set probability value threshold. Pixels with a probability value greater than the threshold are set to a non-zero specified value, such as 255. Pixels with an eigenvalue of 255 are in the sky area. Pixels with a probability value less than or equal to the threshold are set to zero. Pixels with an eigenvalue of 0 are in the non-sky area. In this way, the sky area and non-sky area in the slice are divided, and the determined sky area is masked. The sky mask images of each slice are spliced ​​together to obtain a panoramic sky image.

[0082] Accordingly, the sky area and the non-sky area of ​​the current infrared image are distinguished according to the indication of the mask in the image area of ​​the panoramic sky image that has the same visual sense as the current infrared image.

[0083] Optionally, the above-mentioned process of obtaining a panoramic sky image is executed offline by a neural-network processing unit (NPU) embedded in the thermal imaging panoramic radar. This application generates a panoramic sky image offline through the NPU, uses a high-precision Transformer segmentation network to crop the panoramic image into multiple slices, performs semantic segmentation, and finally splices it into a panoramic sky image. The panoramic sky image only needs to be generated offline once and can be called in the subsequent detection process. Compared with the related art in which the panoramic sky image is usually generated during the real-time detection process, this reduces the computational burden of real-time detection and helps improve the detection efficiency and accuracy of subsequent target detection.

[0084] Let's explain how to detect targets in the sky area:

[0085] As an example, target detection can be performed based on the target local contrast value of each pixel in the sky region and in accordance with the target detection method corresponding to the sky region. For each pixel in the sky region, if the target local contrast value of the pixel is greater than or equal to a set local contrast threshold for the sky region, the pixel is determined to be a target. If the target local contrast value of the pixel is less than the set local contrast threshold for the sky region, the pixel is determined not to be a target. Thus, through the above steps, target detection results for the sky region are obtained.

[0086] Finally, target detection in non-sky areas is described:

[0087] Generally speaking, if there is a target in the current infrared image, the target local contrast value of the corresponding pixel point is relatively large, that is, the target appears as grayscale singularity in the infrared image. Therefore, the detection rule is to detect the target in the sky area by comparing the target local contrast value with the local contrast threshold of the sky area.

[0088] However, in the non-sky area, the background is relatively complex, and there are interference sources such as fixed heat sources in the non-sky area, resulting in a large number of fixed grayscale singular points in the non-sky area. This leads to the detection rule of directly comparing the local contrast value of the target with the local contrast threshold of the non-sky area. This is not applicable in the non-sky area, and the detection accuracy is not high.

[0089] Therefore, it is necessary to make full use of the motion information between different frames of infrared images collected at the same viewing angle of the target to perform target detection, which is more conducive to the robust detection of the target. At the same time, considering that the image acquisition device has poor uniform rotation performance, the background of the non-sky area in different infrared images at the same viewing angle will also be offset, that is, the information collected in the current infrared image and the previous infrared image at the same viewing angle cannot be completely aligned. Therefore, through Figure 6 The method shown is used to detect targets in non-sky areas to improve the accuracy of target detection.

[0090] See also Figure 6 , Figure 6 A flowchart of non-sky area target detection provided in an embodiment of the present application.

[0091] like Figure 6 As shown, the process may include the following steps:

[0092] S601 , obtaining a previous infrared image frame captured at the same viewing angle as that of the current infrared image.

[0093] S602, for each pixel point in the non-sky area, performing a third specified operation on the target local contrast value of the pixel point and the target local contrast value of the target pixel point that matches the pixel point in the previous frame of infrared image; the target pixel point is: the pixel point in the previous frame of infrared image that has the smallest difference in target local contrast value from the pixel point within a reference area centered on a pixel point at the same coordinate position as the pixel point and having a size of the specified area.

[0094] The target pixel is the pixel whose pixel value (here, the target local contrast value) is closest to the pixel in the neighborhood of the pixel at the same coordinate position in the previous frame of infrared image. The so-called closest means that the absolute value of the difference between the target local contrast values ​​is the smallest.

[0095] For example, the pixel point at the pixel coordinate (i0, j0) in the current infrared image k is recorded as (i0, j 0, , k).

[0096] The pixel point at the pixel coordinate (i0, j0) in the previous frame of infrared image k-1 is recorded as (i0, j 0, , k-1), with pixel (i0, j 0, , k-1) as the center of the reference area is recorded as (i0-i m <i′0<i0+i m , j0-j m <j′0<j0+j m , k-1), reference area see Figure 7 shown.

[0097] The target pixel is (i0-i m <i′0<i0+i m , j0-j m <j′0<j0+j m , k-1) and the corresponding target local contrast value and pixel point (i0, j 0, , k) is the pixel with the smallest absolute value of the difference between the target local contrast values. The local to target local contrast value of the minimum pixel is recorded as:

[0098]

[0099] Optionally, as an embodiment, the third designated operation is difference calculation, which calculates the difference between the target local contrast value of the pixel point and the target local contrast value of the target pixel point matching the pixel point in the previous frame of infrared image.

[0100] The third specified operation result corresponding to any pixel point can be calculated using the following formula:

[0101] Δf=f(i0,j0,k)-f(i′0,j′0,k-1)

[0102]

[0103] Where f(i0, j0, k) is the pixel point (i0, j0) at the pixel coordinate (i0, j0) in the current infrared image k. 0, , k) target local contrast value;

[0104] f(i′0,j′0,k-1) is the pixel in the previous frame of infrared image k-1 that corresponds to the pixel (i0, j 0, , k) the target local contrast value of the matched target pixel

[0105]

[0106] It should be noted that the above steps S601 and S602 can also be referred to as inter-frame neighborhood difference filtering operations.

[0107] S603 , for each pixel in the non-sky area, if the third specified operation result of the pixel is greater than or equal to the set non-sky area local contrast threshold, determine that the pixel belongs to the target.

[0108] S604: If the third specified operation result of the pixel point is less than the set non-sky area local contrast threshold, it is determined that the pixel point does not belong to the target.

[0109] Through the above steps S603 and S604, the target detection result in the non-sky area is obtained.

[0110] So far, completed Figure 6 The process shown.

[0111] pass Figure 6 As can be seen from the process shown, for each pixel in the non-sky area, the target pixel is the pixel with the pixel value closest to the pixel in the neighborhood of the pixel at the same coordinate position as the pixel in the previous frame of infrared image, rather than the pixel at the same coordinate position as the pixel in the previous frame of infrared image. Calculations based on this can effectively eliminate misalignment caused by machine shaking.

[0112] On this basis, target detection in non-sky areas is performed by comparing the target local contrast values ​​of pixel points in the non-sky area of ​​the two frames of infrared images collected at the same angle. This makes full use of the fact that the target has more significant inter-frame motion characteristics than the background. For example, the motion characteristics of the target to be detected (such as a drone) in the two frames of infrared images are more significant than those of the background facilities in the non-sky area (such as buildings), thereby reducing interference from the ground or other interference sources, highlighting the characteristics of the moving target, and further improving the accuracy of target detection.

[0113] As an embodiment, after obtaining target detection results for the sky region and target detection results for the non-sky region, the method further includes: concatenating the target detection results for the sky region of the current infrared image and the target detection results for the non-sky region of the current infrared image to obtain a target detection result for the current infrared image. The target detection results for the current infrared image are output to the obtained target classification model to identify the category of each target in the target detection results.

[0114] In the above embodiment, the target detection result of the current infrared image is classified by the target classification model to identify targets such as drones, birds, and clouds, thereby further improving the detection accuracy and reducing the possibility of false alarms.

[0115] As a further embodiment, the current infrared image is matched with the global image, and the category of each target in the current infrared image is displayed in the global image.

[0116] In order to explain the method provided by this application in more detail, Figure 4 The solution provided in this application is described in more detail by way of specific embodiments.

[0117] The process may include the following steps:

[0118] 1. Offline generation of panoramic sky images:

[0119] The panoramic image is formed by stitching together the infrared images collected by the thermal imaging panoramic radar rotating 360°.

[0120] The NPU selects a window that matches the device's resolution offline, performs sliding slicing, and crops the panorama into multiple slices. Each slice is input into the Transformer semantic segmentation network to obtain a predicted value for each pixel. Pixels with a probability value greater than the threshold are set to 255, and pixels with an eigenvalue of 255 are sky areas. Pixels with a probability value less than or equal to the threshold are set to zero, and pixels with an eigenvalue of 0 are non-sky areas, thereby obtaining a sky mask image for the slice. The sky mask images of each slice are spliced ​​together to obtain a sky panoramic image.

[0121] 2. Comparison of local adoption under different sliding window sizes:

[0122] DSP determines the sliding window size as 3×3, 5×5, and 7×7. Under each sliding window size, the current infrared image is traversed pixel by pixel from left to right and from top to bottom.

[0123] Substitute the grayscale value of the pixel at the center of the central sub-window in the current sliding window under the sliding window size and the grayscale value of the pixel at the center of the non-central sub-window into the following formula to calculate the local contrast value of the current pixel under the sliding window size.

[0124]

[0125] After the traversal is completed, for each pixel point, the maximum local contrast value among the local contrast values ​​of the pixel point under different sliding window sizes is determined as the target local contrast value of the pixel point.

[0126] In this way, the target local contrast value of each pixel in the current infrared image is obtained.

[0127] 3. Distinguish the sky area and the non-sky area of ​​the current infrared image according to the indication of the mask in the image area of ​​the panoramic sky image that has the same visual sense as the current infrared image.

[0128] 4. Based on the local contrast value of the target of each pixel in the sky area and according to the target detection method corresponding to the sky area, target detection is performed to obtain the target detection result of the sky area.

[0129] 5. Perform inter-frame neighborhood difference filtering on the non-sky area and obtain the neighborhood difference result of each pixel in the non-sky area:

[0130] An inter-frame neighborhood difference filtering operation is performed on a non-sky area in the current infrared image and a non-sky area in a previous infrared image acquired at the same viewing angle as the current infrared image.

[0131] For each pixel in the non-sky area, the target local contrast value of the pixel and each pixel in the reference area that matches the pixel in the previous infrared image are calculated according to the following formula to determine the target pixel in the reference area that matches the pixel.

[0132] Target pixel When i′0,j′0.

[0133] The target local comparison value of the pixel point and the target local comparison value of the target pixel point that matches the pixel point are calculated by the following formula as the neighborhood difference result of the pixel point.

[0134] Δf=f(i0,j0,k)-f(i′0,j′0,k-1)

[0135]

[0136] 6. For each pixel in the non-sky area, target detection is performed based on the neighborhood difference result of the pixel and the set local contrast threshold of the non-sky area to obtain the target detection result of the non-sky area.

[0137] For each pixel in the non-sky area, if the neighborhood difference result of the pixel is greater than or equal to the set non-sky area local contrast threshold, the pixel is determined to belong to the target.

[0138] If the neighborhood difference result of the pixel point is less than the set local contrast threshold of the non-sky area, it is determined that the pixel point does not belong to the target.

[0139] 7. The target detection result of the sky area of ​​the current infrared image obtained is spliced ​​with the target detection result of the non-sky area of ​​the current infrared image obtained to obtain the target detection result of the current infrared image.

[0140] 8. Output the target detection result of the current infrared image to the target classification model to identify the category of each target in the target detection result.

[0141] 9. Match the current infrared image with the global image, and display the categories of each target in the current infrared image in the global image.

[0142] The above describes the method provided in the embodiment of the present application. The following describes the device provided in the embodiment of the present application:

[0143] See also Figure 8 , Figure 8 This is a diagram of the structure of the device provided in the embodiment of the present application. The device is applied to electronic equipment, such as Figure 8As shown, the device may include: an obtaining module 801 , a determining module 802 , and a detecting module 803 .

[0144] An obtaining module 801 is configured to obtain a local contrast value for each pixel in the current infrared image at different sliding window sizes. The local contrast value of any pixel at any sliding window size indicates a comparison between the grayscale value of a central subwindow and the grayscale value of a non-central subwindow within a sliding window of the current infrared image centered on the pixel and having a size equal to the sliding window size. The central subwindow in the sliding window is defined as an area containing the center of the sliding window, and the non-central subwindow is defined as an area adjacent to the central subwindow in the sliding window.

[0145] A determination module 802 is configured to determine a target local contrast value of each pixel in the current infrared image based on the local contrast value of the pixel under different sliding window sizes;

[0146] Detection module 803 is used to distinguish the sky area and the non-sky area from the current infrared image based on the sky panoramic image in the current application scenario; perform target detection based on the target local contrast value of each pixel point in the sky area and in accordance with the target detection method corresponding to the sky area; perform target detection based on the target local contrast value of each pixel point in the non-sky area and in accordance with the target detection method corresponding to the non-sky area; the target detection method corresponding to the sky area is different from the target detection method corresponding to the non-sky area.

[0147] As an embodiment, obtaining the local contrast value of each pixel in the current infrared image under different sliding window sizes includes:

[0148] For each sliding window size, first traverse the current infrared image in order, and use the current pixel point traversed as the center of the sliding window to obtain the current sliding window corresponding to the sliding window size;

[0149] Determine the local contrast value of the current pixel in the sliding window size based on the grayscale value of the central subwindow and the grayscale value of the non-central subwindow in the current sliding window; the non-central subwindow and the central subwindow have the same size, and the non-central subwindow is a subwindow of the central subwindow along a specified direction and adjacent to the central subwindow;

[0150] If the current pixel is not the last pixel, continue to traverse in order and return the current pixel as the center of the sliding window to obtain the current sliding window corresponding to the sliding window size.

[0151] As an embodiment, determining the local contrast value of the current pixel point in the sliding window size based on the grayscale value of the central sub-window and the grayscale values ​​of the non-central sub-windows in the current sliding window includes:

[0152] For each non-center sub-window in the current sliding window, if the grayscale value of the center sub-window is less than or equal to the grayscale value of the non-center sub-window, determine the first specified value as the reference local contrast value between the current pixel and the non-center sub-window at the sliding window size;

[0153] If the grayscale value of the central sub-window is greater than the grayscale value of the non-central sub-window, a first specified operation is performed on the grayscale value of the central sub-window and the grayscale value of the non-central sub-window, and the result of the first specified operation is determined as a reference local contrast value between the current pixel point and the non-central sub-window at the sliding window size;

[0154] The maximum value of the reference local contrast values ​​between the current pixel and each central sub-window in the sliding window size is determined as the local contrast value of the current pixel in the sliding window size.

[0155] As an embodiment, determining a target local contrast value of each pixel in the current infrared image based on the local contrast value of the pixel under different sliding window sizes includes:

[0156] For each pixel in the current infrared image, performing a second specified operation on the local contrast value of the pixel under different sliding window sizes, and determining the second specified operation result as the target local contrast value of the pixel;

[0157] or,

[0158] A designated local contrast value among the local contrast values ​​of the pixel under different sliding window sizes is determined as a target local contrast value of the pixel.

[0159] As an embodiment, target detection based on the target local contrast value of each pixel point in the sky area and in accordance with the target detection method corresponding to the sky area includes:

[0160] For each pixel in the sky area, if the target local contrast value of the pixel is greater than or equal to the set sky area local contrast threshold, the pixel is determined to belong to the target;

[0161] If the target local contrast value of the pixel is less than the set local contrast threshold of the sky area, it is determined that the pixel does not belong to the target.

[0162] As an embodiment, target detection based on the target local contrast value of each pixel in the non-sky area and in accordance with the target detection method corresponding to the non-sky area includes:

[0163] Obtain the previous infrared image captured at the same viewing angle as the current infrared image;

[0164] For each pixel in the non-sky area, a third specified operation is performed on the target local contrast value of the pixel and the target local contrast value of the target pixel matching the pixel in the previous infrared image frame; the target pixel is the pixel with the smallest difference in target local contrast value from the pixel within a reference area of ​​the specified area centered at the pixel at the same coordinate position as the pixel in the previous infrared image frame;

[0165] For each pixel in the non-sky area, if the result of the third specified operation on the pixel is greater than or equal to the set non-sky area local contrast threshold, the pixel is determined to belong to the target;

[0166] If the third specified operation result of the pixel point is less than the set non-sky area local contrast threshold, it is determined that the pixel point does not belong to the target.

[0167] As an embodiment, the sky panoramic image is obtained by the following steps:

[0168] Obtain a panoramic image of the current application scenario; the panoramic image is formed by stitching together the infrared images captured by the image acquisition device through 360-degree rotation scanning of the device;

[0169] Segment the panoramic image according to the resolution of the device;

[0170] For each slice, perform semantic segmentation on the slice to determine the sky area and non-sky area in the slice, and perform mask processing on the determined sky area to obtain a sky mask image of the slice;

[0171] The sky mask images of each tile are stitched together to obtain a sky panoramic image.

[0172] As an embodiment, the detection module is further configured to:

[0173] splicing the obtained target detection result of the sky area of ​​the current infrared image and the obtained target detection result of the non-sky area of ​​the current infrared image to obtain the target detection result of the current infrared image;

[0174] The target detection result of the current infrared image is output to the target classification model, and the category of each target in the target detection result is identified.

[0175] So far, completed Figure 8 Structural description of the device shown.

[0176] See Figure 9 , Figure 9 This is a structural diagram of an electronic device provided in an embodiment of the present application. Figure 9As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0177] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.

[0178] Exemplarily, the machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that may contain or store information, such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, a storage drive (such as a hard disk drive), a solid-state drive, any type of storage disk (such as a CD, DVD, etc.), or similar storage media, or a combination thereof.

[0179] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A target detection method, characterized in that: The method comprises: Obtaining a local contrast value for each pixel in the current infrared image under different sliding window sizes; the local contrast value of any pixel under any sliding window size is used to indicate a comparison between the grayscale value of a central subwindow and the grayscale value of a non-central subwindow within a sliding window of the current infrared image centered on the pixel and having a size of the sliding window; the central subwindow in the sliding window is an area containing the center of the sliding window, and the non-central subwindow is an area in the sliding window adjacent to the central subwindow; determining a target local contrast value of each pixel in the current infrared image based on the local contrast value of the pixel under different sliding window sizes; Based on the panoramic image of the sky in the current application scenario, the sky area and the non-sky area are distinguished from the current infrared image; target detection is performed based on the target local contrast value of each pixel point in the sky area and in accordance with the target detection method corresponding to the sky area; target detection is performed based on the target local contrast value of each pixel point in the non-sky area and in accordance with the target detection method corresponding to the non-sky area; the target detection method corresponding to the sky area is different from the target detection method corresponding to the non-sky area.

2. The method according to claim 1, characterized in that The obtaining of the local contrast value of each pixel in the current infrared image under different sliding window sizes includes: For each sliding window size, the current infrared image is first traversed in order, and the current pixel point traversed is used as the center of the sliding window to obtain a current sliding window corresponding to the sliding window size; Determine the local contrast value of the current pixel in the sliding window size based on the grayscale value of the central subwindow and the grayscale value of the non-central subwindow in the current sliding window; the non-central subwindow and the central subwindow have the same size, and the non-central subwindow is a subwindow of the central subwindow along a specified direction and adjacent to the central subwindow; If the current pixel point is not the last pixel point, the traversal continues in order, and the current pixel point traversed is returned as the center of the sliding window to obtain the current sliding window corresponding to the sliding window size.

3. The method according to claim 2, characterized in that Determining the local contrast value of the current pixel point in the sliding window size based on the grayscale value of the central sub-window and the grayscale values ​​of the non-central sub-windows in the current sliding window includes: For each non-center sub-window in the current sliding window, if the grayscale value of the center sub-window is less than or equal to the grayscale value of the non-center sub-window, determine the first specified value as a reference local contrast value between the current pixel and the non-center sub-window at the sliding window size; If the grayscale value of the central sub-window is greater than the grayscale value of the non-central sub-window, performing a first specified operation on the grayscale value of the central sub-window and the grayscale value of the non-central sub-window, and determining the result of the first specified operation as a reference local contrast value between the current pixel and the non-central sub-window at the sliding window size; The maximum value of the reference local contrast values ​​between the current pixel and each central sub-window in the sliding window size is determined as the local contrast value of the current pixel in the sliding window size.

4. The method according to claim 2, characterized in that Determining a target local contrast value of each pixel in the current infrared image based on the local contrast value of the pixel under different sliding window sizes includes: For each pixel in the current infrared image, performing a second specified operation on the local contrast value of the pixel under different sliding window sizes, and determining the second specified operation result as the target local contrast value of the pixel; or, A designated local contrast value among the local contrast values ​​of the pixel under different sliding window sizes is determined as a target local contrast value of the pixel.

5. The method according to claim 2, characterized in that The target detection based on the target local contrast value of each pixel point in the sky area and in accordance with the target detection method corresponding to the sky area includes: For each pixel in the sky area, if the target local contrast value of the pixel is greater than or equal to the set sky area local contrast threshold, then the pixel is determined to belong to the target; If the target local contrast value of the pixel point is less than the set local contrast threshold of the sky area, it is determined that the pixel point does not belong to the target.

6. The method according to claim 2, characterized in that The target detection based on the target local contrast value of each pixel point in the non-sky area and in accordance with the target detection method corresponding to the non-sky area includes: Obtaining a previous infrared image captured at the same viewing angle as that of the current infrared image; For each pixel in the non-sky area, a third specified operation is performed on a target local contrast value of the pixel and a target local contrast value of a target pixel in the previous infrared image frame that matches the pixel; the target pixel being a pixel in the previous infrared image frame that has a minimum difference in target local contrast value from the pixel within a reference area having a size of a specified area and centered at a pixel at the same coordinate position as the pixel; For each pixel in the non-sky area, if the third specified operation result of the pixel is greater than or equal to the set non-sky area local contrast threshold, then the pixel is determined to belong to the target; If the third designated operation result of the pixel point is less than the set non-sky area local contrast threshold, it is determined that the pixel point does not belong to the target.

7. The method according to claim 2, characterized in that The sky panoramic image is obtained by the following steps: Obtaining a panoramic image in the current application scenario; the panoramic image is formed by stitching together infrared images acquired by 360-degree rotation scanning of an image acquisition device; Slicing the panoramic image according to the resolution of the device; For each slice, perform semantic segmentation on the slice to determine the sky area and non-sky area in the slice, and perform mask processing on the determined sky area to obtain a sky mask image of the slice; The sky mask images of each slice are stitched together to obtain the sky panoramic image.

8. The method according to claim 1, characterized in that The method further comprises: splicing the obtained target detection result of the sky area of ​​the current infrared image and the obtained target detection result of the non-sky area of ​​the current infrared image to obtain the target detection result of the current infrared image; The obtained target detection result of the current infrared image is output to the obtained target classification model to identify the category of each target in the target detection result.

9. A target detection device, characterized in that: The device comprises: An obtaining module is configured to obtain a local contrast value of each pixel in a current infrared image under different sliding window sizes; the local contrast value of any pixel under any sliding window size is configured to indicate a comparison between the grayscale value of a central subwindow and the grayscale value of a non-central subwindow within a sliding window having a size of the sliding window and centered on the pixel in the current infrared image; the central subwindow in the sliding window is an area containing the center of the sliding window, and the non-central subwindow is an area adjacent to the central subwindow in the sliding window; a determination module, configured to determine a target local contrast value of each pixel in the current infrared image based on the local contrast value of the pixel under different sliding window sizes; A detection module is used to distinguish the sky area and the non-sky area from the current infrared image based on the sky panoramic image in the current application scenario; perform target detection based on the target local contrast value of each pixel point in the sky area and in accordance with the target detection method corresponding to the sky area; perform target detection based on the target local contrast value of each pixel point in the non-sky area and in accordance with the target detection method corresponding to the non-sky area; the target detection method corresponding to the sky area is different from the target detection method corresponding to the non-sky area.

10. The device according to claim 9, characterized in that The obtaining of the local contrast value of each pixel in the current infrared image under different sliding window sizes includes: For each sliding window size, the current infrared image is first traversed in order, and the current pixel point traversed is used as the center of the sliding window to obtain a current sliding window corresponding to the sliding window size; Determine the local contrast value of the current pixel in the sliding window size based on the grayscale value of the central subwindow and the grayscale value of the non-central subwindow in the current sliding window; the non-central subwindow and the central subwindow have the same size, and the non-central subwindow is a subwindow of the central subwindow along a specified direction and adjacent to the central subwindow; If the current pixel point is not the last pixel point, continue to traverse in order, return the current pixel point traversed as the center of the sliding window to obtain the current sliding window corresponding to the sliding window size; and / or, Determining the local contrast value of the current pixel point in the sliding window size based on the grayscale value of the central sub-window and the grayscale values ​​of the non-central sub-windows in the current sliding window includes: For each non-center sub-window in the current sliding window, if the grayscale value of the center sub-window is less than or equal to the grayscale value of the non-center sub-window, determine the first specified value as a reference local contrast value between the current pixel and the non-center sub-window at the sliding window size; If the grayscale value of the central sub-window is greater than the grayscale value of the non-central sub-window, performing a first specified operation on the grayscale value of the central sub-window and the grayscale value of the non-central sub-window, and determining the result of the first specified operation as a reference local contrast value between the current pixel and the non-central sub-window at the sliding window size; The maximum value of the reference local contrast values ​​between the current pixel and each central sub-window under the sliding window size is determined as the local contrast value of the current pixel under the sliding window size; and / or, Determining a target local contrast value of each pixel in the current infrared image based on the local contrast value of the pixel under different sliding window sizes includes: For each pixel in the current infrared image, performing a second specified operation on the local contrast value of the pixel under different sliding window sizes, and determining the second specified operation result as the target local contrast value of the pixel; or, determining a designated local contrast value of the local contrast values ​​of the pixel point under different sliding window sizes as a target local contrast value of the pixel point; and / or, The target detection based on the target local contrast value of each pixel point in the sky area and in accordance with the target detection method corresponding to the sky area includes: For each pixel in the sky area, if the target local contrast value of the pixel is greater than or equal to the set sky area local contrast threshold, then the pixel is determined to belong to the target; If the target local contrast value of the pixel point is less than the set local contrast threshold of the sky area, it is determined that the pixel point does not belong to the target; and / or, The target detection based on the target local contrast value of each pixel point in the non-sky area and in accordance with the target detection method corresponding to the non-sky area includes: Obtaining a previous infrared image captured at the same viewing angle as that of the current infrared image; For each pixel in the non-sky area, a third specified operation is performed on a target local contrast value of the pixel and a target local contrast value of a target pixel in the previous infrared image frame that matches the pixel; the target pixel being a pixel in the previous infrared image frame that has a minimum difference in target local contrast value from the pixel within a reference area having a size of a specified area and centered at a pixel at the same coordinate position as the pixel; For each pixel in the non-sky area, if the third specified operation result of the pixel is greater than or equal to the set non-sky area local contrast threshold, then the pixel is determined to belong to the target; If the third specified operation result of the pixel point is less than the set non-sky area local contrast threshold, it is determined that the pixel point does not belong to the target; and / or, The sky panoramic image is obtained by the following steps: Obtaining a panoramic image in the current application scenario; the panoramic image is formed by stitching together infrared images acquired by 360-degree rotation scanning of an image acquisition device; Slicing the panoramic image according to the resolution of the device; For each slice, perform semantic segmentation on the slice to determine the sky area and non-sky area in the slice, and perform mask processing on the determined sky area to obtain a sky mask image of the slice; splicing the sky mask images of each slice to obtain the sky panoramic image; and / or, The detection module is further used for: splicing the obtained target detection result of the sky area of ​​the current infrared image and the obtained target detection result of the non-sky area of ​​the current infrared image to obtain the target detection result of the current infrared image; The obtained target detection result of the current infrared image is output to the obtained target classification model to identify the category of each target in the target detection result.

11. An electronic device, characterized in that: The electronic device includes: processor; and A computer-readable storage medium having computer program instructions stored therein, wherein the computer program instructions, when executed by the processor, cause the processor to perform the steps of the method according to any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer program instructions, which, when executed by a processor, enable the processor to perform the steps of the method according to any one of claims 1 to 8.

Images