A three-dimensional target detection method based on remote sensing images

Abstract

The application relates to the technical field of image region area, in particular to a three-dimensional target detection method based on a remote sensing image, which comprises the following steps: acquiring a to-be-detected image, performing illumination influence degree division on the to-be-detected image; determining a corresponding gray fluctuation index of each target illumination region; correcting a preset similarity threshold; screening a similar pixel point group from each preset direction of each pixel point in the target illumination region; determining a corresponding structure feature and an abnormal score of each pixel point; according to the abnormal scores of all the pixel points in the to-be-detected image, extracting feature points from the to-be-detected image through a SIFT (Scale-Invariant Feature Transform) corner point detection algorithm; and performing target identification on the to-be-detected image through a DETR (DEtection TRansformer) algorithm according to all the extracted feature points. Through image data processing on the to-be-detected image, the application realizes target detection on the to-be-detected image and improves the accuracy of target detection.

Description

Technical Field

[0001] This invention relates to the field of image region area technology, and more specifically to a three-dimensional target detection method based on remote sensing images. Background Technology

[0002] Remote sensing images are images of the Earth's surface acquired through satellite sensors. Because remote sensing images have a wide field of view and relatively complete details of ground features, they are widely used in the investigation of Earth's surface resources, environmental monitoring, and the analysis and prediction of disasters. However, the uneven illumination of target areas in ground feature information leads to low target detection accuracy in remote sensing images.

[0003] The DETR algorithm is a common deep learning-based object detection algorithm. When performing DETR model recognition, it is often necessary to extract feature points from the image to construct the end-to-end model. Currently, the SIFT corner detection algorithm is commonly used to extract feature points from the image.

[0004] However, when using the SIFT corner detection algorithm to extract feature points from an image, the following technical problems often arise:

[0005] Since the feature points extracted by the SIFT algorithm are often local extrema in scale space, when using the DETR algorithm to identify external objects, the feature points extracted by the SIFT corner detection algorithm may be mistaken for dirt spots and other features, resulting in poor accuracy of subsequent target detection. Summary of the Invention

[0006] The summary section of this invention provides a brief overview of the concepts, which will be described in detail in the detailed description section that follows. This summary section is not intended to identify key or essential features of the claimed invention, nor is it intended to limit the scope of the claimed invention.

[0007] To address the technical problem of poor accuracy in target detection, this invention proposes a three-dimensional target detection method based on remote sensing images.

[0008] This invention provides a three-dimensional target detection method based on remote sensing images, the method comprising:

[0009] Acquire the image to be detected, and divide the image to be detected into a number of target illumination regions by the degree of illumination influence.

[0010] Based on the grayscale values ​​of all pixels within each target illumination area, determine the grayscale fluctuation index corresponding to each target illumination area.

[0011] Based on the number of gray values, area, and gray fluctuation index of each target illumination area, the preset similarity threshold is corrected to obtain the target similarity threshold for each target illumination area.

[0012] Based on the target similarity threshold corresponding to each target illumination region, similar pixel point groups are selected from each preset direction of each pixel point within the target illumination region;

[0013] Based on the similar pixel groups in all preset directions of each pixel within each target illumination area, and the grayscale in the vertical direction of all preset directions of each pixel, determine the structural features corresponding to each pixel.

[0014] Based on the structural features corresponding to all pixels in the image to be detected, the anomaly score corresponding to each pixel is obtained through the isolated forest algorithm.

[0015] Based on the anomaly scores corresponding to all pixels in the image to be detected, feature points are extracted from the image to be detected using the SIFT corner detection algorithm;

[0016] Based on all extracted feature points, the DETR algorithm is used to perform target recognition on the image to be detected.

[0017] Optionally, the step of dividing the image to be detected into a predetermined number of target illumination regions by dividing the image into regions affected by illumination includes:

[0018] Based on the preset window corresponding to each pixel in the image to be detected, determine the light source confidence level corresponding to each pixel;

[0019] Pixels with a light source confidence greater than a preset light source threshold are selected from the image to be detected and designated as light source pixels. Pixels in the image to be detected other than light source pixels are designated as non-light source pixels.

[0020] The halo probability of each non-light source pixel is determined based on the minimum distance between each non-light source pixel and all light source pixels, the mean gray value of all pixels in the preset window corresponding to each non-light source pixel, and the variance of the gray value of all pixels in the preset window corresponding to each non-light source pixel.

[0021] Non-light source pixels with a halo probability greater than a preset halo threshold are selected from the image to be detected and designated as halo pixels. Each halo pixel and each light source pixel in the image to be detected are recorded as a target light source point.

[0022] The region containing all target light source points in the image to be detected is defined as the light source region;

[0023] Using the Otsu method, the region in the image to be detected, excluding the light source region, is divided to obtain the shadow region and the illumination region.

[0024] The light source area, shadow area, and illumination area are all denoted as target illumination areas, resulting in a preset number of target illumination areas, of which the preset number is 3.

[0025] Optionally, determining the light source confidence level corresponding to each pixel based on a preset window corresponding to each pixel in the image to be detected includes:

[0026] Any pixel in the image to be detected is designated as a marker pixel. The light source confidence level of the marker pixel is determined based on the mean of the gray values ​​of all pixels within a preset window corresponding to the marker pixel and the variance of the gray values ​​of all pixels within the preset window corresponding to the marker pixel. The mean of the gray values ​​of all pixels within the preset window is positively correlated with the light source confidence level, and the variance of the gray values ​​of all pixels within the preset window is negatively correlated with the light source confidence level.

[0027] Optionally, the formula for the halo probability corresponding to a non-light source pixel is:

[0028] ;in, is the halo probability corresponding to the j-th non-light source pixel in the image to be detected; j is the index of the non-light source pixel in the image to be detected; It is a normalization function; It is the minimum distance between the j-th non-light source pixel and all light source pixels; It is the average grayscale value of all pixels within the preset window corresponding to the j-th non-light source pixel; It is the variance of the grayscale values ​​of all pixels within the preset window corresponding to the j-th non-light source pixel.

[0029] Optionally, determining the grayscale fluctuation index corresponding to each target illumination area based on the grayscale values ​​corresponding to all pixels within each target illumination area includes:

[0030] Based on the variance of the gray values ​​corresponding to all pixels within the target illumination area and the number of gray values ​​corresponding to the target illumination area, a gray fluctuation index corresponding to the target illumination area is determined. The variance of the gray values ​​corresponding to all pixels within the target illumination area and the number of gray values ​​corresponding to them are positively correlated with the corresponding gray fluctuation index.

[0031] Optionally, the formula for the target similarity threshold corresponding to the target illumination area is:

[0032] ;in, It is the target similarity threshold corresponding to the t-th target illumination region; t is the index of the target illumination region. It is the number of grayscale values ​​corresponding to the t-th target illumination region; It is the grayscale fluctuation index corresponding to the t-th target illumination area; It is an exponential function with the natural constant as its base; It is the area corresponding to the t-th target illumination region; It is a preset similarity threshold.

[0033] Optionally, the step of filtering similar pixel groups from each pixel in each preset direction within the target illumination area based on a target similarity threshold corresponding to each target illumination area includes:

[0034] Any pixel within the target illumination area is designated as a reference pixel, any preset direction is designated as a marking direction, and the absolute value of the difference between the gray value corresponding to the reference pixel and the gray value corresponding to each pixel in the marking direction is designated as the gray value difference.

[0035] Pixels whose grayscale difference is less than or equal to the target similarity threshold corresponding to the target illumination area to which the reference pixel belongs are selected from the marked direction of the reference pixel and are regarded as similar pixels.

[0036] Form a segment of similar pixels by dividing consecutively distributed similar pixels;

[0037] Select similar pixel segments that are adjacent to the reference pixel from all similar pixel segments to form a similar pixel group.

[0038] Optionally, determining the structural features corresponding to each pixel based on the similar pixel groups in all preset directions of each pixel within each target illumination area, and the grayscale values ​​in the vertical direction of all preset directions of each pixel, includes:

[0039] Any pixel within the target illumination area is designated as a reference pixel, and any preset direction is determined as a marking direction. The grayscale similarity index of the reference pixel in the marking direction is determined based on the number of similar pixels in the similar pixel group in the marking direction of the reference pixel.

[0040] The direction perpendicular to the marking direction is designated as the reference direction. Any similar pixel in the group of similar pixels in the marking direction of the reference pixel is determined as a candidate pixel. The two pixels closest to the candidate pixel are selected from the reference direction of the candidate pixel. The maximum value of the absolute difference between the gray value corresponding to the candidate pixel and the gray value corresponding to these two pixels is determined as the initial difference corresponding to the candidate pixel.

[0041] When the initial difference corresponding to the candidate pixel is less than or equal to the target similarity threshold corresponding to the target illumination area to which the candidate pixel belongs, the preset value is determined as the target difference corresponding to the candidate pixel.

[0042] When the initial difference corresponding to the candidate pixel is greater than the target similarity threshold corresponding to the target illumination area to which the candidate pixel belongs, the initial difference corresponding to the candidate pixel is determined as the target difference corresponding to the candidate pixel.

[0043] Based on the grayscale similarity index of the reference pixel in all preset directions, and the target difference corresponding to all similar pixels in the similar pixel group in all preset directions of the reference pixel, the structural features corresponding to the reference pixel are determined.

[0044] Optionally, determining the grayscale similarity index in the marking direction of the reference pixel based on the number of similar pixels in the similar pixel group in the marking direction of the reference pixel includes:

[0045] When the number of similar pixels in the group of similar pixels in the marking direction of the reference pixel is less than a preset threshold, the preset value is determined as the grayscale similarity index in the marking direction of the reference pixel.

[0046] When the number of similar pixels in the similar pixel group in the marking direction of the reference pixel is greater than or equal to a preset threshold, the grayscale similarity index in the marking direction of the reference pixel is determined based on the number of similar pixels in the similar pixel group in the marking direction of the reference pixel, wherein the number of similar pixels in the similar pixel group is positively correlated with the grayscale similarity index.

[0047] Optionally, the formula corresponding to the structural features of a pixel is:

[0048] ;in, It is the structural feature corresponding to the i-th pixel in the image to be detected; i is the index of the pixel in the image to be detected; It is the normalization function; N is the number of preset directions; k is the index of the preset direction; It is the gray-level similarity index of the i-th pixel in the k-th preset direction in the image to be detected; q is the number of similar pixels in the group of similar pixels in the k-th preset direction of the i-th pixel in the image to be detected; q is the index of the similar pixel in the group of similar pixels in the k-th preset direction of the i-th pixel. It is the target difference corresponding to the qth similar pixel in the group of similar pixels in the kth preset direction of the i-th pixel in the image to be detected.

[0049] The present invention has the following beneficial effects:

[0050] This invention discloses a 3D target detection method based on remote sensing images. By processing image data, it achieves target detection in the image, solving the technical problem of poor target detection accuracy and improving its overall accuracy. First, the acquired image is divided into categories based on the degree of illumination influence, which can mitigate interference caused by illumination effects and prevent misidentification of illumination changes as feature points. Next, the same object often exhibits different grayscale changes under different illumination conditions. Therefore, by modifying the preset similarity threshold based on the number of grayscale values, area, and grayscale fluctuation index corresponding to the illuminated area, the target similarity threshold corresponding to the illuminated area can be quantified more accurately, reducing interference caused by different illumination effects. Then, based on the target similarity threshold corresponding to the illuminated area, similar pixel groups that approximately belong to the same object can be selected from each preset direction of each pixel within the illuminated area. Continuing, since pixels on edges often have significantly different grayscale values ​​from pixels perpendicular to that edge, and pixels on textures often have different grayscale values ​​from pixels perpendicular to that texture direction, the quantized structural features of a pixel, based on similar pixel groups in all preset directions and the grayscale values ​​perpendicular to all preset directions, can characterize the contour edge texture of that pixel. Next, since feature points in isolated forests often exhibit relatively high anomalies, quantifying the anomaly score for each pixel using the isolated forest algorithm, based on the structural features of all pixels in the image to be detected, facilitates subsequent feature point extraction. Then, based on the anomaly scores of all pixels, the SIFT corner detection algorithm can accurately extract feature points from the image to be detected, avoiding the misidentification of dirt or other contaminants as feature points. Finally, based on all extracted feature points, the DETR algorithm enables target recognition in the image to be detected, improving the accuracy of target detection. Attached Figure Description

[0051] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart of a three-dimensional target detection method based on remote sensing images according to the present invention. Detailed Implementation

[0053] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the specific implementation methods, structures, features, and effects of the technical solution proposed according to the present invention are described in detail below with reference to the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0055] This invention provides a three-dimensional target detection method based on remote sensing images, the method comprising the following steps:

[0056] The image to be detected is acquired, and the image to be detected is divided into different levels of illumination influence to obtain a preset number of target illumination regions.

[0057] Based on the grayscale values ​​of all pixels within each target illumination area, determine the grayscale fluctuation index corresponding to each target illumination area.

[0058] Based on the number of gray values, area, and gray fluctuation index of each target illumination area, the preset similarity threshold is corrected to obtain the target similarity threshold for each target illumination area.

[0059] Based on the target similarity threshold corresponding to each target illumination area, similar pixel groups are selected from each preset direction of each pixel within the target illumination area;

[0060] Based on the similar pixel groups in all preset directions of each pixel within each target illumination area, and the grayscale in the vertical direction of all preset directions of each pixel, determine the structural features corresponding to each pixel.

[0061] Based on the structural features corresponding to all pixels in the image to be detected, the anomaly score corresponding to each pixel is obtained through the isolated forest algorithm.

[0062] Based on the anomaly scores of all pixels in the image to be detected, feature points are extracted from the image using the SIFT corner detection algorithm.

[0063] Based on all extracted feature points, the DETR algorithm is used to perform target recognition on the image to be detected.

[0064] The following is a detailed explanation of each of the above steps:

[0065] refer to Figure 1 The diagram illustrates the flowchart of some embodiments of a three-dimensional target detection method based on remote sensing images according to the present invention. This three-dimensional target detection method based on remote sensing images includes the following steps:

[0066] Step S1: Obtain the image to be detected and divide the image to be detected into different levels of illumination influence to obtain a preset number of target illumination regions.

[0067] In some embodiments, an image to be detected can be acquired, and the image to be detected can be divided into regions affected by illumination to obtain a preset number of target illumination regions.

[0068] The image to be detected can be any image for which object detection is to be performed. The preset number can be a pre-set number. For example, the preset number can be 3.

[0069] It should be noted that classifying the degree of illumination influence on the acquired image to be detected can, to some extent, avoid interference caused by illumination influence and prevent changes in illumination from being misjudged as feature points.

[0070] As an example, this step may include the following steps:

[0071] The first step is to acquire the image to be detected.

[0072] For example, an image of the robot at work can be captured by a monocular RGB (Red, Green, Blue) camera mounted on the robot, the image can be converted to grayscale, and the grayscale image can be used as the image to be detected. The monocular RGB camera can be a single-lens camera used to capture RGB images.

[0073] The second step is to determine the light source confidence level for each pixel in the image to be detected based on the preset window corresponding to each pixel.

[0074] The preset window can be a pre-defined window. For example, the preset window can be a 3x3 window. A pixel can be located at the center of the preset window corresponding to that pixel.

[0075] For example, any pixel in the image to be detected can be designated as a marker pixel. The light source confidence level corresponding to the marker pixel is determined based on the mean of the gray values ​​of all pixels within a preset window corresponding to the marker pixel, and the variance of the gray values ​​of all pixels within the preset window corresponding to the marker pixel. The mean of the gray values ​​of all pixels within the preset window can be positively correlated with the light source confidence level. The variance of the gray values ​​of all pixels within the preset window can be negatively correlated with the light source confidence level.

[0076] For example, the formula for determining the light source confidence level for each pixel can be:

[0077] ;in, is the light source confidence score corresponding to the i-th pixel in the image to be detected. i is the index of the pixel in the image to be detected. It is the average grayscale value of all pixels within the preset window corresponding to the i-th pixel in the image to be detected. It is an exponential function with the natural constant as its base. It is the variance of the grayscale values ​​of all pixels within the preset window corresponding to the i-th pixel in the image to be detected.

[0078] It should be noted that robots often work indoors, where environments are often dimly lit and require artificial lighting. While lighting illuminates the room, it also increases the contrast between light and dark areas. Based on the influence of lighting, three main areas can be identified: the light source area, the illuminated area, and the shadow area. Generally, the light source area has the highest relative grayscale value and its grayscale fluctuations are relatively stable. The edge area of ​​the light source area, also known as the halo area, typically has a slightly lower grayscale value than the interior of the light source area, and its grayscale fluctuations are relatively larger. The interior and edge areas of the light source area together constitute the light source area. The grayscale values ​​of the illuminated and shadow areas are often lower than those of the interior of the light source area. Therefore, when... A larger value usually indicates that the grayscale value of the i-th pixel is relatively higher, and the grayscale fluctuation within the preset window corresponding to the i-th pixel is relatively stable; it also usually indicates that the i-th pixel is more likely to be a pixel inside the light source area.

[0079] The third step is to select pixels from the image to be detected whose light source confidence is greater than a preset light source threshold as light source pixels, and to determine the pixels in the image to be detected other than light source pixels as non-light source pixels.

[0080] The preset light source threshold can be a pre-set threshold. For example, the preset light source threshold can be 0.8.

[0081] The fourth step is to determine the halo probability of each non-light source pixel based on the minimum distance between each non-light source pixel and all light source pixels, the mean gray value of all pixels in the preset window corresponding to each non-light source pixel, and the variance of the gray value of all pixels in the preset window corresponding to each non-light source pixel.

[0082] Among them, the minimum value of the distance between non-light source pixels and all light source pixels is the minimum distance between non-light source pixels and each light source pixel.

[0083] For example, the formula for determining the halo probability corresponding to a non-light source pixel is:

[0084] ;in, is the halo probability corresponding to the j-th non-light source pixel in the image to be detected. j is the index of the non-light source pixel in the image to be detected. It is a normalization function. It is the minimum distance between the j-th non-light source pixel and all light source pixels. It is the average grayscale value of all pixels within the preset window corresponding to the j-th non-light source pixel. It is the variance of the grayscale values ​​of all pixels within the preset window corresponding to the j-th non-light source pixel.

[0085] It should be noted that since non-light source pixels include halo pixels, shadow pixels, and pixels within the illuminated area, halo pixels tend to be relatively closer to the light source area compared to the other two types of pixels. In other words, halo pixels are often closer to the light source pixels, and their corresponding grayscale values ​​are usually relatively higher, with greater grayscale fluctuations. The smaller the value, the closer the j-th non-light source pixel is to the light source pixel, and the more likely the j-th non-light source pixel is to be a halo pixel. A larger value generally indicates a higher grayscale value for the j-th non-light source pixel, suggesting that the j-th non-light source pixel is more likely to be a halo pixel. A larger value generally indicates a more pronounced grayscale change within the preset window corresponding to the j-th non-light source pixel, and thus a higher likelihood that the j-th non-light source pixel is a halo pixel. Therefore, when The larger the value, the more likely the j-th non-light source pixel is to be a halo pixel.

[0086] The fifth step is to select non-light source pixels with a halo probability greater than a preset halo threshold from the above image to be detected, and to record each halo pixel and each light source pixel in the above image to be detected as a target light source point.

[0087] The preset halo threshold can be a pre-set threshold. For example, the preset halo threshold can be 0.75.

[0088] The sixth step is to determine the area where all target light source points are located in the above-mentioned image to be detected as the light source area.

[0089] The seventh step involves using the Otsu method to divide the area in the image to be detected, excluding the light source area, to obtain the shadow area and the illumination area.

[0090] For example, the Otsu method can be used to perform threshold segmentation on the region of the image to be detected, excluding the light source region. The region with the lower average gray value among the two regions can be taken as the shadow region, and the region with the higher average gray value among the two regions can be taken as the illumination region. The average gray value can be the mean of the gray values ​​of all pixels in the region.

[0091] It should be noted that since there is often a large difference in gray levels between the shadow area and the illuminated area, this is often reflected in the gray-level histogram as a distinct bimodal feature. Therefore, the shadow area and the illuminated area can be divided according to Otsu's method.

[0092] Step 8: Record the light source area, shadow area, and illumination area as target illumination areas to obtain a preset number of target illumination areas.

[0093] The preset quantity can be 3.

[0094] Step S2: Determine the grayscale fluctuation index corresponding to each target illumination area based on the grayscale values ​​of all pixels within each target illumination area.

[0095] In some embodiments, the grayscale fluctuation index corresponding to each target illumination area can be determined based on the grayscale values ​​corresponding to all pixels within each target illumination area.

[0096] As an example, the grayscale fluctuation index corresponding to the target illumination area can be determined based on the variance of the grayscale values ​​corresponding to all pixels within the target illumination area, and the number of grayscale values ​​corresponding to the target illumination area. Specifically, the variance of the grayscale values ​​corresponding to all pixels within the target illumination area, and the number of grayscale values ​​corresponding to them, are both positively correlated with the corresponding grayscale fluctuation index. The number of grayscale values ​​corresponding to the target illumination area is essentially the quantity of grayscale values ​​within that target illumination area, i.e., the number of different grayscale values ​​within that target illumination area.

[0097] For example, the formula for determining the grayscale fluctuation index corresponding to the target illumination area can be:

[0098] ;in, This is the grayscale fluctuation index corresponding to the t-th target illumination area. t is the index of the target illumination area. It is a normalization function. It is the number of grayscale values ​​corresponding to the t-th target illumination area. It is the variance of the grayscale values ​​of all pixels within the t-th target illumination region.

[0099] It should be noted that when A larger value generally indicates a greater change in grayscale within the t-th target illumination area. When A larger value generally indicates a more discrete grayscale distribution within the t-th target illumination area. Therefore, when A larger value usually indicates a more significant change in grayscale fluctuation within the t-th target illumination area.

[0100] Step S3: Based on the number of grayscale values, area, and grayscale fluctuation index corresponding to each target illumination area, the preset similarity threshold is corrected to obtain the target similarity threshold corresponding to each target illumination area.

[0101] In some embodiments, the preset similarity threshold can be corrected based on the number of grayscale values, area, and grayscale fluctuation index corresponding to each target illumination area to obtain the target similarity threshold corresponding to each target illumination area.

[0102] The area corresponding to the target illuminated region, which is the area of ​​the target illuminated region, can be represented by the number of pixels within that region. The preset similarity threshold can be a pre-set threshold. For example, the preset similarity threshold could be 10.

[0103] It should be noted that the grayscale changes of the same object often differ under different lighting conditions. Therefore, by modifying the preset similarity threshold based on the number of grayscale values, area, and grayscale fluctuation index of the target lighting area, the target similarity threshold corresponding to the target lighting area can be quantified more accurately, which can reduce the interference caused by different lighting conditions to a certain extent.

[0104] As an example, the formula for determining the target similarity threshold corresponding to the target illumination area can be:

[0105] ;in, It is the target similarity threshold corresponding to the t-th target illumination region. t is the index of the target illumination region. It is the number of grayscale values ​​corresponding to the t-th target illumination area. It is the grayscale fluctuation index corresponding to the t-th target illumination area. It is an exponential function with the natural constant as its base. It is the area corresponding to the t-th target illumination region. It is a preset similarity threshold.

[0106] It should be noted that, generally speaking, the grayscale changes of the same object often differ under different lighting conditions. For example, the lighting in shadow or illuminated areas is often relatively stable, and it usually does not cause significant changes in brightness or darkness for the same object. In this case, the similarity threshold used to determine whether they are the same object can often be appropriately reduced. However, in light-emitting areas that include halos, since halos often cause relatively significant changes in brightness or darkness for the same object, the similarity threshold used to determine whether they are the same object can often be appropriately increased. A larger value generally indicates a more pronounced change in grayscale intensity within the t-th target illumination area. A larger value usually indicates a greater change in grayscale within the t-th target illumination area. Therefore, when A larger value usually indicates a greater impact of illumination on the t-th target illumination region, suggesting a greater need to appropriately increase the similarity threshold used to determine whether objects are the same. A larger value generally indicates that the grayscale level within the t-th target illumination area is relatively more stable. When A larger value usually indicates a relatively smaller area of ​​the illuminated region of the t-th target. A larger value often indicates a smaller relatively stable region within the t-th target illumination area, suggesting a greater need to appropriately increase the similarity threshold used to determine whether objects are the same. Therefore, It can characterize the similarity threshold after adaptive adjustment of the t-th target illumination region.

[0107] Step S4: Based on the target similarity threshold corresponding to each target illumination area, filter similar pixel groups from each preset direction of each pixel within the target illumination area.

[0108] In some embodiments, similar pixel groups can be selected from each pixel in each preset direction within the target illumination area based on the target similarity threshold corresponding to each target illumination area.

[0109] The preset direction can be a pre-set direction. The preset direction of a pixel can be the extension direction of a preset ray of that pixel. The endpoint of the preset ray of a pixel can be that pixel. For example, the angle corresponding to the preset direction can be in the range of [0°, 360°].

[0110] It should be noted that, based on the target similarity threshold corresponding to the target illumination area, similar pixel groups that are approximately belonging to the same object can be selected from each preset direction of each pixel within the target illumination area.

[0111] As an example, this step may include the following steps:

[0112] The first step is to designate any pixel within the target illumination area as a reference pixel, determine any preset direction as a marking direction, and determine the absolute value of the difference between the gray value corresponding to the reference pixel and the gray value corresponding to each pixel in the marking direction as the gray value difference.

[0113] The second step is to select pixels whose grayscale difference is less than or equal to the target similarity threshold corresponding to the target illumination area to which the above reference pixels belong from the marked direction, and then use them as similar pixels.

[0114] The third step is to form a segment of similar pixels by combining the continuously distributed similar pixels.

[0115] For example, if similar pixels are adjacent, they can be grouped into the same segment of similar pixels. For instance, if the first pixel is adjacent to the second pixel, the second pixel is adjacent to the third pixel, and the third pixel is adjacent to the fourth pixel, then the first pixel, the second pixel, the third pixel, and the fourth pixel can be grouped into the same segment of similar pixels.

[0116] The fourth step is to select similar pixel segments that are adjacent to the reference pixel from all similar pixel segments and form them as similar pixel groups.

[0117] For example, if a segment of similar pixels contains similar pixels that are adjacent to a reference pixel, then that segment of similar pixels can be considered as a group of similar pixels.

[0118] Step S5: Determine the structural features corresponding to each pixel based on the similar pixel groups in all preset directions of each pixel within each target illumination area, and the grayscale in the vertical direction of all preset directions of each pixel.

[0119] In some embodiments, the structural features corresponding to each pixel can be determined based on the groups of similar pixels in all preset directions of each pixel within each target illumination area, and the grayscale values ​​in the vertical direction of all preset directions of each pixel.

[0120] The perpendicular direction of the preset direction can be the extension direction of a straight line that is perpendicular to the preset direction.

[0121] It should be noted that since the grayscale of pixels on the edge is often significantly different from that of pixels at the vertical position of the edge, and the grayscale of pixels on the texture is often different from that of pixels at the vertical position of the texture direction, the quantized structural features of a pixel can characterize the contour edge texture of the pixel based on the similar pixel groups in all preset directions and the grayscale in the vertical direction of all preset directions of the pixel.

[0122] As an example, this step may include the following steps:

[0123] The first step involves designating any pixel within the target illumination area as a reference pixel and determining any preset direction as the marking direction. Based on the number of similar pixels in the similar pixel group along the marking direction of the reference pixel, the grayscale similarity index along the marking direction of the reference pixel is determined. This process may include the following sub-steps:

[0124] The first sub-step is to determine the grayscale similarity index of the reference pixel in the marking direction when the number of similar pixels in the similar pixel group in the marking direction is less than a preset threshold.

[0125] The preset quantity threshold can be a pre-set threshold. For example, the preset quantity threshold can be 5. The preset value can also be a pre-set threshold. For example, the preset value can be 0.

[0126] The second sub-step is to determine the grayscale similarity index of the reference pixel in the marking direction when the number of similar pixels in the similar pixel group in the marking direction of the reference pixel is greater than or equal to a preset number threshold.

[0127] Among them, the number of similar pixels in a group of similar pixels can be positively correlated with the grayscale similarity index.

[0128] For example, the formula for determining the grayscale similarity index for each pixel in each preset direction can be:

[0129] ;in, It is the gray-level similarity index of the i-th pixel in the image to be detected along the k-th preset direction. i is the index of the pixel in the image to be detected. k is the index of the preset direction. It is a normalization function. This represents the number of similar pixels in the k-th preset direction group of the i-th pixel in the image to be detected. m is a preset threshold, such as 5. b is a preset value, such as 0.

[0130] It should be noted that when A larger value usually indicates that the i-th pixel has more similar pixels in the k-th preset direction, and that the grayscale of the i-th pixel in the k-th preset direction is relatively more stable.

[0131] The second step is to denote the direction perpendicular to the marked direction as the reference direction, and to determine any one similar pixel in the similar pixel group in the marked direction of the reference pixel as a candidate pixel. Then, to select the two pixels closest to the candidate pixel from the reference direction of the candidate pixel, and to determine the maximum value of the absolute difference between the gray value corresponding to the candidate pixel and the gray value corresponding to these two pixels as the initial difference corresponding to the candidate pixel.

[0132] The third step is to determine the target difference corresponding to the candidate pixel as the preset value when the initial difference is less than or equal to the target similarity threshold of the target illumination area to which the candidate pixel belongs.

[0133] The preset value can be a pre-set value, such as 0. The target difference can characterize the information carried.

[0134] It should be noted that when the initial difference corresponding to a candidate pixel is less than or equal to the target similarity threshold corresponding to the target illumination area to which the candidate pixel belongs, it can often be considered that the candidate pixel does not carry any information. Therefore, the target difference corresponding to the candidate pixel can be set to 0.

[0135] Fourth step: When the initial difference corresponding to the above candidate pixel is greater than the target similarity threshold corresponding to the target illumination area to which the above candidate pixel belongs, the initial difference corresponding to the above candidate pixel is determined as the target difference corresponding to the above candidate pixel.

[0136] It should be noted that when the initial difference corresponding to a candidate pixel is greater than the target similarity threshold corresponding to the target illumination area to which the candidate pixel belongs, it can often be considered that the candidate pixel carries information. Therefore, the initial difference corresponding to the candidate pixel can be used to represent the target difference corresponding to the candidate pixel.

[0137] Fifth, based on the grayscale similarity indices of the above-mentioned reference pixel in all preset directions, and the target differences corresponding to all similar pixels in the similar pixel groups in all preset directions of the above-mentioned reference pixel, the formula for determining the structural features corresponding to the above-mentioned reference pixel can be:

[0138] ;in, It represents the structural feature corresponding to the i-th pixel in the image to be detected. i is the index of the pixel in the image to be detected. It is the normalization function. N is the number of preset directions. k is the index of the preset direction. It is the gray-level similarity index of the i-th pixel in the k-th preset direction in the image to be detected. is the number of similar pixels in the group of similar pixels in the k-th preset direction of the i-th pixel in the image to be detected. q is the index of the similar pixel in the group of similar pixels in the k-th preset direction of the i-th pixel. It is the target difference corresponding to the qth similar pixel in the group of similar pixels in the kth preset direction of the i-th pixel in the image to be detected.

[0139] It should be noted that when A larger value generally indicates that the i-th pixel has more similar pixels in the k-th preset direction, and that the grayscale of the i-th pixel in the k-th preset direction is relatively more stable. A larger value generally indicates a more significant difference between the grayscale value of the i-th pixel in the k-th preset direction and its grayscale value in the vertical direction. Therefore, when... A larger value often indicates that the i-th pixel is more likely to be a pixel on the edge contour or texture, and its structural features are more obvious.

[0140] Step S6: Based on the structural features corresponding to all pixels in the image to be detected, the anomaly score corresponding to each pixel is obtained through the isolated forest algorithm.

[0141] In some embodiments, the anomaly score for each pixel can be obtained using the isolated forest algorithm based on the structural features corresponding to all pixels in the image to be detected.

[0142] It should be noted that since feature points in isolated forests are often characterized by a relatively high degree of anomalousness, quantifying the anomalous score of each pixel based on the structural features of all pixels in the image to be detected using the isolated forest algorithm can facilitate subsequent feature point extraction.

[0143] As an example, based on the structural features corresponding to all pixels in the image to be detected, a Gaussian convolution kernel can be used to process each target illumination region to obtain the scale space of the target illumination region. Using the structural features of the pixels as the data feature input for the Isolation Forest algorithm, 100 binary trees constructed from the above input features are randomly obtained. The anomaly score of each data point in the binary tree is calculated, and the anomaly score of each pixel in the scale space can be obtained by comprehensive calculation.

[0144] Step S7: Based on the anomaly scores corresponding to all pixels in the image to be detected, feature points are extracted from the image to be detected using the SIFT corner detection algorithm.

[0145] In some embodiments, the gray values ​​corresponding to all pixels in the image to be detected can be updated to their corresponding anomaly scores, and feature points can be extracted from the image to be detected using the SIFT corner detection algorithm.

[0146] As an example, we can obtain the anomaly score of each pixel in the scale space, set a local region in the scale space, and obtain the maximum anomaly score of the local region. This point is the extreme point of the local region. Following the subsequent operations of SIFT corner detection, we perform corner detection on the image to be detected, which is to obtain the feature points in the image to be detected.

[0147] Step S8: Based on all extracted feature points, target recognition is performed on the image to be detected using the DETR algorithm.

[0148] In some embodiments, the above-mentioned image to be detected can be identified using the DETR (DEtectionTRansformer, end-to-end target detection) algorithm based on all extracted feature points.

[0149] As an example, based on all extracted feature points, target recognition in the image to be detected using the DETR algorithm can include: obtaining feature points in the image to be detected; simultaneously extracting features for both 2D and 3D target detection tasks using a 3-layer DAB-DETR encoder and decoder structure. The feature extraction capability of this DETR model structure is acquired by training the model on a category-level 3D target detection dataset. After the DAB-DETR model extracts features for both 2D and 3D target detection tasks, these features are input into five simple multilayer perceptron networks to predict the target object category, 2D target detection box, rotation matrix, translation vector, and target size, respectively, thus obtaining the final 3D target detection result. The training results for the 2D target detection task can guide the model's learning of the 3D target detection task.

[0150] In summary, based on the anomaly scores corresponding to all pixels, the SIFT corner detection algorithm can accurately extract feature points from the image to be detected, which can, to some extent, avoid misidentifying dirt spots as feature points. Secondly, based on all extracted feature points, the DETR algorithm can achieve target recognition in the image to be detected, thus improving the accuracy of target detection.

[0151] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A three-dimensional target detection method based on remote sensing images, characterized in that, Includes the following steps: Acquire the image to be detected, and divide the image to be detected into a number of target illumination regions by the degree of illumination influence. Based on the grayscale values ​​of all pixels within each target illumination area, determine the grayscale fluctuation index corresponding to each target illumination area. Based on the number of gray values, area, and gray fluctuation index of each target illumination area, the preset similarity threshold is corrected to obtain the target similarity threshold for each target illumination area. The number of gray values ​​corresponding to the target illumination area is the number of gray values ​​in the target illumination area, that is, the number of different gray values ​​in the target illumination area. Based on the target similarity threshold corresponding to each target illumination region, similar pixel point groups are selected from each preset direction of each pixel point within the target illumination region; Based on the similar pixel groups in all preset directions of each pixel within each target illumination area, and the grayscale in the vertical direction of all preset directions of each pixel, determine the structural features corresponding to each pixel. Based on the structural features corresponding to all pixels in the image to be detected, the anomaly score corresponding to each pixel is obtained through the isolated forest algorithm. Based on the anomaly scores corresponding to all pixels in the image to be detected, feature points are extracted from the image to be detected using the SIFT corner detection algorithm; Based on all extracted feature points, the DETR algorithm is used to perform target recognition on the image to be detected; The step of determining the grayscale fluctuation index corresponding to each target illumination area based on the grayscale values ​​corresponding to all pixels within each target illumination area includes: determining the grayscale fluctuation index corresponding to the target illumination area based on the variance of the grayscale values ​​corresponding to all pixels within the target illumination area and the number of grayscale values ​​corresponding to the target illumination area, wherein the variance of the grayscale values ​​corresponding to all pixels within the target illumination area and the number of grayscale values ​​corresponding to them are positively correlated with their corresponding grayscale fluctuation index. The formula for the target similarity threshold corresponding to the target illumination area is: ;in, It is the target similarity threshold corresponding to the t-th target illumination region; t is the index of the target illumination region. It is the number of grayscale values ​​corresponding to the t-th target illumination region; It is the grayscale fluctuation index corresponding to the t-th target illumination area; It is an exponential function with the natural constant as its base; It is the area corresponding to the t-th target illumination region; It is a preset similarity threshold.

2. The three-dimensional target detection method based on remote sensing images according to claim 1, characterized in that, The step of dividing the image to be detected into a predetermined number of target illumination regions by dividing the image into regions affected by illumination includes: Based on the preset window corresponding to each pixel in the image to be detected, determine the light source confidence level corresponding to each pixel; Pixels with a light source confidence greater than a preset light source threshold are selected from the image to be detected and designated as light source pixels. Pixels in the image to be detected other than light source pixels are designated as non-light source pixels. The halo probability of each non-light source pixel is determined based on the minimum distance between each non-light source pixel and all light source pixels, the mean gray value of all pixels in the preset window corresponding to each non-light source pixel, and the variance of the gray value of all pixels in the preset window corresponding to each non-light source pixel. Non-light source pixels with a halo probability greater than a preset halo threshold are selected from the image to be detected and designated as halo pixels. Each halo pixel and each light source pixel in the image to be detected are recorded as a target light source point. The region containing all target light source points in the image to be detected is defined as the light source region; Using the Otsu method, the region in the image to be detected, excluding the light source region, is divided to obtain the shadow region and the illumination region. The light source area, shadow area, and illumination area are all denoted as target illumination areas, resulting in a preset number of target illumination areas, of which the preset number is 3.

3. The three-dimensional target detection method based on remote sensing images according to claim 2, characterized in that, The step of determining the light source confidence level corresponding to each pixel in the image to be detected based on a preset window includes: Any pixel in the image to be detected is designated as a marker pixel. The light source confidence level of the marker pixel is determined based on the mean of the gray values ​​of all pixels within a preset window corresponding to the marker pixel and the variance of the gray values ​​of all pixels within the preset window corresponding to the marker pixel. The mean of the gray values ​​of all pixels within the preset window is positively correlated with the light source confidence level, and the variance of the gray values ​​of all pixels within the preset window is negatively correlated with the light source confidence level.

4. The three-dimensional target detection method based on remote sensing images according to claim 2, characterized in that, The formula for the halo probability of non-light source pixels is: ;in, is the halo probability corresponding to the j-th non-light source pixel in the image to be detected; j is the index of the non-light source pixel in the image to be detected; It is a normalization function; It is the minimum distance between the j-th non-light source pixel and all light source pixels; It is the average grayscale value of all pixels within the preset window corresponding to the j-th non-light source pixel; It is the variance of the grayscale values ​​of all pixels within the preset window corresponding to the j-th non-light source pixel.

5. The three-dimensional target detection method based on remote sensing images according to claim 1, characterized in that, The step of filtering similar pixel groups from each pixel in each preset direction within the target illumination area based on a target similarity threshold corresponding to each target illumination area includes: Any pixel within the target illumination area is designated as a reference pixel, any preset direction is determined as a marking direction, and the absolute value of the difference between the gray value corresponding to the reference pixel and the gray value corresponding to each pixel in the marking direction is determined as the gray value difference. Pixels whose grayscale difference is less than or equal to the target similarity threshold corresponding to the target illumination area to which the reference pixel belongs are selected from the marked direction of the reference pixel and are regarded as similar pixels. Form a segment of similar pixels by dividing consecutively distributed similar pixels; Select similar pixel segments that are adjacent to the reference pixel from all similar pixel segments to form a similar pixel group.

6. The three-dimensional target detection method based on remote sensing images according to claim 1, characterized in that, The step of determining the structural features corresponding to each pixel based on the similar pixel groups in all preset directions of each pixel within each target illumination area, and the grayscale values ​​in the vertical direction of all preset directions of each pixel, includes: Any pixel within the target illumination area is designated as a reference pixel, and any preset direction is determined as a marking direction. The grayscale similarity index of the reference pixel in the marking direction is determined based on the number of similar pixels in the similar pixel group in the marking direction of the reference pixel. The direction perpendicular to the marking direction is designated as the reference direction. Any similar pixel in the group of similar pixels in the marking direction of the reference pixel is determined as a candidate pixel. The two pixels closest to the candidate pixel are selected from the reference direction of the candidate pixel. The maximum value of the absolute difference between the gray value corresponding to the candidate pixel and the gray value corresponding to these two pixels is determined as the initial difference corresponding to the candidate pixel. When the initial difference corresponding to the candidate pixel is less than or equal to the target similarity threshold corresponding to the target illumination area to which the candidate pixel belongs, the preset value is determined as the target difference corresponding to the candidate pixel. When the initial difference corresponding to the candidate pixel is greater than the target similarity threshold corresponding to the target illumination area to which the candidate pixel belongs, the initial difference corresponding to the candidate pixel is determined as the target difference corresponding to the candidate pixel. Based on the grayscale similarity index of the reference pixel in all preset directions, and the target difference corresponding to all similar pixels in the similar pixel group in all preset directions of the reference pixel, the structural features corresponding to the reference pixel are determined.

7. The three-dimensional target detection method based on remote sensing images according to claim 6, characterized in that, The step of determining the grayscale similarity index in the marking direction of the reference pixel based on the number of similar pixels in the similar pixel group in the marking direction of the reference pixel includes: When the number of similar pixels in the group of similar pixels in the marking direction of the reference pixel is less than a preset threshold, the preset value is determined as the grayscale similarity index in the marking direction of the reference pixel. When the number of similar pixels in the similar pixel group in the marking direction of the reference pixel is greater than or equal to a preset threshold, the grayscale similarity index in the marking direction of the reference pixel is determined based on the number of similar pixels in the similar pixel group in the marking direction of the reference pixel, wherein the number of similar pixels in the similar pixel group is positively correlated with the grayscale similarity index.

8. The three-dimensional target detection method based on remote sensing images according to claim 6, characterized in that, The formula corresponding to the structural features of a pixel is: ;in, It is the structural feature corresponding to the i-th pixel in the image to be detected; i is the index of the pixel in the image to be detected; It is the normalization function; N is the number of preset directions; k is the index of the preset direction; It is the gray-level similarity index of the i-th pixel in the k-th preset direction in the image to be detected; q is the number of similar pixels in the group of similar pixels in the k-th preset direction of the i-th pixel in the image to be detected; q is the index of the similar pixel in the group of similar pixels in the k-th preset direction of the i-th pixel. It is the target difference corresponding to the qth similar pixel in the group of similar pixels in the kth preset direction of the i-th pixel in the image to be detected.

Images