A vehicle detection method and system for shadow regions of remote sensing images

Abstract

The application relates to a vehicle detection method and system for a shadow area of a remote sensing image, wherein the method comprises the following steps: constructing a vehicle target detection network, training the vehicle target detection network to obtain a trained vehicle target detection network; compensating a shadow area image based on an infrared spectrum method to obtain a compensated shadow area image; inputting the compensated shadow area image into the trained vehicle target detection network to obtain rectangular detection frame information and positioning information; generating a mask detection frame based on the rectangular detection frame information, the positioning information and remote sensing image information, replacing the rectangular detection frame of the trained vehicle target detection network with the mask detection frame; and outputting a vehicle target detection result containing the mask detection frame through the trained vehicle target detection network. The method solves the problem of how to improve target detection effect and target detection accuracy in a shadow area of a remote sensing image, optimizes a detection frame, and effectively improves target detection effect and target detection accuracy.

Description

Technical Field

[0001] This application relates to the field of target detection technology, and in particular to a method and system for vehicle detection in shadowed areas of remote sensing images. Background Technology

[0002] Object detection, a part of computer vision, describes the content of an entire image and combines it with the feature information of target objects to determine their category and location. Object detection combines object localization and recognition, enabling real-time and accurate detection of targets in specific environments. Currently, the mainstream deep learning object detection algorithms fall into two categories: two-stage object detection algorithms based on region proposals and single-stage object detection algorithms based on regression analysis. Among them, single-stage object detection algorithms based on regression analysis pursue both speed and accuracy. They have a simple algorithm structure, can directly detect image output results, and do not involve candidate region classification, thus resulting in faster speed and higher computational efficiency.

[0003] Currently, when using object detection to detect densely packed vehicles in shadowed areas of remote sensing images, the applicant has found at least the following problems:

[0004] First, the presence of shadowed areas hinders the extraction of ground feature information, thereby reducing the effectiveness and accuracy of vehicle target detection. Because ground features block sunlight, many shadows are generated in remote sensing images. These shadowed areas are both a source of information in remote sensing images (reflecting building height, estimating building volume ratio, and extracting 3D information) and a source of noise (affecting ground feature characteristics and thus impacting the extraction of ground feature information).

[0005] Secondly, overlapping or intersecting target bounding boxes can reduce the accuracy of target detection. Because remote sensing images are captured from long distances and cover a wide area, vehicle targets are considered small in these images. In densely populated areas of remote sensing images, the distance between vehicles is small; therefore, traditional vehicle bounding boxes may partially overlap or intersect, affecting the effectiveness of vehicle target detection.

[0006] Currently, no effective solution has been proposed for improving target detection performance and accuracy in shadowed areas of remote sensing images. Summary of the Invention

[0007] This application provides a method and system for vehicle detection in shadowed areas of remote sensing images, which at least addresses the problem of how to improve target detection effect and target detection accuracy in shadowed areas of remote sensing images in related technologies.

[0008] In a first aspect, embodiments of this application provide a vehicle detection method for shadowed areas in remote sensing images, including:

[0009] Construct a vehicle target detection network, train the vehicle target detection network, and obtain a trained vehicle target detection network;

[0010] The image of the shadow area of ​​the remote sensing image is compensated based on the infrared spectroscopy method to obtain the compensated image of the shadow area of ​​the remote sensing image.

[0011] The compensated remote sensing image of the shadow region is input into the trained vehicle target detection network to obtain the rectangular detection box information and positioning information of the vehicle target in the remote sensing image of the shadow region.

[0012] Based on the rectangular detection box information, the positioning information, and the remote sensing image information, a mask detection box is generated, and the rectangular detection box of the trained vehicle target detection network is replaced with the mask detection box.

[0013] The trained vehicle target detection network outputs vehicle target detection results containing the mask detection boxes.

[0014] In some embodiments, training the vehicle target detection network to obtain the trained vehicle target detection network includes:

[0015] The pre-trained image dataset is input into the vehicle target detection network for first training to obtain the first trained vehicle target detection network.

[0016] The infrared spectroscopy method was used to compensate for the shadow areas of remote sensing images, and training image datasets and test image datasets were obtained.

[0017] The training image dataset is input into the first trained vehicle target detection network for a second training, resulting in the second trained vehicle target detection network.

[0018] The test image dataset is input into the second trained vehicle target detection network, and the detection performance of the second trained vehicle target detection network is tested to obtain the trained vehicle target detection network.

[0019] In some embodiments, the compensation for shadow areas in remote sensing images based on infrared spectroscopy includes:

[0020] The radiance of the remote sensing image is calculated using the first formula based on the surface irradiance of the remote sensing image.

[0021] The radiance of the remote sensing image is corrected by the second formula to obtain the radiance of the corrected remote sensing image.

[0022] The radiance of the shadowed area in the corrected remote sensing image is compensated based on the direct irradiance of the shadowed and unshadowed areas in the remote sensing image.

[0023] In some embodiments, calculating the radiance of the remote sensing image using the first formula includes:

[0024] The radiance of the shadowed and unshadowed areas of the remote sensing image is calculated using a first formula, which is:

[0025]

[0026]

[0027] In the formula, L s The radiance of the shaded area, π is the surface reflectivity of the shaded area, π is the solid angle, and E is the surface reflectivity of the shaded area. h E represents ambient irradiance. s L represents diffuse irradiance. n The radiance of the non-shaded area, E represents the surface reflectance of the non-shaded area. z This refers to direct irradiance.

[0028] In some embodiments, correcting the radiance of the remote sensing image using a second formula includes:

[0029] The radiance of the shadowed and unshadowed areas of the remote sensing image is corrected using a second formula, which is:

[0030]

[0031]

[0032] In the formula, L sx This is the corrected radiance of the shadow area. E represents the surface reflectance of the shaded area. h E represents ambient irradiance. s L represents diffuse irradiance. nx The corrected radiance for non-shaded areas. E represents the surface reflectance of the non-shaded area. z Let L be the direct irradiance, τ be the atmospheric transmittance, and L be the atmospheric transmittance. f Radiation reflected by the atmosphere.

[0033] In some embodiments, generating a mask detection box based on the rectangular detection box information, the positioning information, and the remote sensing image information includes:

[0034] The mask detection box is a square mask detection box, used to determine the scene type of the shadow area image of the remote sensing image;

[0035] If the scene type is a scene type where vehicles are parked or driving in parallel, then the side length of the square mask detection box is determined as the first side length based on the rectangular detection box information, the positioning information and the remote sensing image information, and a square mask detection box is generated based on the first side length.

[0036] If the scenario type is a scenario where vehicles are parked or driving with their heads and tails connected, then the side length of the square mask detection box is determined as the second side length based on the rectangular detection box information, the positioning information and the remote sensing image information, and a square mask detection box is generated based on the second side length.

[0037] If the scenario type is a scenario where the vehicle is parked or driven irregularly, then the side length of the square mask detection box is determined to be the third side length based on the rectangular detection box information, and a square mask detection box is generated based on the third side length.

[0038] In some embodiments, constructing the vehicle target detection network includes:

[0039] Using the last layer of the ResNet-50 network as the baseline feature layer, three upsampling operations are performed using subpixel convolution. By adjusting the number of convolution channels and fusing them with the feature layer after the first max pooling of the ResNet-50 network, three feature layers of different sizes are obtained.

[0040] Global average pooling is performed on the three feature layers of different sizes to obtain three initial weights. After adjusting them to have the same number of channels, they are stacked to obtain stacked weights. Feature learning is performed on the stacked weights using convolution to obtain fusion weights. The fusion weights are adjusted to three weights with different numbers of channels through three convolutions. Each of these weights is added to the initial weights with the same number of channels. The initial weights after addition are then fused with feature layers of corresponding sizes to obtain three enhanced feature layers.

[0041] The three enhanced feature layers are classified and regressed using convolution to obtain the vehicle target detection network.

[0042] In some embodiments, the first training process includes:

[0043] In the first training process, a variable-interval cosine annealing function with restart is added. The initial restart parameters and change rate parameters are set based on the preset number of training iterations, and the learning rate is adjusted.

[0044] In some embodiments, the second training includes:

[0045] Freeze the network portion extracted from the backbone features, and then perform frozen training on the frozen network portion;

[0046] When the number of iterations of the frozen training reaches a preset value, the frozen training process is completed.

[0047] Unfreeze the network portion of the backbone feature extraction and perform unfreeze training on the unfrozen network portion. When the number of iterations of the unfreeze training reaches a preset value, the unfreeze training process is completed.

[0048] The second trained vehicle target detection network is obtained, wherein the training processes of both the frozen training and the unfrozen training are incorporating a cosine annealing function without restart.

[0049] Secondly, embodiments of this application provide a vehicle detection system for shadowed areas in remote sensing images, comprising: a target detection network construction module, a target detection network training module, an infrared spectroscopy compensation module, a vehicle target detection result module, and a mask detection box module; wherein...

[0050] The target detection network construction module is used to build a vehicle target detection network;

[0051] The target detection network training module is used to train the vehicle target detection network to obtain the trained vehicle target detection network.

[0052] An infrared spectroscopy compensation module is used to compensate for the shadow areas of remote sensing images to obtain the compensated image of the shadow areas of the remote sensing images.

[0053] The vehicle target information module is used to input the compensated remote sensing image of the shadow area into the trained vehicle target detection network to obtain the rectangular detection box information and positioning information of the vehicle target in the remote sensing image of the shadow area.

[0054] The mask detection box module is used to generate a mask detection box based on the rectangle information, the positioning information and the remote sensing image information, and replace the rectangle detection box of the trained vehicle target detection network with the mask detection box.

[0055] The vehicle target detection result module is used to output vehicle target detection results containing masked detection boxes through the trained vehicle target detection network.

[0056] Compared to related technologies, this application provides a vehicle detection method for shadowed areas in remote sensing images. This method constructs and trains a vehicle target detection network to obtain a trained network. It then compensates for the shadowed areas of the remote sensing image using infrared spectroscopy to obtain a compensated image of the shadowed area. This compensated image is input into the trained network to obtain rectangular detection boxes and location information of vehicle targets within the shadowed area. Based on these rectangular detection boxes, location information, and remote sensing image information, a mask detection box is generated. This mask detection box replaces the rectangular detection boxes in the trained network. Finally, the trained network outputs vehicle target detection results containing the mask detection boxes. This method solves the problem of improving target detection performance and accuracy in shadowed areas of remote sensing images. It enables the detection of densely packed vehicles in these areas by compensating for the shadows, restoring the true feature information of vehicles in the shadowed areas, and optimizing the detection boxes. This addresses the problem of overlapping rectangular detection boxes caused by dense vehicle traffic, effectively improving both the target detection performance and accuracy. Attached Figure Description

[0057] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0058] Figure 1 This is a flowchart of the steps of a vehicle detection method for shadowed areas in remote sensing images according to an embodiment of this application.

[0059] Figure 2 This is an overall flowchart of a vehicle detection method for shadowed areas in remote sensing images according to an embodiment of this application.

[0060] Figure 3 This is a schematic diagram of the feature extraction module structure according to an embodiment of this application.

[0061] Figure 4 This is a schematic diagram of the feature enhancement module structure according to an embodiment of this application.

[0062] Figure 5 This is a structural framework diagram of a vehicle detection system for shadowed areas in remote sensing images according to an embodiment of this application.

[0063] Figure 6 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application.

[0064] The attached diagrams are labeled as follows: 51. Target detection network construction module; 52. Target detection network training module; 53. Infrared spectroscopy compensation module; 54. Vehicle target information module; 55. Mask detection box module; 56. Vehicle target detection result module. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0066] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0067] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0068] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0069] Example 1

[0070] This application provides a method for vehicle detection in shadowed areas of remote sensing images. Figure 1 This is a flowchart illustrating the steps of the vehicle detection method in the shadowed area of ​​remote sensing imagery according to the application embodiment. Figure 2 This is an overall flowchart of the vehicle detection method for shadowed areas in remote sensing images, as described in the application embodiment. Figure 1 and Figure 2 As shown, the method includes the following steps:

[0071] Step S102: Construct a vehicle target detection network, train the vehicle target detection network, and obtain a trained vehicle target detection network.

[0072] Specifically, such as Figure 2 As shown, the vehicle target detection network consists of a feature extraction module, a feature enhancement module, and a target detection module. The network will be trained first using a pre-training dataset. Figure 2 The network pre-training process involves incorporating a cosine annealing function to obtain a pre-trained vehicle target detection network. Infrared spectroscopy is used to compensate for shadow areas in remote sensing images. Figure 2 Shadow compensation in the image processing (and other methods) to obtain the training image dataset ( Figure 2 The training dataset and the test image dataset are adjusted. Figure 2 The training image dataset is input into the first trained vehicle target detection network for a second training iteration. A cosine annealing function is added during the second training to obtain the second trained vehicle target detection network. The test image dataset is then input into the second trained vehicle target detection network to test its detection performance, resulting in the fully trained vehicle target detection network.

[0073] Step S104: Compensate the shadow area image of the remote sensing image based on infrared spectroscopy to obtain the compensated shadow area image of the remote sensing image.

[0074] Step S106: Input the compensated remote sensing image of the shadow area into the trained vehicle target detection network to obtain the rectangular detection box information and localization information of the vehicle target in the shadow area image of the remote sensing image; specifically, as follows... Figure 2 As shown, the rectangular detection box information specifically includes the detection width and height of the rectangular box, and the positioning information specifically includes the vehicle target positioning information.

[0075] Step S108: Generate masked detection boxes based on rectangular detection box information, positioning information, and remote sensing image information; replace the rectangular detection boxes of the trained vehicle target detection network with masked detection boxes; specifically, as follows... Figure 2 As shown, the remote sensing image information specifically refers to the remote sensing image resolution information, and the mask detection box specifically refers to a square mask.

[0076] Step S110: Output the vehicle target detection result containing the mask detection box through the trained vehicle target detection network.

[0077] Through steps S102 to S110 in the embodiments of this application, the problem of how to improve the target detection effect and target detection accuracy in the shadow area of ​​remote sensing image is solved. The detection of dense vehicles in the shadow area of ​​remote sensing image is realized. The shadow area is compensated, the real feature information of vehicles in the shadow area of ​​remote sensing image is restored, and the detection box is optimized, which effectively improves the target detection effect and target detection accuracy.

[0078] In some embodiments, step S102, constructing the vehicle target detection network, includes:

[0079] The feature extraction module uses the last layer of the ResNet-50 network as the baseline feature layer, performs three upsampling operations using subpixel convolution, and fuses the feature layers after the first max pooling of the ResNet-50 network by adjusting the number of convolution channels, finally obtaining three feature layers of different sizes.

[0080] Specifically, Figure 3This is a schematic diagram of the feature extraction module structure according to an embodiment of this application, such as... Figure 3 As shown, the feature extraction module is used to obtain multi-scale feature layers. The specific implementation process is as follows: First, using CB(Conv) as the feature extraction module... The ResNet-50 network is built based on the CB (Block) and IB (IdentityBlock) blocks. The CB block has different input and output dimensions, and its function is to change the network dimension. The IB block has the same input and output dimensions, and its function is to deepen the network learning. The last layer obtained by the ResNet-50 network is used as the baseline feature layer (16×16×2048). Subpixel convolution is used to perform three upsampling operations to obtain feature layer A (32×32×512), feature layer B (64×64×128), and feature layer C1 (128×128×32) in sequence. Feature layer C1 is adjusted for the number of channels by 1×1 convolution to obtain feature layer C2 (128×128×64). Feature layer C2 is added to the corresponding elements of the feature layer (128×128×64) after the first max pooling of the ResNet-50 network to obtain feature layer C (128×128×64). Finally, three feature layers of different sizes, C, B, and A, can be obtained. The main function of subpixel convolution is to reconstruct high-resolution feature maps from low-resolution feature maps through convolution operations and the recombination of feature map channel information. Using subpixel convolution instead of ordinary upsampling in the feature extraction module can improve the utilization rate of feature layer channel information, retain more and finer vehicle target feature information during feature learning, and reduce the false negative rate of vehicle targets in remote sensing images.

[0081] The feature enhancement module uses three feature layers of different sizes obtained from the feature extraction module to perform global average pooling to obtain three initial weights. After adjusting them to have the same number of channels, they are stacked to obtain stacked weights. Convolution is used to learn features from the stacked weights to obtain fusion weights. The fusion weights are adjusted to three weights with different numbers of channels through three convolutions. Each of these weights is added to the initial weights with the same number of channels. The added weights are then fused with the feature layers of the corresponding sizes to obtain three enhanced feature layers.

[0082] Specifically, Figure 4 This is a schematic diagram of the feature enhancement module structure according to an embodiment of this application, such as... Figure 4As shown, firstly, global average pooling is performed on feature layers C, B, and A to obtain weights C1 (1×1×64), B1 (1×1×128), and A1 (1×1×512). Weights B1 and A1 are then adjusted to have 64 channels using a 1×1 convolution, resulting in weights B2 (1×1×64) and A2 (1×1×64). Next, weights C1, B2, and A2 are stacked along their height to obtain stacked weights (3×1×64). A 3×1 convolution is then used to learn features from the stacked weights to obtain fused weights (1×1×64). Finally, three 1×1 convolutions are used to adjust the channel count. The number of channels is increased to obtain weights C2 (1×1×64), B3 (1×1×128), and A3 (1×1×512). Weights C2, B3, and A3 are added to the elements of weights C1, B1, and A1 respectively, and after passing through the activation function, weights C (1×1×64), B (1×1×128), and A (1×1×512) are obtained. Finally, weights C, B, and A are multiplied by feature layers C, B, and A respectively to obtain enhanced feature layer C (128×128×64), enhanced feature layer B (64×64×128), and enhanced feature layer A (32×32×512). In the feature enhancement module, global average pooling is used to construct weight information between feature layer channels, which improves the information correlation between feature layer channels. By stacking and learning to fuse the weights of feature layers of different sizes, the information correlation between feature layers of different sizes is improved, enhancing the network's ability to detect small targets, thereby improving the accuracy of vehicle target detection in remote sensing images.

[0083] Object detection module: It uses convolutional enhanced feature layers for classification and regression, and outputs rectangular detection boxes containing the predicted category and location of vehicle targets.

[0084] Specifically, the target detection module uses an enhanced feature layer C (128×128×64) to perform three convolutions to predict the type and probability of the object, the center point of the object, and the width and height of the object, respectively. Finally, it outputs a rectangular detection box containing the predicted category and location of the vehicle target.

[0085] In some embodiments, step S102 involves training the vehicle target detection network to obtain a trained vehicle target detection network, including...

[0086] The pre-trained image dataset is input into the vehicle target detection network for the first training, resulting in the vehicle target detection network after the first training.

[0087] Specifically, images containing vehicle targets are selected as the initial dataset;

[0088] Obtain the vehicle targets and corresponding annotation information of each image in the initial dataset, and convert the dataset annotation format;

[0089] The images are then adjusted to obtain a pre-trained image dataset.

[0090] During the pre-training process, a variable-interval cosine annealing function with restart is added. The initial restart parameters and the rate of change parameters are set according to the set number of training iterations, and the learning rate is adjusted to obtain the vehicle target detection network after the first training (pre-training).

[0091] Specifically, based on the NWPU VHR-10 dataset annotated by Northwestern Polytechnical University, images containing vehicle targets were selected as the initial dataset. Vehicle targets and their corresponding annotations were obtained from each image in the initial dataset. The txt annotation format of the NWPU VHR-10 dataset was converted to the xml annotation format of the VOC dataset. Then, according to the input image size requirements of the vehicle target detection network, the images were cropped, stitched, and enhanced through random rotation, scaling, and blurring to create a pre-training image dataset. This pre-training image dataset was input into the vehicle target detection network for the first training iteration. The initial learning rate was set to 0.0001. During training, a variable-interval cosine annealing function with restart was added. The number of pre-training iterations was set to 2000, the initial restart parameter to 20, and the rate of change parameter to 2. The learning rate was adjusted to obtain the pre-trained vehicle target detection network. Adding a cosine annealing function during training improved the network convergence speed. The training configuration environment is shown in the table below.

[0092]

[0093] The infrared spectroscopy method was used to compensate for the shadow areas of remote sensing images, and training image datasets and test image datasets were obtained.

[0094] The training image dataset is input into the first trained vehicle target detection network for a second training, resulting in the second trained vehicle target detection network, which includes:

[0095] Freeze the network portion for backbone feature extraction, and then freeze the network portion for training.

[0096] The freeze training process is complete when the number of iterations reaches the preset value.

[0097] Unfreeze the network part for backbone feature extraction, and perform unfreeze training on the unfrozen network part. When the number of iterations of unfreeze training reaches a preset value, the unfreeze training process is completed.

[0098] The second trained vehicle target detection network is obtained, in which the training processes of both freeze training and unfreeze training are incorporating a cosine annealing function without restart.

[0099] Specifically, the training image dataset is input into the vehicle target detection network after the first training iteration, and the network is fine-tuned again. First, the network part for extracting the backbone features is frozen without adjusting its weights. The frozen training process is completed after 1000 iterations. Then, the network part for extracting the backbone features is unfrozen, and the unfrozen training process is completed after 2000 iterations, resulting in the fine-tuned vehicle target detection network. A cosine annealing function without restart is added during training. In the frozen training, the maximum number of iterations is set to 50; in the unfrozen training, the maximum number of iterations is set to 100. Adding the cosine annealing function during training improves the network's convergence speed.

[0100] The test image dataset is input into the second trained vehicle target detection network to test the detection performance of the second trained vehicle target detection network, thus obtaining the trained vehicle target detection network.

[0101] Step S104: Compensate the shadow area image of the remote sensing image based on infrared spectroscopy to obtain the compensated shadow area image of the remote sensing image, including:

[0102] The radiance of the remote sensing image is calculated using the first formula based on the surface irradiance of the remote sensing image.

[0103] The radiance of the remote sensing image is corrected using the second formula to obtain the corrected radiance of the remote sensing image.

[0104] The radiance of the shadowed areas in the corrected remote sensing image is compensated based on the direct irradiance of the shadowed and unshadowed areas in the remote sensing image.

[0105] Specifically, a method based on improved infrared spectroscopy is used to compensate for shadow areas, where infrared spectroscopy (IR) is used. First, the radiance of the remote sensing image is expressed as its surface irradiance, as shown in the following formula (first formula):

[0106]

[0107]

[0108] In the formula, L s The radiance of the shaded area, π is the surface reflectivity of the shaded area, π is the solid angle, and E is the surface reflectivity of the shaded area. h E represents ambient irradiance. s L represents the diffuse irradiance. n The radiance of the non-shaded area, E represents the surface reflectance of the non-shaded area. z This refers to direct irradiance.

[0109] Considering atmospheric influences from the Earth's surface to satellite sensors, the radiance formulas for shaded and unshaded areas are modified, as shown in the second formula:

[0110]

[0111]

[0112] In the formula, L sx This is the corrected radiance of the shadow area. E represents the surface reflectance of the shaded area. h E represents ambient irradiance. s L represents diffuse irradiance. nx The corrected radiance for non-shaded areas. E represents the surface reflectance of the non-shaded area. z Let L be the direct irradiance, τ be the atmospheric transmittance, and L be the atmospheric transmittance. f Radiation reflected by the atmosphere.

[0113] Assume that there is direct irradiance in the shaded area, and that the direct irradiance in the unshaded area has the following relationship:

[0114]

[0115] In the formula, Let α and β be the assumed direct irradiance of the shaded area, and α and β be adjustment coefficients. The α adjustment coefficient is related to the number of vehicles in the shaded area; the more vehicles in the shaded area, the fewer the types of objects within the shaded area, and the closer the direct irradiance is to that of the non-shaded area. The β adjustment coefficient is related to the area of ​​the shaded area; the smaller the area of ​​the shaded area, the fewer the types of objects within the shaded area, and the closer the direct irradiance is to that of the non-shaded area. To avoid... The difference in direct irradiance between the shaded and unshaded areas is too large, so the range of values ​​for α and β is limited. The formulas for calculating the adjustment coefficients for α and β are as follows:

[0116]

[0117] In the formula, v is the number of vehicles in the shaded area, m is the area of ​​the shaded area, and α and β range from 0.5 to 1.0.

[0118] The formula for the radiance of the compensated shadow area is as follows:

[0119]

[0120] In the formula, The radiance of the shaded area after compensation.

[0121] By L sx With Lnx By simultaneously solving and simplifying the formulas, we can obtain:

[0122]

[0123] In the formula, L sx L represents the corrected radiance of the shadow area. nx For the corrected radiance of the non-shaded area, L f As the radiation reflected by the atmosphere, it should be noted that by using the compensated radiance of the shadow area to compensate for the shadow area of ​​the remote sensing image, the image details of the shadow area of ​​the remote sensing image are improved, and the true feature information of the vehicles in the shadow area of ​​the remote sensing image is restored.

[0124] Parking lots and intersections with dense vehicles under shadow cover were extracted from WorldView-2 satellite imagery data. The images were cropped to 512×512 pixels to obtain RGB three-channel color satellite remote sensing images. Then, the bounding box information of vehicle targets was labeled using the open-source labeling tool LabelImg. Finally, 90% of the satellite remote sensing images were randomly selected as the training image dataset, and the remaining 10% of the satellite remote sensing images were used as the test image dataset.

[0125] Step S106: Input the compensated remote sensing image of the shadow area into the trained vehicle target detection network to obtain the rectangular detection box information and positioning information of the vehicle target in the remote sensing image of the shadow area.

[0126] Step S108: Generate masked detection boxes based on rectangular detection box information, positioning information, and remote sensing image information, replacing the rectangular detection boxes of the trained vehicle target detection network with masked detection boxes, including:

[0127] The mask detection box is a square mask detection box, used to determine the scene type of the shadow area image in the remote sensing image;

[0128] If the scene type is a scene where vehicles are parked or driving in parallel, the side length of the square mask detection box is determined as the first side length based on the rectangular detection box information, positioning information and remote sensing image information, and the square mask detection box is generated based on the first side length.

[0129] If the scenario is a vehicle parked or moving with its head and tail connected, the side length of the square mask detection box is determined as the second side length based on the rectangular detection box information, positioning information and remote sensing image information, and the square mask detection box is generated based on the second side length.

[0130] If the scenario type is an irregularly parked or moving vehicle scenario, then the side length of the square mask detection box is determined to be the third side length based on the rectangular detection box information, and the square mask detection box is generated based on the third side length.

[0131] Specifically, step S106 yields the vehicle target's location information (x, y), and the width and height (w, h) of the rectangular detection box; its resolution (r) is obtained from the remote sensing imagery of the test dataset. When converting between the square mask and the rectangular detection box, the center coordinates of the square mask are consistent with the center coordinates (x, y) of the vehicle target. b represents the smaller of the width and height of the rectangular detection box, c represents the larger of the width and height, and t represents the side length of the square mask.

[0132] Dense vehicles in parking areas and intersections are divided into two types: regular vehicles, where the direction of the vehicle is consistent with the direction of the rectangular detection box (east, west, south, and north), and irregular vehicles, where the direction of the vehicle is inconsistent with the direction of the rectangular detection box (there is an angle). Based on this, three different scenarios are identified, and the transformation relationship of the square mask is designed for each.

[0133] (1) In the scenario where vehicles are parked or moving in parallel (perpendicular regular vehicles), determine the side length of the square mask as the first side length.

[0134] For vehicles parked perpendicularly within a parking area, considering the safe distance between vehicles (the sides of the two vehicles) in the width direction, the formula for determining the side length (t1) of the square mask is as follows:

[0135] t1 = a1 × b

[0136] In the formula, a1 is the scaling factor, which is determined by the following formula:

[0137]

[0138] In the formula, r is the resolution of the remote sensing image, and s1 is the sum of the average width of the vehicle and the safe distances on the left and right sides.

[0139] (2) In the scenario where vehicles are parked or driving in a line (parallel regular vehicles), determine the side length of the square mask as the second side length.

[0140] For vehicles parked parallel within a parking area and vehicles waiting in congested areas at intersections, considering the safe distance between vehicles (end to end) along the length direction, the formula for determining the side length (t2) of the square mask is as follows:

[0141] t2 = a2 × b

[0142] In the formula, a2 is the scaling factor, which is determined by the following formula:

[0143]

[0144] In the formula, r is the resolution of the remote sensing image, and s2 is the sum of the average length of the vehicle and the safety distances on the front and rear sides.

[0145] (3) In the scenario of irregular parking or driving of vehicles (irregular vehicles), the side length of the square mask is determined to be the third side length;

[0146] For irregular vehicles within parking areas and intersection areas, the formula for determining the side length (t3) of the square mask is as follows:

[0147]

[0148] In the formula, s3 is the average width of the vehicle.

[0149] It should be noted that using a square mask instead of a rectangular detection box, and outputting a square detection box that includes the predicted category and location of the vehicle target, can solve the problem of overlapping rectangular detection boxes caused by dense vehicles, thus optimizing the detection effect.

[0150] Step S110: Output the vehicle target detection result containing the mask detection box through the trained vehicle target detection network.

[0151] Example 2

[0152] This application provides a vehicle detection system for shadowed areas in remote sensing images. Figure 5 This is a structural framework diagram of a vehicle detection system for shadowed areas in remotely sensed images according to an embodiment of this application, as shown below. Figure 5 As shown, the system includes: a target detection network construction module 51, a target detection network training module 52, an infrared spectroscopy compensation module 53, a vehicle target information module 54, a mask detection box module 55, and a vehicle target detection result module 56; among which,

[0153] Target detection network construction module 51 is used to construct a vehicle target detection network;

[0154] The target detection network training module 52 is used to train the vehicle target detection network to obtain a trained vehicle target detection network.

[0155] The infrared spectroscopy compensation module 53 is used to compensate for the shadow area of ​​the remote sensing image to obtain the compensated image of the shadow area of ​​the remote sensing image.

[0156] The vehicle target information module 54 is used to input the compensated remote sensing image shadow area image into the trained vehicle target detection network to obtain the rectangular detection box information and positioning information of the vehicle target in the remote sensing image shadow area image;

[0157] The mask detection box module 55 is used to generate a mask detection box based on the rectangle information, positioning information and remote sensing image information, and replace the rectangle detection box of the trained vehicle target detection network with the mask detection box.

[0158] The vehicle target detection result module 56 is used to output vehicle target detection results containing masked detection boxes through the trained vehicle target detection network.

[0159] The target detection network construction module 51, target detection network training module 52, infrared spectroscopy compensation module 53, vehicle target information module 54, mask detection box module 55, and vehicle target detection result module 56 in this embodiment solve the problem of improving target detection effect and accuracy in shadow areas of remote sensing images. It achieves the detection of dense vehicles in shadow areas of remote sensing images, compensates for shadow areas, restores the true feature information of vehicles in shadow areas of remote sensing images, and optimizes the detection boxes, effectively improving target detection effect and accuracy.

[0160] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0161] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0162] Example 3

[0163] This embodiment also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0164] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0165] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0166] Step S102: Construct a vehicle target detection network, train the vehicle target detection network, and obtain a trained vehicle target detection network.

[0167] Step S104: Compensate the shadow area image of the remote sensing image based on infrared spectroscopy to obtain the compensated shadow area image of the remote sensing image.

[0168] Step S106: Input the compensated remote sensing image of the shadow area into the trained vehicle target detection network to obtain the rectangular detection box information and positioning information of the vehicle target in the remote sensing image of the shadow area.

[0169] Step S108: Generate a masked detection box based on the rectangular detection box information, positioning information and remote sensing image information, and replace the rectangular detection boxes of the trained vehicle target detection network with masked detection boxes.

[0170] Step S110: Output the vehicle target detection result containing the mask detection box through the trained vehicle target detection network.

[0171] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0172] Furthermore, in conjunction with the vehicle detection method for shadowed areas of remote sensing images in the above embodiments, this application embodiment can provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the vehicle detection methods for shadowed areas of remote sensing images in the above embodiments.

[0173] Example 4

[0174] In one embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, memory, a network interface, a display screen, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it implements a method for vehicle detection in shadowed areas of remotely sensed images. The display screen may be a liquid crystal display (LCD) or an e-ink display. The input device may be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0175] In one embodiment, Figure 6 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application, such as... Figure 6As shown, an electronic device is provided, which can be a server, and its internal structure diagram can be as follows. Figure 6 As shown, the electronic device includes a processor, a network interface, internal memory, and non-volatile memory connected via an internal bus. The non-volatile memory stores an operating system, computer programs, and a database. The processor provides computing and control capabilities, the network interface communicates with external terminals via a network connection, the internal memory provides an environment for the operation of the operating system and computer programs, the computer programs are executed by the processor to implement a method for vehicle detection in shadowed areas of remotely sensed images, and the database stores data.

[0176] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0177] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0178] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0179] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A vehicle detection method for shadow regions of remote sensing images, characterized in that, The method comprises: constructing a vehicle target detection network, training the vehicle target detection network to obtain a trained vehicle target detection network; According to the surface irradiance of the remote sensing image, the radiance of the remote sensing image is calculated by a first formula; the radiance of the remote sensing image is corrected by a second formula to obtain the corrected radiance of the remote sensing image; the radiance of the shadow area of the corrected remote sensing image is compensated according to the direct irradiance of the shadow area and the non-shadow area of the remote sensing image to obtain the compensated remote sensing image shadow area image, wherein the direct irradiance of the shadow area is determined according to the product relationship of the direct irradiance of the non-shadow area, the alpha adjustment coefficient and the beta adjustment coefficient, the alpha adjustment coefficient is related to the number of vehicles in the shadow area, and the beta adjustment coefficient is related to the area of the shadow area; The compensated remote sensing image shadow area image is input into the trained vehicle target detection network to obtain the rectangular detection frame information and the positioning information of the vehicle target in the remote sensing image shadow area image; Based on the rectangular detection frame information, the positioning information and the remote sensing image information, the scene type of vehicle parking or driving is judged, and a square mask detection frame is generated according to the scene type, and the rectangular detection frame of the trained vehicle target detection network is replaced by the square mask detection frame; The trained vehicle target detection network outputs the vehicle target detection result containing the square mask detection frame.

2. The method of claim 1, wherein, Training the vehicle target detection network to obtain the trained vehicle target detection network comprises: inputting a pre-training image data set into the vehicle target detection network for first training to obtain the vehicle target detection network after the first training; Compensate the shadow area of the remote sensing image based on infrared spectroscopy to obtain a training image data set and a test image data set; The training image data set is input into the vehicle target detection network after the first training for second training to obtain the vehicle target detection network after the second training; The test image data set is input into the vehicle target detection network after the second training to test the detection effect of the vehicle target detection network after the second training to obtain the trained vehicle target detection network.

3. The method of claim 1, wherein, The radiance of the remote sensing image is calculated by a first formula, which comprises: The radiance of the shadow area and the radiance of the non-shadow area of the remote sensing image are calculated by a first formula, the first formula is: where, is the radiance of the shaded area, is the surface reflectance of the shaded area, is the solid angle, is the ambient irradiance, is the diffuse irradiance; is the radiance of the unshaded area, is the surface reflectance of the unshaded area, is the direct irradiance.

4. The method of claim 1, wherein, The radiance of the remote sensing image is corrected by a second formula, which comprises: The radiance of the shadow area and the radiance of the non-shadow area of the remote sensing image are corrected by a second formula, the second formula is: E = E + E E = E + E E = E + E E = E + E E = E + E E = E + E E = E + E E = E + E E = E + E E = E + E E = E + E 5. The method of claim 1, wherein, Based on the rectangular detection frame information, the positioning information and the remote sensing image information, the scene type of vehicle parking or driving is judged, and a square mask detection frame is generated according to the scene type, and the rectangular detection frame of the trained vehicle target detection network is replaced by the square mask detection frame; If the scene type is a scene type of vehicles parked or driven in parallel, a side length of the square mask detection frame is determined as a first side length based on the rectangular detection frame information, the positioning information, and remote sensing image information, and a square mask detection frame is generated based on the first side length; If the scene type is a scene type of vehicles parked or driven in a head-to-tail manner, a side length of the square mask detection frame is determined as a second side length based on the rectangular detection frame information, the positioning information, and remote sensing image information, and a square mask detection frame is generated based on the second side length; If the scene type is a scene type of vehicles parked or driven in an irregular manner, a side length of the square mask detection frame is determined as a third side length based on the rectangular detection frame information, and a square mask detection frame is generated based on the third side length.

6. The method of claim 1, wherein, The vehicle target detection network comprises: Taking the last layer of a ResNet-50 network as a benchmark feature layer, performing three times of up-sampling by using sub-pixel convolution, and fusing the up-sampled features with the feature layer after the first maximum pooling of the ResNet-50 network by adjusting the number of convolution channels, three feature layers with different sizes are obtained; Performing global average pooling on the three feature layers with different sizes to obtain three initial weights, stacking the initial weights after adjusting the number of channels to be the same, performing feature learning on the stacked weights by convolution to obtain fusion weights, adjusting the fusion weights to weights with three different numbers of channels by three convolutions, adding the initial weights with the same number of channels respectively, and fusing the added initial weights with the feature layers with corresponding sizes respectively to obtain three reinforced feature layers; Performing classification and regression on the three reinforced feature layers by convolution to obtain a vehicle target detection network.

7. A vehicle detection system for shadowed areas of remote sensing imagery, characterized in that, The system comprises a target detection network construction module, a target detection network training module, an infrared spectrum compensation module, a vehicle target information module, a mask detection frame module, and a vehicle target detection result module; wherein, The target detection network construction module is configured to construct a vehicle target detection network. The target detection network training module is configured to train the vehicle target detection network to obtain the trained vehicle target detection network. The infrared spectrum compensation module is configured to calculate the radiance of the remote sensing image by a first formula according to the surface irradiance of the remote sensing image, correct the radiance of the remote sensing image by a second formula to obtain the corrected radiance of the remote sensing image, and compensate the radiance of the shadow area of the corrected remote sensing image according to the direct irradiance of the shadow area and the non-shadow area of the remote sensing image to obtain the compensated shadow area image of the remote sensing image, wherein the direct irradiance of the shadow area is determined according to the product relationship of the direct irradiance of the non-shadow area, an alpha adjustment coefficient, and a beta adjustment coefficient, the alpha adjustment coefficient is related to the number of vehicles in the shadow area, and the beta adjustment coefficient is related to the area of the shadow area. A vehicle target information module is configured to input the compensated shadow area image of the remote sensing image into the trained vehicle target detection network to obtain rectangular detection frame information and positioning information of a vehicle target in the shadow area image of the remote sensing image. A mask detection frame module is configured to generate a square mask detection frame according to the rectangular frame information, the positioning information and remote sensing image information, and replace the rectangular detection frame of the trained vehicle target detection network with the square mask detection frame. A vehicle target detection result module is configured to output a vehicle target detection result containing the mask detection frame through the trained vehicle target detection network.

Images