Remote sensing image road target detection method and device, medium

Abstract

A method, apparatus, and medium for detecting road targets in remote sensing images are disclosed. The method includes: acquiring a remote sensing image of a target to be detected; using a YOLOv5 model to detect the target remote sensing image and determine the target objects in the target remote sensing image, including: acquiring target-scale path features, wherein the target-scale path features represent micro-path features and small path features; and inputting the target path into the head network of the YOLOv5 model to determine the target objects in the target remote sensing image. Using the above scheme, small target objects can be accurately located and detected.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, and medium for detecting road targets in remote sensing images. Background Technology

[0002] With rapid socio-economic development and accelerated construction of transportation infrastructure, urban road networks are becoming increasingly complex. In urban traffic, mixed vehicle traffic and congestion can easily lead to traffic accidents. Intersections, side streets, and pedestrian crossings exhibit highly complex traffic composition and characteristics, with conflicts and interference between vehicles and pedestrians, and between vehicles themselves, making them frequent accident hotspots.

[0003] In remote sensing images, intersections and pedestrian crossings are typically classified as small targets. Small targets have limited visual information, making it difficult to extract discriminative features, and they are easily affected by external environmental factors, resulting in detection models struggling to accurately locate and detect them. Summary of the Invention

[0004] This invention addresses the technical problem of the inability to accurately locate and detect small target objects.

[0005] To address the aforementioned technical problems, this invention provides a method for detecting road targets in remote sensing images, comprising: acquiring a remote sensing image of a target to be detected; using a YOLOv5 model to detect the target remote sensing image and determine the target object in the target remote sensing image, including: acquiring target-scale path features, wherein the target-scale path features represent micro-path features and small-path features; and inputting the target path into the head network of the YOLOv5 model to determine the target object in the target remote sensing image.

[0006] Optionally, obtaining the target-scale path features includes: obtaining micro-path features f1 and small-path features f′2; extracting the first channel domain internal information F1 of the micro-path feature f1 and the second channel domain internal information F′2 of the small-path feature f′2; wherein, F1 = CA(f1), F′2 = CA(f′2), CA(f1) is the channel attention CA operation performed on f1, and CA(f′2) is the CA operation performed on f′2; and downsampling the first channel domain internal information F1 to obtain the downsampled first channel domain internal information f′2. d (CA(f1)); The information within the second channel domain is fused with the information within the first channel domain after the downsampling operation to obtain the target-scale path features; the target-scale path features are: in, For CA(f′2) and f d(CA(f1)) performs the splicing operation.

[0007] Optionally, the loss function of the YOLOv5 model is: L = αL nwd +βL IoU Where α is the first weighting coefficient L nwd =1-NWD(N p N g )L IoU = 1 - IoU, where IoU is the intersection-union ratio between the predicted bounding box and the ground truth bounding box. β is the second weighting coefficient; cx p The x-coordinate representing the center of the predicted bounding box, cy p The ordinate of the center of the predicted bounding box, w p h represents the width of the predicted bounding box p Characterizes the height of the predicted bounding box; cx g The x-coordinate of the center of the true bounding box, cy g The ordinate w representing the center of the true bounding box g h represents the width of the true bounding box g Represents the height of the true bounding box; For calculation transpose, For calculation transpose, The Wasserstein distance between the predicted bounding box and the ground truth bounding box is calculated.

[0008] The present invention also provides a remote sensing image road target detection device, comprising: an acquisition unit for acquiring a remote sensing image of a target to be detected; and a determination unit for detecting the target remote sensing image using a YOLOv5 model and determining the target object in the target remote sensing image, including: acquiring target-scale path features, wherein the target-scale path features represent micro-path features and small-path features; and inputting the target path into the head network of the YOLOv5 model to determine the target object in the target remote sensing image.

[0009] Optionally, the determining unit is configured to acquire the target-scale path features by the following steps: acquiring micro-path features f1 and small-path features f′2; extracting the first channel domain internal information F1 of the micro-path feature f1 and the second channel domain internal information F′2 of the small-path feature f′2; wherein, F1 = CA(f1), F′2 = CA(f′2), CA(f1) is the channel attention CA operation performed on f1, and CA(f′2) is the CA operation performed on f′2; and downsampling the first channel domain internal information F1 to obtain the downsampled first channel domain internal information f′2. d(CA(f1)); The information within the second channel domain is fused with the information within the first channel domain after the downsampling operation to obtain the target-scale path features; the target-scale path features are: in, For CA(f′2) and f d (CA(f1)) performs the splicing operation.

[0010] Optionally, the loss function of the YOLOv5 model is: L = αL nwd +βL IoU Where α is the first weighting coefficient and β is the second weighting coefficient; L nwd =1-NWD(N p N g )L IoU = 1 - IoU, where IoU is the intersection-union ratio between the predicted bounding box and the ground truth bounding box. C represents the average absolute size of all objects in the target remote sensing image; cx p The x-coordinate representing the center of the predicted bounding box, cy p The ordinate of the center of the predicted bounding box, w p h represents the width of the predicted bounding box p Characterizes the height of the predicted bounding box; cx g The x-coordinate of the center of the true bounding box, cy g The ordinate w representing the center of the true bounding box g h represents the width of the true bounding box g Represents the height of the true bounding box; For calculation transpose, For calculation transpose, The Wasserstein distance between the predicted bounding box and the ground truth bounding box is calculated.

[0011] The present invention also provides a computer-readable storage medium, which is a non-volatile storage medium or a non-transient storage medium, on which a computer program is stored, wherein the computer program, when run by a processor, executes the steps of any of the above-described remote sensing image road target detection methods.

[0012] The present invention also provides another remote sensing image road target detection device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the steps of any of the above-described remote sensing image road target detection methods when running the computer program.

[0013] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0014] The YOLOv5 model is used to detect the target objects in the remote sensing image. Specifically, the target-scale path features input to the YOLOv5 model represent minute path features and small path features, thereby improving the detection accuracy of the YOLOv5 model for minute path features and small path features. Attached Figure Description

[0015] Figure 1 This is a flowchart of a remote sensing image road target detection method according to an embodiment of the present invention;

[0016] Figure 2 This is a schematic diagram of the structure of a remote sensing image road target detection device according to an embodiment of the present invention. Detailed Implementation

[0017] As described in the background section, intersections and pedestrian crossings are typically classified as small targets in remote sensing images. Small targets have limited visual information, making it difficult to extract discriminative features, and they are easily affected by external environmental factors, resulting in detection models struggling to accurately locate and detect them.

[0018] In this embodiment of the invention, the YOLOv5 model is used to detect the target remote sensing image and determine the target object in the target remote sensing image. Specifically, the target-scale path features input to the YOLOv5 model represent micro-path features and small path features, thereby improving the detection accuracy of the YOLOv5 model for micro-path features and small path features.

[0019] To make the above-mentioned objectives, features and beneficial effects of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0020] This invention provides a method for detecting road targets in remote sensing images, referring to... Figure 1 The following will provide a detailed explanation through specific steps.

[0021] Step 101: Obtain the remote sensing image of the target to be detected.

[0022] Step 102: Use the YOLOv5 model to detect the target remote sensing image and determine the target object in the target remote sensing image.

[0023] In this embodiment of the invention, the target object may include small-scale objects such as intersections and pedestrian crossings.

[0024] In practice, after acquiring the target remote sensing image, the target remote sensing image can be input into the YOLOv5 model.

[0025] In practical applications, the YOLOv5 model mainly consists of a backbone network, a neck network, and a head network. When an image is input into the YOLOv5 model, the backbone network is designed to extract low-level texture features and high-level semantic features from the input image. These low-level texture features and high-level semantic features are then passed to the head network to construct an enhanced feature pyramid network. This network transmits robust semantic features from top to bottom and strong responses to local texture and pattern features from bottom to top.

[0026] The YOLOv5 model uses CSPnet as its backbone network to extract feature information. CSPnet includes a CBS module and a CSP module. The CBS module consists of operations such as convolution, batch normalization, and the SiLu activation function. The CSP module replicates the feature map of the previous layer into two branches, and then halves the number of channels through 1×1 convolution, thereby reducing computation. Low-level texture features and high-level semantic features are extracted through a stacked structure of CSP, CBS, and SPP modules.

[0027] In practical applications, it is known that in the existing YOLOv5 model, by stacking the CBS and C3 modules, five sets of feature maps at different scales can be extracted, namely feature map P1, feature map P2, feature map P3, feature map P4, and feature map P5. Here, Pi represents a resolution of 1 / 2 the target remote sensing image. i .

[0028] Specifically, P1 represents a resolution of 1 / 2 of the target remote sensing image, and correspondingly, P5 represents a resolution of 1 / 32 of the target remote sensing image.

[0029] In some embodiments, the feature map scale of feature map P5 is 20×20, the feature map scale of feature map P4 is 40×40, the feature map scale of feature map P3 is 80×80, and the feature map scale of feature map P2 is 160×160.

[0030] In the feature map Pi, the smaller the value of i, the fewer downsampling operations are performed, and therefore the more low-level feature information is contained within it.

[0031] In remote sensing images of targets, intersections, pedestrian crossings, and other similar objects are considered small targets.

[0032] In this embodiment of the invention, the minute path feature f1 corresponding to feature map P1 and the small path feature f′2 corresponding to feature map P2 can be obtained first. The internal information F1 of the first channel domain corresponding to minute path feature f1 and the internal information F′2 of the second channel domain corresponding to small path feature f′2 are calculated respectively.

[0033] Specifically, F1 = CA(f1), F′2 = CA(f′2), CA(f1) is the channel attention CA operation performed on f1, and CA(f′2) is the CA operation performed on f′2.

[0034] In practice, Channel Attention (CA) is an attention mechanism in deep learning used to enhance the model's ability to recognize the correlation between features across different channels.

[0035] Then, a downsampling operation is performed on the information F1 within the first channel domain to obtain the downsampling information f within the first channel domain. d (CA(f1)). The information within the second channel domain is fused with the information within the first channel domain after downsampling to obtain the target-scale path features: in, For CA(f′2) and f d (CA(f1)) performs the splicing operation.

[0036] Thus, the obtained target-scale path features fuse the feature information of the micro-path and the small path. The obtained target-scale path features are then input into the Head network.

[0037] In its implementation, the YOLOv5 model's Head network comprises three distinct output layers for detecting large-scale, medium-scale, and small-scale target objects, respectively. Specifically, the output layer for detecting small-scale target objects is the first output layer (YH1), the output layer for detecting medium-scale target objects is the second output layer (YH2), and the output layer for detecting large-scale target objects is the third output layer (YH3).

[0038] Specifically, the target scale path information is input into the first output layer (YH1) to obtain the detection result output by the first output layer. The detection result contains the target object, which includes small path objects and micro path objects.

[0039] In practice, the YOLOv5 model uses a loss function consisting of three parts: bounding box loss, classification loss, and confidence loss. The classification and confidence losses use binary cross-entropy loss, while the regression box loss uses IoU loss.

[0040] In practice, IoU represents the intersection-union ratio between the predicted and ground truth boxes, which is calculated as the intersection of the predicted and ground truth boxes divided by the union of the predicted and ground truth boxes. If the predicted and ground truth boxes completely overlap, the IoU value is 1.

[0041] In existing technologies, IoU loss has the following problems: when the ground truth box and the predicted box do not overlap at all, IoU cannot reflect the distance between the ground truth box and the predicted box, and the IoU loss is 0 at this time, which will affect gradient backpropagation and thus cause the model to fail to train; when the ground truth box contains the predicted box, and the size of the ground truth box and the predicted box is fixed, IoU is the ratio of the area of ​​the predicted box to the area of ​​the ground truth box, and the IoU will not change no matter where the predicted box is located in the ground truth box.

[0042] Based on the above problems, in this embodiment of the invention, the loss function of the YOLOv5 model is updated, and a new loss function is proposed to solve the IoU problem existing in the above-mentioned loss functions.

[0043] In this embodiment of the invention, the Wasserstein distance metric can be used to reflect the true bounding box G(cx). g ,cy g ,w g ,h g ) and prediction box P(cx p ,cy p ,w p ,h p The positional relationship between ) . Specifically, cx p The x-coordinate representing the center of the predicted bounding box, cy p The ordinate of the center of the predicted bounding box, w p h represents the width of the predicted bounding box p Characterizes the height of the predicted bounding box; cx g The x-coordinate of the center of the true bounding box, cy g The ordinate w representing the center of the true bounding box g h represents the width of the true bounding box g It represents the height of the actual bounding box.

[0044] In practice, the Wasserstein distance between the ground truth bounding box and the predicted bounding box can be calculated using the following formula:

[0045]

[0046] Then, an exponential nonlinear transformation function is used to map the calculated Wassstein distance in order to normalize the Wassstein distance.

[0047] Specifically, the Wasserstein distance can be normalized using the following formula:

[0048]

[0049] Where C is the average absolute size of all objects in the target remote sensing image, and N p For the predicted bounding box, N gThis refers to the true bounding box. The objects mentioned above include the small target objects and larger-scale objects.

[0050] The normalized Wasserstein distance has a range of values ​​(0, 1).

[0051] In practice, the normalized Wasserstein distance is used to measure the positional relationship between the ground truth box and the predicted box, which can consistently reflect the distance between the ground truth box and the predicted box, even if there is no overlap between them.

[0052] In this embodiment of the invention, the loss function of the YOLOv5 model is ultimately expressed as:

[0053] L=αL nwd +βL IoU

[0054] Where α is the first weighting coefficient and β is the second weighting coefficient; L nwd =1-NWD(N p N g )L IoU =1-IoU.

[0055] Therefore, it can be seen that the loss function of the YOLOv5 model provided in this embodiment of the invention is related to both the IoU loss and the Wasserstein distance loss, thus solving the problem of applying only the IoU loss. In other words, the loss function of the YOLOv5 model is obtained by combining the IoU loss and the Wasserstein distance loss.

[0056] In summary, in this embodiment of the invention, the target-scale path features input to the YOLOv5 model represent micro-path features and small path features, thereby improving the detection accuracy of the YOLOv5 model for micro-path features and small path features.

[0057] The present invention also provides a remote sensing image road target detection device 20, comprising: an acquisition unit 201 and a determination unit 202, wherein:

[0058] Acquisition unit 201 is used to acquire remote sensing images of the target to be detected;

[0059] The determination unit 202 is used to detect the target remote sensing image using the YOLOv5 model and determine the target object in the target remote sensing image, including: acquiring target scale path features, wherein the target scale path features represent micro path features and small path features; inputting the target path into the head network of the YOLOv5 model to determine the target object in the target remote sensing image.

[0060] In a specific implementation, the determining unit 201 can be used to obtain the target-scale path features by the following steps: obtaining micro-path features f1 and small path features f′2; extracting the first channel domain internal information F1 of the micro-path feature f1 and the second channel domain internal information F′2 of the small path feature; wherein, F1 = CA(f1), F′2 = CA(f′2), CA(f1) is the channel attention CA operation on f1, and CA(f′2) is the CA operation on f′2; and downsampling the first channel domain internal information F1 to obtain the downsampled first channel domain internal information f′2. d (CA(f1)); The information within the second channel domain is fused with the information within the first channel domain after the downsampling operation to obtain the target-scale path features; the target-scale path features are: in, For CA(f′2) and f d (CA(f1)) performs the splicing operation.

[0061] In practical implementation, the loss function of the YOLOv5 model is: L = αL nwd +βL IoU Where α is the first weighting coefficient L nwd =1-NWD(N p N g )L IoU = 1 - IoU, where IoU is the intersection-union ratio between the predicted bounding box and the ground truth bounding box. β is the second weighting coefficient; cx p The x-coordinate representing the center of the predicted bounding box, cy p The ordinate of the center of the predicted bounding box, w p h represents the width of the predicted bounding box p Characterizes the height of the predicted bounding box; cx g The x-coordinate of the center of the true bounding box, cy g The ordinate w representing the center of the true bounding box g h represents the width of the true bounding box g Represents the height of the true bounding box; For calculation transpose, For calculation transpose, The Wasserstein distance between the predicted bounding box and the ground truth bounding box is calculated.

[0062] This invention also provides a computer-readable storage medium, which is a non-volatile or non-transient storage medium, storing a computer program thereon. When the computer program is run by a processor, it executes the steps of the remote sensing image road target detection method described in any of the above embodiments.

[0063] This invention also provides another remote sensing image road target detection device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it executes the steps of the remote sensing image road target detection method described in any of the above embodiments.

[0064] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include ROM, RAM, disk, or optical disk, etc.

[0065] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. A method for detecting road targets in remote sensing images, characterized in that, include: Acquire remote sensing images of the target to be detected; The YOLOv5 model is used to detect the target remote sensing image and determine the target object in the target remote sensing image, including: obtaining target scale path features, which represent micro path features and small path features; inputting the target path into the head network of the YOLOv5 model to determine the target object in the target remote sensing image; The acquisition of target-scale path features includes: acquiring micro-path features f1 and small-path features f2′; extracting the first channel domain internal information F1 of the micro-path feature f1 and the second channel domain internal information F′2 of the small-path feature f2′; wherein, F1 = CA(f1), F′2 = CA(f′2), CA(f1) is the channel attention CA operation performed on f1, and CA(f′2) is the CA operation performed on f′2; and downsampling the first channel domain internal information F1 to obtain the downsampled first channel domain internal information f2′. d (CA(f1)); The information within the second channel domain is fused with the information within the first channel domain after the downsampling operation to obtain the target-scale path features; the target-scale path features are: in, For CA(f′2) and f d (CA(f1)) is used for concatenation; the loss function of the YOLOv5 model is: L=αL nwd +βL IoU ; Where α is the first weighting coefficient, β is the second weighting coefficient, and the sum of α and β is 1; L nwd =1-NWD(N p N g )L loU = 1 - IoU, where IoU is the intersection-union ratio between the predicted bounding box and the ground truth bounding box. C represents the average absolute size of all objects in the target remote sensing image; cx p The x-coordinate representing the center of the predicted bounding box, cy p The ordinate of the center of the predicted bounding box, w p h represents the width of the predicted bounding box p Characterizes the height of the predicted bounding box; cx g The x-coordinate of the center of the true bounding box, cy g The ordinate w representing the center of the true bounding box g h represents the width of the true bounding box g Represents the height of the true bounding box; For calculation transpose, For calculation transpose, The Wasserstein distance between the predicted bounding box and the ground truth bounding box is calculated.

2. A remote sensing image road target detection device, characterized in that, include: The acquisition unit is used to acquire remote sensing images of the target to be detected. The determination unit is used to detect the target remote sensing image using the YOLOv5 model and determine the target object in the target remote sensing image, including: acquiring target scale path features, wherein the target scale path features represent micro path features and small path features; inputting the target path into the head network of the YOLOv5 model to determine the target object in the target remote sensing image; The acquisition of target-scale path features includes: acquiring micro-path features f1 and small-path features f2′; extracting the first channel domain internal information F1 of the micro-path feature f1 and the second channel domain internal information F′2 of the small-path feature f2′; wherein, F1 = CA(f1), F′2 = CA(f′2), CA(f1) is the channel attention CA operation performed on f1, and CA(f′2) is the CA operation performed on f′2; and downsampling the first channel domain internal information F1 to obtain the downsampled first channel domain internal information f2′. d (CA(f1)); The information within the second channel domain is fused with the information within the first channel domain after the downsampling operation to obtain the target-scale path features; the target-scale path features are: in, For CA(f′2) and f d (CA(f1)) is used for concatenation; the loss function of the YOLOv5 model is: L=αL nwd +βL IoU ; Where α is the first weighting coefficient and β is the second weighting coefficient; L nwd =1-NWD(N p N g )L loU = 1 - IoU, where IoU is the intersection and union of the predicted bounding box and the ground truth bounding box. cx p The x-coordinate representing the center of the predicted bounding box, cy p The ordinate of the center of the predicted bounding box, w p h represents the width of the predicted bounding box p Characterizes the height of the predicted bounding box; cx g The x-coordinate of the center of the true bounding box, cy g The ordinate w representing the center of the true bounding box g h represents the width of the true bounding box g Represents the height of the true bounding box; For calculation transpose, For calculation transpose, The Wasserstein distance between the predicted bounding box and the ground truth bounding box is calculated.

3. A computer-readable storage medium, wherein the computer-readable storage medium is a non-volatile storage medium or a non-transient storage medium, and a computer program is stored thereon, characterized in that, The computer program, when run by the processor, executes the steps of the remote sensing image road target detection method according to claim 1.

4. A remote sensing image road target detection device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor runs the computer program, it performs the steps of the remote sensing image road target detection method according to claim 1.

Images