Track small target detection method and device, electronic equipment and storage medium

Abstract

The application provides a track small target detection method and device, electronic equipment and storage medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring a to-be-detected image; inputting the to-be-detected image into a pre-trained small target detection model to obtain a small target detection result output by the small target detection model; the small target detection model is obtained by pre-training an initial model based on an improved yolo11 algorithm based on a sample detection image and a sample small target detection result corresponding to the sample detection image; the small target detection model is used for feature extraction of the to-be-detected image, generation of image features, feature alignment and feature fusion of the image features, generation of fused features, and small target detection based on the fused features. In the foregoing manner, small targets such as pedestrians or falling rocks in the to-be-detected image can be accurately detected, and the accuracy of the track small target detection result is improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, electronic device, and storage medium for detecting small targets on orbits. Background Technology

[0002] With the development of deep learning, the problem of visual object detection has been well solved. Deep learning algorithms automatically learn multi-level features of images through convolutional neural networks (CNNs), breaking away from the limitations of traditional methods that rely on manually designed features, significantly improving the feature representation ability in complex scenes, and thus facilitating the detection and recognition of image objects.

[0003] Generally, object detection in images requires simultaneous object localization (i.e., coordinate regression) and object classification. Mainstream methods can be divided into two categories: two-stage detection methods, which first generate candidate regions and then perform object classification and coordinate regression; and single-stage detection methods, which directly predict the object's location and classification using an end-to-end neural network. Two-stage detection methods are slower due to the candidate region generation step, while single-stage detection methods achieve a more balanced performance in terms of accuracy and speed.

[0004] However, when the target in the image is small, existing target detection methods struggle to accurately detect and identify small targets. Small object detection, due to its small size, lack of feature information, and susceptibility to background interference, faces the following key challenges in current technologies: First, the model's ability to extract features from small targets is insufficient. Traditional target detection networks extract semantic features of small targets through deep convolutional downsampling, but multiple downsampling operations lead to severe loss of detailed features. While shallow features of small targets can retain high-resolution information, their semantic abstraction ability is weak, and deep features suffer from insufficient feature map resolution, ultimately making it difficult for the model to distinguish small targets from noise. Second, the model's multi-scale feature fusion for small targets is insufficient. Shallow and deep features of small targets are difficult to combine effectively, further hindering the model's ability to accurately detect small targets in the image.

[0005] Because existing small target detection methods have the aforementioned shortcomings, when this method is applied to track target detection, although the model can accurately identify larger targets such as tracks or trains, it is difficult to accurately detect smaller targets such as pedestrians or falling rocks, resulting in inaccurate track small target detection results. Summary of the Invention

[0006] This invention provides a method, apparatus, electronic device, and storage medium for detecting small targets on orbits, in order to solve the shortcomings of existing technologies that make it difficult to accurately detect small targets such as pedestrians or falling rocks, resulting in inaccurate detection results for small targets on orbits.

[0007] This invention provides a method for detecting small objects on a track, comprising: acquiring an image to be detected; the image to be detected is obtained by photographing a track region containing small objects, the small objects including pedestrians and / or falling rocks; inputting the image to be detected into a pre-trained small object detection model to obtain the small object detection result output by the small object detection model; wherein, the small object detection model is obtained by pre-training an initial model based on an improved YOLO11 algorithm based on sample detection images and the corresponding sample small object detection results; the small object detection model is used to extract features from the image to be detected, generate image features, perform feature alignment and feature fusion on the image features to generate fused features, and perform small object detection based on the fused features.

[0008] According to the present invention, a method for detecting small targets on an orbit is provided. The small target detection model includes a backbone network, a neck network, and a head network connected in sequence. The backbone network is used to extract features from the image to be detected and generate image features. The neck network is used to perform feature alignment and feature fusion on the image features to generate fused features. The head network is used to perform small target detection based on the fused features and generate small target detection results.

[0009] According to the present invention, a method for detecting small targets on an orbit includes a backbone network comprising a first network layer, a second network layer, a third network layer, a fourth network layer, and a fifth network layer connected in sequence, and image features including a first image feature, a second image feature, a third image feature, and a fourth image feature. Specifically, the first network layer is used to extract features from the image to be detected, generating initial image features; the second network layer is used to extract features from the initial image features, generating the first image feature; the third network layer is used to extract features from the first image features, generating the second image feature; the fourth network layer is used to extract features from the second image features, generating the third image feature; and the fifth network layer is used to extract features from the third image features, generating the fourth image feature.

[0010] According to the present invention, a method for detecting small targets on an orbit includes a neck network comprising an MLP module, a feature alignment network, and a feature fusion network connected in sequence. The feature alignment network and the feature fusion network are deformable convolutional networks. The fused features include a first fused feature, a second fused feature, a third fused feature, and a fourth fused feature. The MLP module performs channel filtering on the first, second, third, and fourth image features to generate a first feature map, a second feature map, a third feature map, and a fourth feature map. The feature alignment network aligns the first, second, third, and fourth feature maps to obtain a first feature map to be fused, a second feature map to be fused, a third feature map to be fused, and a fourth feature map to be fused. The feature fusion network performs feature fusion based on the first, second, third, and fourth feature maps to generate the first fused feature, the second fused feature, the third fused feature, and the fourth fused feature, respectively.

[0011] According to the present invention, a method for detecting small targets on an orbit includes a head network comprising a first prediction head, a second prediction head, a third prediction head, and a fourth prediction head. The second, third, and fourth prediction heads are used to perform target detection based on a second, third, and fourth fusion feature, generating an orbit target detection result. The first prediction head is used to perform small target detection based on a first fusion feature, generating a small target detection result. The small target detection result includes small target location information and small target category information.

[0012] According to the present invention, a method for detecting small targets on an orbit is provided. The small target detection model is pre-trained based on a preset loss function, which is determined based on localization loss, classification loss, confidence loss and self-supervised contrastive loss.

[0013] According to the present invention, a method for detecting small objects on a track includes the following steps before inputting the image to be detected into a pre-trained small object detection model and obtaining the small object detection result output by the small object detection model: constructing a dataset; the dataset includes multiple sample detection images collected in test line scenarios and actual running line scenarios, and the sample small object detection result corresponding to each sample detection image; dividing the dataset into a training set, a test set, and a validation set; pre-training an initial model based on an improved YOLO11 algorithm based on the training set and the validation set to obtain a small object detection model; and testing the performance indicators of the small object detection model based on the test set.

[0014] This invention also provides a track small target detection device, comprising: an acquisition module for acquiring an image to be detected; the image to be detected is obtained by photographing a track region containing small targets, the small targets including pedestrians and / or falling rocks; a small target detection module for inputting the image to be detected into a pre-trained small target detection model to obtain the small target detection result output by the small target detection model; wherein, the small target detection model is obtained by pre-training an initial model based on an improved YOLO11 algorithm based on sample detection images and the corresponding sample small target detection results; the small target detection model is used to extract features from the image to be detected, generate image features, perform feature alignment and feature fusion on the image features to generate fused features, and perform small target detection based on the fused features.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-described methods for detecting small orbital targets.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for detecting small orbital targets.

[0017] The present invention provides a method, apparatus, electronic device, and storage medium for detecting small objects on a track. The method involves acquiring an image to be detected, which is obtained by photographing a track region containing small objects, including pedestrians and / or falling rocks. The image is then input into a pre-trained small object detection model to obtain the small object detection result output by the model. The small object detection model is pre-trained on an initial model based on an improved YOLOv11 algorithm, using sample detection images and their corresponding small object detection results. The small object detection model is used to extract features from the image to be detected, generate image features, perform feature alignment and feature fusion on the image features to generate fused features, and perform small object detection based on the fused features. By introducing a feature alignment-based fusion method into the small target detection model built on the improved YOLO11 algorithm, the small target detection model can extract features from the image to be detected, generate image features, and then perform feature alignment and feature fusion to generate fused features. Since the feature alignment-based fusion method can effectively fuse image features at different levels and reduce the spatial offset problem caused by feature downsampling, it is beneficial for the model to accurately locate small targets and perform small target detection based on fused features. This can accurately detect small targets such as pedestrians or falling rocks in the image to be detected, and improve the accuracy of track small target detection results. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a schematic flowchart of the orbital small target detection method provided by the present invention.

[0020] Figure 2 This is a schematic diagram of the structure of the small target detection model provided by the present invention.

[0021] Figure 3 This is a schematic diagram of the feature fusion network provided by the present invention.

[0022] Figure 4 This is a schematic diagram of the feature alignment network provided by the present invention.

[0023] Figure 5 This is a schematic diagram of the training process of the small target detection model provided by the present invention.

[0024] Figure 6 This is a schematic diagram of the structure of the small target detection device for orbit provided by the present invention.

[0025] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] Please see Figures 1 to 5 , Figure 1 This is a schematic flowchart of the orbital small target detection method provided by the present invention. Figure 2 This is a schematic diagram of the small target detection model provided by the present invention. Figure 3 This is a schematic diagram of the feature fusion network provided by the present invention. Figure 4 This is a schematic diagram of the feature alignment network provided by the present invention. Figure 5 This is a schematic diagram of the training process of the small target detection model provided by the present invention.

[0028] like Figure 1As shown, in this embodiment, the orbital small target detection method includes steps S110 to S120, and the specific steps are as follows: S110: Acquire the image to be detected.

[0029] The image to be detected is obtained by taking pictures of a track area containing small targets, including pedestrians and / or falling rocks.

[0030] S120: Input the image to be detected into the pre-trained small object detection model to obtain the small object detection result output by the small object detection model.

[0031] Among them, the small target detection model is obtained by pre-training an initial model based on the improved YOLO11 algorithm, based on the sample detection image and the corresponding sample small target detection results.

[0032] The small object detection model is used to extract features from the image to be detected, generate image features, perform feature alignment and feature fusion on the image features, generate fused features, and perform small object detection based on the fused features.

[0033] The track small target detection method provided in this embodiment acquires an image to be detected. The image to be detected is obtained by photographing a track region containing small targets, including pedestrians and / or falling rocks. The image to be detected is input into a pre-trained small target detection model to obtain the small target detection result output by the small target detection model. The small target detection model is obtained by pre-training an initial model based on an improved YOLO11 algorithm based on sample detection images and the corresponding sample small target detection results. The small target detection model is used to extract features from the image to be detected, generate image features, perform feature alignment and feature fusion on the image features to generate fused features, and perform small target detection based on the fused features. By introducing a feature alignment-based fusion method into the small target detection model built on the improved YOLO11 algorithm, the small target detection model can extract features from the image to be detected, generate image features, and then perform feature alignment and feature fusion to generate fused features. Since the feature alignment-based fusion method can effectively fuse image features at different levels and reduce the spatial offset problem caused by feature downsampling, it is beneficial for the model to accurately locate small targets and perform small target detection based on fused features. This can accurately detect small targets such as pedestrians or falling rocks in the image to be detected, and improve the accuracy of track small target detection results.

[0034] In some embodiments, the small target detection model includes a backbone network, a neck network, and a head network connected in sequence; wherein, the backbone network is used to extract features from the image to be detected and generate image features; the neck network is used to perform feature alignment and feature fusion on the image features and generate fused features; and the head network is used to perform small target detection based on the fused features and generate small target detection results.

[0035] like Figure 2 As shown, the small target detection model in this embodiment is an improvement on the existing YOLO11 network model. The small target detection model includes a backbone network, a neck network, and a head network connected in sequence.

[0036] The backbone network is used to extract multi-scale features from the image to be detected, generating multiple image features at different scales, including first image features, second image features, third image features, and fourth image features.

[0037] Specifically, the backbone network includes a first network layer connected in sequence. Second network layer Third network layer Fourth network layer and the fifth network layer Among them, the first network layer The first layer is used to extract features from the image to be detected and generate initial image features; the second network layer... The first image feature is extracted from the initial image features to generate the first image feature; the third network layer... The fourth network layer is used to extract features from the first image features and generate the second image features. The fifth network layer is used to extract features from the second image features and generate the third image features. This is used to extract features from the third image features and generate the fourth image features.

[0038] The neck network is used to perform feature alignment and feature fusion on image features of multiple different scales, generating multiple fused features, including a first fused feature, a second fused feature, a third fused feature, and a fourth fused feature.

[0039] Specifically, the first and second image features are adjacent image features, the second and third image features are adjacent image features, and the third and fourth image features are adjacent image features. The neck network can perform feature fusion operations based on feature alignment on adjacent image features. That is, before feature fusion, adjacent image features are first aligned. A deformable convolutional network (i.e., DCNv4 module) is used to construct a feature offset prediction network to predict the offset between high-level and low-level features (i.e., two adjacent image features). Then, the predicted offset is used to correct the high-level features layer by layer. Finally, the corrected high-level features are added to the low-level features and fused to obtain the fused feature. After all adjacent image features have completed feature alignment and feature fusion, the first fused feature can be generated. Second fusion feature Third fusion feature and the fourth fusion feature .

[0040] The existing YOLOv11 network model's head network includes three detection branches for using second fusion features. Third fusion feature and the fourth fusion feature The system performs target detection and generates track target detection results, which include the location information and target category information of large targets such as tracks or trains.

[0041] In this embodiment, an additional high-resolution detection branch is added to the existing YOLO11 network head network to improve the response to small targets. Since each detection branch includes one detection head, the head network of the small target detection model includes a first prediction head, a second prediction head, a third prediction head, and a fourth prediction head.

[0042] Among them, the second prediction head, the third prediction head, and the fourth prediction head are used based on the second fusion feature. Third fusion feature and the fourth fusion feature Target detection is performed to generate orbital target detection results; a first prediction head is used to perform target detection based on the first fusion features. It performs small target detection and generates small target detection results, which include small target location information and small target category information.

[0043] The small target detection method for orbits provided in this embodiment adds a high-resolution feature detection branch to the head network, which can effectively improve the model's response to small target detection.

[0044] In some embodiments, the backbone network includes a first network layer, a second network layer, a third network layer, a fourth network layer, and a fifth network layer connected in sequence, and the image features include a first image feature, a second image feature, a third image feature, and a fourth image feature; wherein, the first network layer is used to extract features from the image to be detected to generate initial image features; the second network layer is used to extract features from the initial image features to generate the first image features; the third network layer is used to extract features from the first image features to generate the second image features; the fourth network layer is used to extract features from the second image features to generate the third image features; and the fifth network layer is used to extract features from the third image features to generate the fourth image features.

[0045] like Figure 2 As shown, the backbone network is used to extract multi-scale features from the image to be detected, generating multiple image features at different scales, including first image features, second image features, third image features, and fourth image features.

[0046] Specifically, the backbone network includes a first network layer connected in sequence. Second network layer Third network layer Fourth network layer and the fifth network layer .

[0047] Among them, the first network layer The first layer is used to extract features from the image to be detected and generate initial image features; the second network layer... The first image feature is extracted from the initial image features to generate the first image feature; the third network layer... The fourth network layer is used to extract features from the first image features and generate the second image features. The fifth network layer is used to extract features from the second image features and generate the third image features. This is used to extract features from the third image features and generate the fourth image features.

[0048] Optionally, the stride convolutional modules of each network layer in the backbone network can be replaced with SPD-conv (spatial depth transformation convolution) modules. This allows the backbone network to retain all information of the image features while performing downsampling, which is beneficial for the subsequent detection of small targets on the track and reduces the false negative and false positive rates.

[0049] In some embodiments, the neck network includes an MLP module, a feature alignment network, and a feature fusion network connected in sequence. The feature alignment network and the feature fusion network are deformable convolutional networks. The fused features include a first fused feature, a second fused feature, a third fused feature, and a fourth fused feature. The MLP module is used to perform channel filtering on the first, second, third, and fourth image features respectively to generate a first feature map, a second feature map, a third feature map, and a fourth feature map. The feature alignment network is used to align the first, second, third, and fourth feature maps to obtain a first feature map to be fused, a second feature map to be fused, a third feature map to be fused, and a fourth feature map to be fused. The feature fusion network is used to perform feature fusion based on the first, second, third, and fourth feature maps to be fused, generating the first fused feature, the second fused feature, the third fused feature, and the fourth fused feature, respectively.

[0050] Specifically, the neck network includes an MLP module, a feature alignment network, and a feature fusion network connected in sequence, wherein the feature alignment network and the feature fusion network are deformable convolutional networks.

[0051] The MLP (Multilayer Perceptron) module can filter the channel dimensions of the four image features (i.e., the first image feature, the second image feature, the third image feature, and the fourth image feature) extracted by the backbone network based on the channel hybrid attention mechanism. This reduces the channel dimensions of the four image features at different scales, removes redundant features, reduces the computational cost of the neck network, and finally generates the first feature map, the second feature map, the third feature map, and the fourth feature map after removing redundant features.

[0052] like Figure 4 As shown, the feature alignment network is used to align the first feature map, the second feature map, the third feature map, and the fourth feature map to obtain the first feature map to be fused, the second feature map to be fused, the third feature map to be fused, and the fourth feature map to be fused.

[0053] Specifically, the first and second feature maps are adjacent feature maps, the second and third feature maps are adjacent feature maps, and the third and fourth feature maps are adjacent feature maps. For any two adjacent feature maps, the feature alignment network can perform feature alignment on these two feature maps. Since the feature alignment network is a feature offset prediction network constructed using a deformable convolutional network, the feature alignment network can predict the feature offset of these two adjacent feature maps, and then use the predicted offset to perform feature correction, thereby achieving feature alignment.

[0054] For example Figure 4In the middle, for feature maps The feature alignment network can align the feature map. An upsampling operation is performed, and feature offsets are predicted based on the adjacent low-level feature maps. Then, the predicted offsets are used to adjust the feature maps. Perform feature correction to achieve feature alignment and generate aligned feature maps to be fused. .

[0055] like Figure 3 As shown, the feature fusion network is used to perform feature fusion based on the first feature map to be fused, the second feature map to be fused, the third feature map to be fused, and the fourth feature map to be fused, and generate the first fused feature, the second fused feature, the third fused feature, and the fourth fused feature, respectively.

[0056] Specifically, the first and second feature maps to be fused are adjacent feature maps, the second and third feature maps to be fused are adjacent feature maps, and the third and fourth feature maps to be fused are adjacent feature maps. For any two adjacent feature maps to be fused, the feature fusion network can add and fuse these two feature maps to generate the first fused feature map. Second fusion feature Third fusion feature and the fourth fusion feature .

[0057] Among them, the second fusion feature The third fused feature map is obtained by adding and fusing the first and second feature maps. The fourth fused feature map is obtained by adding and fusing the second and third feature maps to be fused. The first feature map is obtained by adding and fusing the third and fourth feature maps to be fused. The first feature map to be fused can be directly used as the first fused feature. .

[0058] The small target detection method for tracks provided in this embodiment adopts a multi-scale feature fusion network based on feature alignment, which can better fuse features of different scales, reduce the spatial offset problem caused by feature downsampling, and help the model to better locate small targets. At the same time, the neck network adopts an attention mechanism based on channel mixing, which can effectively reduce the feature dimension and reduce the computational load of the model while reducing redundant features.

[0059] In some embodiments, the head network includes a first prediction head, a second prediction head, a third prediction head, and a fourth prediction head; wherein the second prediction head, the third prediction head, and the fourth prediction head are used to perform target detection based on the second fusion feature, the third fusion feature, and the fourth fusion feature, and generate an orbital target detection result; the first prediction head is used to perform small target detection based on the first fusion feature and generate a small target detection result; the small target detection result includes small target location information and small target category information.

[0060] The existing YOLOv11 network model's head network includes three detection branches for using second fusion features. Third fusion feature and the fourth fusion feature The system performs target detection and generates track target detection results, which include the location information and target category information of large targets such as tracks or trains.

[0061] In this embodiment, an additional high-resolution detection branch is added to the existing YOLO11 network head network to improve the response to small targets. Since each detection branch includes one detection head, the head network of the small target detection model includes a first prediction head, a second prediction head, a third prediction head, and a fourth prediction head.

[0062] Among them, the second prediction head, the third prediction head, and the fourth prediction head are used based on the second fusion feature. Third fusion feature and the fourth fusion feature Target detection is performed to generate orbital target detection results; a first prediction head is used to perform target detection based on the first fusion features. It performs small target detection and generates small target detection results, which include small target location information and small target category information.

[0063] The small target detection method for orbits provided in this embodiment adds a high-resolution feature detection branch to the head network, which can effectively improve the model's response to small target detection.

[0064] In some embodiments, the small target detection model is pre-trained based on a preset loss function, which is determined based on localization loss, classification loss, confidence loss, and self-supervised contrastive loss.

[0065] Specifically, the small object detection model is pre-trained based on a preset loss function, which is determined based on localization loss, classification loss, confidence loss, and self-supervised contrastive loss. During the pre-training process, the model can perform supervised learning on the prediction results based on the localization loss, classification loss, confidence loss, and self-supervised contrastive loss, continuously updating the network weights until the model pre-training is completed.

[0066] Optionally, the network weights are initialized using pre-trained weights, the main training parameters are set to batch size of 8, epoch size of 100, the optimizer is optimized using stochastic gradient descent (SGD) algorithm, the initial learning rate is set to 0.01, and the network scale is set to... .

[0067] In some embodiments, before inputting the image to be detected into a pre-trained small object detection model and obtaining the small object detection result output by the small object detection model, the method further includes: constructing a dataset; the dataset includes multiple sample detection images collected in test line scenarios and actual running line scenarios, and the sample small object detection result corresponding to each sample detection image; dividing the dataset into a training set, a test set, and a validation set; pre-training the initial model constructed based on the improved YOLO11 algorithm based on the training set and the validation set to obtain the small object detection model; and testing the performance metrics of the small object detection model based on the test set.

[0068] Understandably, a small object detection model needs to be trained before it can be used for small object detection.

[0069] like Figure 5 As shown, multiple sample detection images are collected in the test line scenario, and the small target detection results corresponding to each sample detection image are labeled to obtain the long-distance small target dataset; multiple sample detection images are collected in the actual running line scenario, and the small target detection results corresponding to each sample detection image are labeled to obtain the actual running line small target dataset; the long-distance small target dataset and the actual running line small target dataset are merged to obtain the dataset.

[0070] It should be noted that the sample detection images collected in the test line scenario are images that simulate real-world scenarios. These data are difficult to obtain in actual operational scenarios, and the sample size is small. Therefore, by simulating real-world scenarios, the problem of scarce real-world scenario data can be made up for.

[0071] Specifically, for the test line scenario, pedestrians were collected at different distances (e.g., 150 meters to 20 meters) and within a range of 100 meters to 50 meters under different time periods and weather conditions. Images of scenes such as falling rocks within a cm radius were used as sample detection images.

[0072] Optionally, pedestrians can choose to wear clothing of different colors, and the falling rocks can be simulated stones of different shapes and colors.

[0073] Optionally, the number of sample detection images in the test line scenario is at least 11,000.

[0074] Specifically, for actual operating line scenarios, pedestrian data is collected from different monitoring angles and under different lighting conditions in different railway monitoring scenarios, such as construction worker data at night.

[0075] Optionally, the number of sample detection images in the actual operational scenario is at least 21,000.

[0076] Furthermore, the dataset is divided into training, testing, and validation sets according to a preset ratio (e.g., 7:2:1). Based on the training and validation sets, the initial model built on the improved YOLO11 algorithm is pre-trained to obtain a small object detection model.

[0077] Furthermore, based on the test set, the performance metrics of the pre-trained small object detection model are tested.

[0078] Optionally, performance metrics testing includes recall and average precision metrics testing.

[0079] To further illustrate the improvement effect of the small target detection model in this embodiment, the YOLO11n network model is used as the benchmark model. The accuracy and recall of the benchmark model and the improved small target detection model in this embodiment are tested respectively. The comparison of the indicators of the two models is shown in Table 1.

[0080] Table 1

[0081] As shown in Table 1, the improved small target detection model in this embodiment has improved both accuracy and recall.

[0082] This invention also provides a device for detecting small targets on orbit. Please refer to [link / reference]. Figure 6 , Figure 6 This is a schematic diagram of the orbital small target detection device provided by the present invention. In this embodiment, the orbital small target detection device includes an acquisition module 610 and a small target detection module 620.

[0083] The acquisition module 610 is used to acquire the image to be detected.

[0084] The image to be detected is obtained by taking pictures of a track area containing small targets, including pedestrians and / or falling rocks.

[0085] The small object detection module 620 is used to input the image to be detected into a pre-trained small object detection model to obtain the small object detection result output by the small object detection model.

[0086] Among them, the small target detection model is obtained by pre-training an initial model based on the improved YOLO11 algorithm based on the sample detection image and the sample small target detection results corresponding to the sample detection image; The small object detection model is used to extract features from the image to be detected, generate image features, perform feature alignment and feature fusion on the image features, generate fused features, and perform small object detection based on the fused features.

[0087] In some embodiments, the small target detection model includes a backbone network, a neck network, and a head network connected in sequence; wherein, the backbone network is used to extract features from the image to be detected and generate image features; the neck network is used to perform feature alignment and feature fusion on the image features and generate fused features; and the head network is used to perform small target detection based on the fused features and generate small target detection results.

[0088] In some embodiments, the backbone network includes a first network layer, a second network layer, a third network layer, a fourth network layer, and a fifth network layer connected in sequence, and the image features include a first image feature, a second image feature, a third image feature, and a fourth image feature; wherein, the first network layer is used to extract features from the image to be detected to generate initial image features; the second network layer is used to extract features from the initial image features to generate the first image features; the third network layer is used to extract features from the first image features to generate the second image features; the fourth network layer is used to extract features from the second image features to generate the third image features; and the fifth network layer is used to extract features from the third image features to generate the fourth image features.

[0089] In some embodiments, the neck network includes an MLP module, a feature alignment network, and a feature fusion network connected in sequence. The feature alignment network and the feature fusion network are deformable convolutional networks. The fused features include a first fused feature, a second fused feature, a third fused feature, and a fourth fused feature. The MLP module is used to perform channel filtering on the first, second, third, and fourth image features respectively to generate a first feature map, a second feature map, a third feature map, and a fourth feature map. The feature alignment network is used to align the first, second, third, and fourth feature maps to obtain a first feature map to be fused, a second feature map to be fused, a third feature map to be fused, and a fourth feature map to be fused. The feature fusion network is used to perform feature fusion based on the first, second, third, and fourth feature maps to be fused, generating the first fused feature, the second fused feature, the third fused feature, and the fourth fused feature, respectively.

[0090] In some embodiments, the head network includes a first prediction head, a second prediction head, a third prediction head, and a fourth prediction head; wherein the second prediction head, the third prediction head, and the fourth prediction head are used to perform target detection based on the second fusion feature, the third fusion feature, and the fourth fusion feature, and generate an orbital target detection result; the first prediction head is used to perform small target detection based on the first fusion feature and generate a small target detection result; the small target detection result includes small target location information and small target category information.

[0091] In some embodiments, the small target detection model is pre-trained based on a preset loss function, which is determined based on localization loss, classification loss, confidence loss, and self-supervised contrastive loss.

[0092] In some embodiments, the small object detection module 620 is further configured to construct a dataset; the dataset includes multiple sample detection images collected in test line scenarios and actual operation line scenarios, and the sample small object detection results corresponding to each sample detection image; the dataset is divided into a training set, a test set, and a validation set; based on the training set and the validation set, the initial model constructed based on the improved YOLO11 algorithm is pre-trained to obtain a small object detection model; based on the test set, the performance index of the small object detection model is tested.

[0093] The present invention also provides an electronic device. Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 7As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740. The processor 710, communications interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions from the memory 730 to execute a small target detection method.

[0094] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0095] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the orbital small target detection method provided by the above methods.

[0096] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0097] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting small targets on an orbit, characterized in that, include: Acquire an image to be detected; the image to be detected is obtained by taking a picture of a track area containing small targets, including pedestrians and / or falling rocks; The image to be detected is input into a pre-trained small object detection model to obtain the small object detection result output by the small object detection model. The small target detection model is obtained by pre-training an initial model based on the improved YOLO11 algorithm, based on the sample detection image and the corresponding sample small target detection results. The small target detection model is used to extract features from the image to be detected, generate image features, perform feature alignment and feature fusion on the image features to generate fused features, and perform small target detection based on the fused features; The small target detection model includes a backbone network, a neck network, and a head network connected in sequence. The backbone network is used to extract features from the image to be detected and generate the image features; the image features include a first image feature, a second image feature, a third image feature, and a fourth image feature. The neck network is used to perform feature alignment and feature fusion on the image features to generate the fused features; The head network is used to perform small target detection based on the fused features and generate the small target detection result. The neck network includes an MLP module, a feature alignment network, and a feature fusion network connected in sequence. The feature alignment network and the feature fusion network are deformable convolutional networks. The fusion features include a first fusion feature, a second fusion feature, a third fusion feature, and a fourth fusion feature. The MLP module is used to perform channel filtering processing on the first image feature, the second image feature, the third image feature and the fourth image feature respectively to generate a first feature map, a second feature map, a third feature map and a fourth feature map; The feature alignment network is used to perform feature alignment on the first feature map, the second feature map, the third feature map, and the fourth feature map to obtain a first feature map to be fused, a second feature map to be fused, a third feature map to be fused, and a fourth feature map to be fused. The feature fusion network is used to perform feature fusion based on the first feature map to be fused, the second feature map to be fused, the third feature map to be fused, and the fourth feature map to be fused, to generate the first fused feature, the second fused feature, the third fused feature, and the fourth fused feature, respectively. The small target detection model is pre-trained based on a preset loss function, which is determined based on localization loss, classification loss, confidence loss, and self-supervised contrastive loss.

2. The method for detecting small targets on an orbit according to claim 1, characterized in that, The backbone network includes a first network layer, a second network layer, a third network layer, a fourth network layer, and a fifth network layer connected in sequence. The first network layer is used to extract features from the image to be detected and generate initial image features. The second network layer is used to extract features from the initial image features to generate the first image features; The third network layer is used to extract features from the first image features and generate the second image features. The fourth network layer is used to extract features from the second image features and generate the third image features; The fifth network layer is used to extract features from the third image features and generate the fourth image features.

3. The method for detecting small targets on an orbit according to claim 1, characterized in that, The head network includes a first prediction head, a second prediction head, a third prediction head, and a fourth prediction head; The second prediction head, the third prediction head, and the fourth prediction head are used to perform target detection based on the second fusion feature, the third fusion feature, and the fourth fusion feature, and generate orbital target detection results. The first prediction head is used to perform small target detection based on the first fusion feature and generate the small target detection result; the small target detection result includes small target location information and small target category information.

4. The method for detecting small targets in orbit according to claim 1, characterized in that, Before inputting the image to be detected into the pre-trained small object detection model and obtaining the small object detection result output by the small object detection model, the method further includes: Construct a dataset; the dataset includes multiple sample detection images collected in test line scenarios and actual operation line scenarios, and the sample small target detection results corresponding to each sample detection image; The dataset is divided into a training set, a test set, and a validation set; Based on the training set and the validation set, the initial model constructed based on the improved YOLO11 algorithm is pre-trained to obtain the small target detection model; Based on the test set, the performance metrics of the small target detection model are tested.

5. A device for detecting small targets on an orbit, characterized in that, include: An acquisition module is used to acquire an image to be detected; the image to be detected is obtained by taking a picture of a track area containing small targets, including pedestrians and / or falling rocks; The small object detection module is used to input the image to be detected into a pre-trained small object detection model and obtain the small object detection result output by the small object detection model. The small target detection model is obtained by pre-training an initial model based on the improved YOLO11 algorithm, based on the sample detection image and the corresponding sample small target detection results. The small target detection model is used to extract features from the image to be detected, generate image features, perform feature alignment and feature fusion on the image features to generate fused features, and perform small target detection based on the fused features; The small target detection model includes a backbone network, a neck network, and a head network connected in sequence. The backbone network is used to extract features from the image to be detected and generate the image features; the image features include a first image feature, a second image feature, a third image feature, and a fourth image feature. The neck network is used to perform feature alignment and feature fusion on the image features to generate the fused features; The head network is used to perform small target detection based on the fused features and generate the small target detection result. The neck network includes an MLP module, a feature alignment network, and a feature fusion network connected in sequence. The feature alignment network and the feature fusion network are deformable convolutional networks. The fusion features include a first fusion feature, a second fusion feature, a third fusion feature, and a fourth fusion feature. The MLP module is used to perform channel filtering processing on the first image feature, the second image feature, the third image feature and the fourth image feature respectively to generate a first feature map, a second feature map, a third feature map and a fourth feature map; The feature alignment network is used to perform feature alignment on the first feature map, the second feature map, the third feature map, and the fourth feature map to obtain a first feature map to be fused, a second feature map to be fused, a third feature map to be fused, and a fourth feature map to be fused. The feature fusion network is used to perform feature fusion based on the first feature map to be fused, the second feature map to be fused, the third feature map to be fused, and the fourth feature map to be fused, to generate the first fused feature, the second fused feature, the third fused feature, and the fourth fused feature, respectively. The small target detection model is pre-trained based on a preset loss function, which is determined based on localization loss, classification loss, confidence loss, and self-supervised contrastive loss.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the orbital small target detection method as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the orbital small target detection method as described in any one of claims 1 to 4.

Images