Remote sensing real-time detection method and system based on improved RT-DETR

Abstract

The invention provides a remote sensing real-time detection method and system based on improved RT-DETR, and the method comprises the steps: obtaining a preprocessed remote sensing image; inputting a remote sensing image into the improved RT-DETR model of which the auxiliary algorithm branch is closed to perform preliminary training of the model; starting an auxiliary algorithm branch and training; after auxiliary algorithm training is completed, synchronously training a model trunk and auxiliary algorithm branches; after synchronous training is completed, auxiliary algorithm branches are closed, and the optimal weight is selected for remote sensing image target detection. According to the remote sensing real-time detection method and system based on the improved RT-DETR, the RT-DETR is improved, an improved model is helped to more accurately position a remote sensing detection target by depending on an auxiliary algorithm branch and a training strategy, the speed and precision of model target detection are improved, the computing power overhead is reduced, and further, the real-time detection efficiency is improved. And the reliability of remote sensing image target detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a remote sensing real-time detection method and system based on improved RT-DETR. Background Art

[0002] Remote sensing image target detection technology plays a vital role in numerous fields, including military reconnaissance, environmental monitoring, and urban planning. In recent years, with the continuous advancement of computer vision and artificial intelligence technologies, particularly the booming development of deep learning, the fields of image processing and target detection have ushered in unprecedented opportunities, and remote sensing image target detection technology has made significant progress. Deep learning, with its powerful feature learning and pattern recognition capabilities, can automatically learn rich feature representations from large amounts of remote sensing image data, thereby achieving efficient target detection.

[0003] The YOLO series has become a very popular framework for real-time remote sensing detection due to its excellent balance between high precision and high speed. Since the emergence of the DETR framework, many excellent algorithms have also emerged, such as RT-DETR and D-FINE.

[0004] However, the model based on the current DETR framework algorithm is relatively large, and for edge deployment tasks, the computing power overhead is high and the target detection speed is slow.

[0005] In view of this, there is an urgent need for a remote sensing real-time detection method and system based on improved RT-DETR to at least solve the above-mentioned shortcomings. Summary of the Invention

[0006] One of the purposes of the present invention is to provide a remote sensing real-time detection method and system based on improved RT-DETR, which improves RT-DETR and relies on auxiliary algorithm branches and training strategies to help improve the model to more accurately locate remote sensing detection targets, thereby improving the speed and accuracy of model target detection, reducing computing power overhead, and further improving the reliability of remote sensing image target detection.

[0007] An embodiment of the present invention provides a remote sensing real-time detection method based on an improved RT-DETR, comprising:

[0008] Obtain preprocessed remote sensing images;

[0009] The remote sensing images were input into the improved RT-DETR model with the auxiliary algorithm branch closed for preliminary training of the model;

[0010] Enable auxiliary algorithm branch and train;

[0011] After the auxiliary algorithm training is completed, the model trunk and auxiliary algorithm branch are trained synchronously;

[0012] After the synchronous training is completed, the auxiliary algorithm branch is closed and the optimal weight is selected for remote sensing image target detection.

[0013] Preferably, the improvement steps of improving the RT-DETR model include:

[0014] Adaptive rotation convolution is used to replace the standard convolution in the backbone network of the RT-DETR-R50 model, and an auxiliary algorithm branch is added after the Efficient Hybrid Encoder in the RT-DETR-R50 model.

[0015] Preferably, adaptive rotation convolution is used to replace the standard convolution in the backbone network of the RT-DETR-R50 model, including:

[0016] The 3*3 adaptive rotation convolution is used to replace the last layer of 3*3 standard convolution of the backbone network ResNet50 in the RT-DETR-R50 model.

[0017] Preferably, the auxiliary algorithm branch includes:

[0018] The auxiliary algorithm corresponding to the auxiliary algorithm branch is a one-to-many matching supervision algorithm.

[0019] Preferably, the improvement step of improving the RT-DETR model further includes:

[0020] The CCFF module in the Efficient Hybrid Encoder structure is removed, and the decoder structure is changed from the original 6-layer to a 4-layer decoder.

[0021] Preferably, the auxiliary algorithm branch also includes:

[0022] According to the output features of the backbone network and the Efficient Hybrid Encoder, the predicted anchor box coordinates are output;

[0023] Determine the feature level of the target based on the size of the predicted anchor box;

[0024] Perform RoIAlign operation on the corresponding coordinates of the predicted anchor box coordinates in the feature level to extract the required features;

[0025] The anchor box coordinates and the required features are combined into an auxiliary query and fed into the Decoder for subsequent operations together with the initialization query generated by the model backbone.

[0026] An embodiment of the present invention provides a remote sensing real-time detection method based on improved RT-DETR, further comprising:

[0027] Determine the detection interval between target detection results and associated target detection results of remote sensing image target detection;

[0028] Determining the target type of the first detection target corresponding to the target detection result;

[0029] Determine the expected target detection results based on the evolution model, detection interval, and evolution environment corresponding to the target type;

[0030] Comparing the expected target detection result with the target detection result, and determining a second detection target whose result difference is greater than a preset difference threshold;

[0031] Screening suspicious areas based on the positional relationship and category relationship of the second detection target;

[0032] Determine the auxiliary verification strategy for the suspicious area based on the description of the result difference corresponding to each suspicious area;

[0033] Verify according to the auxiliary verification strategy and output abnormal warning.

[0034] Preferably, determining the auxiliary verification strategy for the suspicious area according to the result difference description corresponding to each suspicious area includes:

[0035] Traverse each suspicious area in turn, and retrieve the knowledge graph knowledge containing the corresponding entity of the second detection target in the target suspicious area being traversed;

[0036] Determine the model based on the knowledge graph knowledge training auxiliary verification strategy;

[0037] The result difference description corresponding to the second detection target in the target suspicious area is input into the auxiliary verification strategy determination model to obtain the auxiliary verification strategy.

[0038] Preferably, verification is performed according to the auxiliary verification strategy and an abnormal warning is output, including:

[0039] Determine the verification type of the auxiliary verification strategy, which includes online verification and offline verification;

[0040] If the verification type is offline verification, determine the verification features of the corresponding suspicious area, which include: the mutual support relationship between the verification subject and the auxiliary verification strategy;

[0041] Construct collaborative verification judgment factors based on verification characteristics;

[0042] Determine the collaborative verification strategy based on the collaborative verification determination factor;

[0043] Determine the verification route for suspicious areas of offline verification based on the collaborative verification strategy.

[0044] An embodiment of the present invention provides a remote sensing real-time detection system based on an improved RT-DETR, comprising:

[0045] A preprocessing module is used to obtain preprocessed remote sensing images;

[0046] The first training module is used to input remote sensing images into the improved RT-DETR model with the auxiliary algorithm branch closed for preliminary training of the model;

[0047] The second training module is used to start the auxiliary algorithm branch and train it;

[0048] The third training module is used to synchronously train the model trunk and auxiliary algorithm branches after the auxiliary algorithm training is completed;

[0049] The detection module is used to close the auxiliary algorithm branch after the synchronous training is completed and select the optimal weight for remote sensing image target detection.

[0050] The beneficial effects of the present invention are:

[0051] The present invention improves RT-DETR and relies on auxiliary algorithm branches and training strategies to help improve the model to more accurately locate remote sensing detection targets, thereby improving the speed and accuracy of model target detection, reducing computing power overhead, and further improving the reliability of remote sensing image target detection.

[0052] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in this application document.

[0053] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0055] Figure 1 Schematic diagram of a remote sensing real-time detection method based on improved RT-DETR in an embodiment of the present invention;

[0056] Figure 2 This is a diagram showing the adaptive rotational convolution function in an embodiment of the present invention;

[0057] Figure 3 A branch display diagram in an embodiment of the present invention;

[0058] Figure 41 is a flowchart of the auxiliary algorithm in an embodiment of the present invention;

[0059] Figure 5 This is a diagram showing the proportions of various structures in the model in an embodiment of the present invention;

[0060] Figure 6 A comparison chart of the quality of basic queries and auxiliary queries in an embodiment of the present invention;

[0061] Figure 7 Schematic diagram of a remote sensing real-time detection system based on improved RT-DETR in an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0063] The embodiment of the present invention provides a remote sensing real-time detection method based on improved RT-DETR, such as Figure 1 As shown, including:

[0064] Step 1: Obtain preprocessed remote sensing images;

[0065] The preprocessed remote sensing images are: the images in the public remote sensing dataset DIOR are uniformly set to 800×800 pixels and divided into training set, validation set and test set in a ratio of 1:1:2;

[0066] Step 2: Input the remote sensing image into the improved RT-DETR model with the auxiliary algorithm branch closed for preliminary training of the model;

[0067] During the initial training, the model backbone is trained with the auxiliary algorithm branches closed until convergence, retaining the optimal weights.

[0068] Among them, when improving the RT-DETR model, the 3*3 adaptive rotation convolution is used to replace the 3*3 standard convolution of the last layer in the original backbone network ResNet50. The adaptive rotation convolution function is shown as follows: Figure 2 As shown in the figure, the model structure is simplified, the CCFF module in the Efficient Hybrid Encoder structure is removed, and the number of layers of the Decoder structure is reduced from the original 6 layers to 4 layers. An additional auxiliary algorithm branch is set after the Efficient Hybrid Encoder, and the branch is shown as follows: Figure 3As shown, the output features of the backbone network and the Efficient Hybrid Encoder are used as the input of the branch, and the anchor frame coordinates and features output by the auxiliary algorithm branch are used to generate auxiliary queries, which enter the subsequent operations together with the basic queries generated by the backbone. The auxiliary algorithm workflow is as follows Figure 4 As shown;

[0069] Step 3: Enable the auxiliary algorithm branch and train it;

[0070] Among them, when opening the auxiliary algorithm branch and training, based on the initial model training, the main part of the model is frozen, the auxiliary algorithm branch is opened, and FCOS is used as a one-to-many matching supervision auxiliary algorithm. It is trained until convergence and the optimal weight is retained;

[0071] Step 4: After the auxiliary algorithm training is completed, the model trunk and auxiliary algorithm branch are trained synchronously;

[0072] During synchronous training, the auxiliary algorithm provides additional high-quality auxiliary queries to the main backbone, enriching the encoder's positive sample supervision and providing the decoder with more accurately positioned initialization queries. The optimal weights are retained until training converges.

[0073] Step 5: After the synchronous training is completed, close the auxiliary algorithm branch and select the optimal weight for remote sensing image target detection.

[0074] Among them, selecting the optimal weight for remote sensing image target detection refers to the process of using the trained optimal weight to perform target detection on the input remote sensing image that requires target detection.

[0075] The working principle and beneficial effects of the above technical solution are:

[0076] This invention introduces adaptive rotational convolution, making the model backbone network rotationally invariant, better addressing the challenge of variable instance orientations in remote sensing. Furthermore, the simplified model structure improves real-time performance and reduces weight. A better query generation method effectively enhances the model's localization capabilities, significantly improving accuracy without introducing any additional computational overhead or latency during target detection.

[0077] Specifically, the experimental platform for measuring target detection delay in this embodiment is CPU: Intel(R) Xeon(R) Gold6230R CPU@2.10GHz, GPU: single NVIDIA GeForce RTX 3090.

[0078] After training under the above conditions, the algorithm performance is shown in Table 1:

[0079] Table 1 Performance statistics of different algorithms

[0080]

[0081] In summary, compared with the RT-DETR-R50 benchmark algorithm, the proposed method shows better performance in terms of accuracy, speed, number of parameters, and amount of computation. 50:95 The accuracy achieved excellent performance.

[0082] The results of the ablation experiment on the simplified model are shown in Table 2:

[0083] Table 2 Model simplification ablation experiment table

[0084]

[0085] Figure 5 The impact of each model module on lightweight and real-time performance is provided. From the presented experimental data, it can be seen that compared with the original model, the simplified model consumes 62% of the original model's computational effort and achieves a 1.38-fold increase in prediction speed.

[0086] The results of ablation experiments on better query generation methods are shown in Table 3:

[0087] Table 3 Ablation experiment table of different auxiliary algorithms for assisted query

[0088]

[0089] At the same time, three auxiliary algorithms were tested: ATSS, Faster R-CNN, and FCOS. The experiment showed that after adding the auxiliary algorithm, AP 50:95 The significant improvement indicates that the generated high-quality queries enable the model to locate targets more accurately. Figure 6 The comparison of positioning quality between auxiliary query and basic query is shown.

[0090] Table 4 Complete ablation experiment table

[0091]

[0092] The complete ablation experiment is shown in Table 4. In this set of experiments, it is clear that the enhanced query generation method effectively improves the accuracy of the model without increasing the computational cost and speed, especially in improving the accuracy of small objects.

[0093]

[0094] The significant comparison between Variant C and Baseline shows the importance of feature rotation invariance for target detection tasks such as remote sensing, where the orientation of instances vary.

[0095] At the same time, the effectiveness of the algorithm was tested on the NWPU VHR-10. The image size was also set to 800*800 pixels, and the dataset was divided into 6:2:2. The dataset classification is shown in Table 5:

[0096] Table 5 Dataset classification table

[0097]

[0098] The experimental data of the algorithm effectiveness on a small dataset are shown in Table 6:

[0099] Table 6 Algorithm effectiveness experiment table on small dataset

[0100]

[0101] As can be seen, the present invention also achieves excellent performance on small datasets, demonstrating its good generalization ability. Therefore, the present invention is not limited to scenarios with large-scale datasets. Even with limited data, it can still effectively learn key features and achieve high-precision detection results.

[0102] The present invention improves RT-DETR and relies on auxiliary algorithm branches and training strategies to help improve the model to more accurately locate remote sensing detection targets, thereby improving the speed and accuracy of model target detection, reducing computing power overhead, and further improving the reliability of remote sensing image target detection.

[0103] The embodiment of the present invention provides a remote sensing real-time detection method based on improved RT-DETR, further comprising:

[0104] Determine the detection interval between target detection results and associated target detection results of remote sensing image target detection;

[0105] The target detection result is: the local remote sensing image of the target (first detection target) detected in the currently input remote sensing image; the associated target detection result is: the local remote sensing image of the first detection target that was most recently identified at the current moment;

[0106] Determining the target type of the first detection target corresponding to the target detection result;

[0107] Wherein, the target type is: the type identifier of the first detection target;

[0108] Determine the expected target detection results based on the evolution model, detection interval, and evolution environment corresponding to the target type;

[0109] The evolution model is a model of target type changes with the environment and time; the evolution environment is the environmental changes of the first detected target within the detection interval; the expected target detection result is the local remote sensing image of the first detected target at the current moment output by the evolution model;

[0110] Comparing the expected target detection result with the target detection result, and determining a second detection target whose result difference is greater than a preset difference threshold;

[0111] The result difference is the image difference between the local remote sensing image of the target (first detection target) detected in the current input remote sensing image and the expected local remote sensing image of the first detection target at the current moment; the preset difference threshold is manually set in advance; the second detection target is the first detection target whose corresponding result difference is greater than the difference threshold;

[0112] Screening suspicious areas based on the positional relationship and category relationship of the second detection target;

[0113] Among them, the position relationship is: the geographical distance between the second detection targets; the category relationship is: the category association relationship between the second detection targets; when screening suspicious areas, the second detection target cluster that meets the screening conditions is determined, and the screening conditions include: the geographical distance is less than the distance threshold, and the category association is greater than the association threshold; the area corresponding to the smallest encirclement containing the corresponding second detection target cluster is regarded as a suspicious area; the distance threshold and the association threshold are both manually preset;

[0114] Determine the auxiliary verification strategy for the suspicious area based on the description of the result difference corresponding to each suspicious area;

[0115] The result difference is described as: the semantic recognition result of the result difference; when determining the auxiliary verification strategy, each suspicious area is traversed in turn, the suspicious area being traversed is taken as the target suspicious area, the knowledge graph knowledge containing the corresponding entity of the second detection target in the target suspicious area is retrieved, the auxiliary verification strategy determination model is trained using the knowledge graph knowledge, the result difference description corresponding to the second detection target in the target suspicious area is input into the auxiliary verification strategy determination model to obtain the output auxiliary verification strategy, and the auxiliary verification strategy is: a strategy that helps verify what causes the result difference;

[0116] Verify according to the auxiliary verification strategy and output abnormal warning.

[0117] The working principle and beneficial effects of the above technical solution are:

[0118] Remote sensing targets evolve differently over time depending on their type. For example, buildings are relatively fixed, while vegetation changes with climate change. Therefore, adaptive anomaly warnings are needed based on the target type and the corresponding degree of change.

[0119] Specifically, for example, the target detection result is a local remote sensing image of vegetation A at the current time t1, the associated target detection result is a local remote sensing image at time t2, the last time vegetation A was detected at time t1, and the target type is the type identifier "vegetation A." The evolution model is a model of how vegetation A changes over time and in the environment. The climate information (evolutionary environment) of vegetation A during the detection interval, the detection interval, and the local remote sensing image of vegetation A at time t2 are input into the evolution model to obtain an expected local remote sensing image of vegetation A at time t1. The expected local remote sensing image of vegetation A at time t1 (expected target detection result) is compared with the actual local remote sensing image (target detection result). If the image difference is greater than, for example, 15%, vegetation A is selected as the second detection target.

[0120] Since target detection types are diverse, there may be more than one second detection target. For example, based on the above-mentioned similar process of determining the second detection target, vegetation B and C are also second detection targets.

[0121] At the same time, the anomaly of the second detection target may not be an independent event. For example, a certain influencing factor occurs in the area, and the vegetation A and C in the area will be affected by the influencing factor. In order to save verification resources in the future, when determining the suspicious area for investigation, the suspicious area can be determined based on the position relationship and category relationship of the second detection target. The number of suspicious areas can be 0, 1 or more.

[0122] When determining a suspicious area, it is necessary to connect with other nodes to further verify the cause of the result difference. Therefore, the auxiliary verification strategy of the suspicious area is determined according to the result difference description corresponding to each suspicious area; when determining the auxiliary verification strategy, the knowledge graph knowledge of the entity containing the second detection target corresponding to the target area being traversed is introduced, such as: the graph knowledge containing the entities "vegetation A" and "vegetation C", and the auxiliary verification strategy determination model is trained according to the corresponding knowledge graph knowledge. The model is based on the input result difference description, such as: "The areas of vegetation A and vegetation C are decreasing rapidly at an unusual rate" and "Vegetation A and vegetation C are close to residential areas" are input into the auxiliary verification strategy determination model, and the model outputs the auxiliary verification strategy of "calling vegetation area monitoring to check whether there is illegal logging".

[0123] Finally, verification is performed according to the auxiliary verification strategy. If there is an abnormality in the missing data, such as private logging, an abnormal warning is output.

[0124] The present invention introduces an evolutionary model, a detection interval, and an evolutionary environment to determine the expected target detection result, compares the expected target detection result with the target detection result to determine the second detection target that requires abnormal verification, introduces the positional relationship and category relationship of the second detection target to determine the suspicious area, and saves subsequent verification resources; introduces a knowledge graph containing the corresponding entity of the second detection target to train an auxiliary verification strategy determination model, and determines the auxiliary verification strategy corresponding to the suspicious area according to the result difference description corresponding to each suspicious area and the auxiliary verification strategy determination model, thereby improving the determination accuracy and efficiency of the auxiliary verification strategy.

[0125] In one embodiment, performing verification according to the auxiliary verification strategy and outputting an abnormality warning includes:

[0126] Determine the verification type of the auxiliary verification strategy, which includes online verification and offline verification;

[0127] Online verification refers to verification by connecting to online network nodes, such as vegetation area monitoring nodes; offline verification requires on-site verification in the corresponding suspicious area, such as a vegetation area without monitoring, requiring an on-site visit.

[0128] If the verification type is offline verification, determine the verification features of the corresponding suspicious area, which include: the mutual support relationship between the verification subject and the auxiliary verification strategy;

[0129] The verification subject relationship refers to the association between verification subjects; the mutual support relationship of auxiliary verification strategies refers to whether it is necessary to verify another auxiliary verification result based on one auxiliary verification result;

[0130] Construct collaborative verification judgment factors based on verification characteristics;

[0131] Among them, the collaborative verification judgment factor is: a feature description vector composed of the mutual support relationship between the verification subject relationship and the auxiliary verification strategy;

[0132] Determine the collaborative verification strategy based on the collaborative verification determination factor;

[0133] Among them, when determining the collaborative verification strategy based on the collaborative verification determination factor, the collaborative verification determination factor can be matched with the preset to-be-matched factors in the preset collaborative verification strategy library. The preset collaborative verification strategy library includes one-to-one corresponding to-be-matched factors and collaborative verification strategies. The collaborative verification strategy is: which verification subject is visited first for verification, and which verification subject is visited next for verification, so as to achieve mutual support between the corresponding auxiliary verification strategies;

[0134] Determine the verification route for suspicious areas of offline verification based on the collaborative verification strategy.

[0135] The working principle and beneficial effects of the above technical solution are:

[0136] There are two situations when performing anomaly verification: the first is that connecting to the online node can directly assist the remote sensing target detection manager to verify the anomaly; the second is that offline verification is required.

[0137] During offline verification, multiple suspicious areas require verification, and verification routes to these areas need to be planned. However, efficient verification is not always achieved by simply planning the shortest route through these areas. For example, a route that passes through suspicious areas D, E, and F in sequence is the shortest route. However, the regional verification subjects in suspicious area E (e.g., residents of area E) and the regional verification subjects in suspicious areas D and F (e.g., residents of areas D and F) are related (e.g., residents of area E can simultaneously observe the behavior of residents in areas D and F). Remote sensing images show that the vegetation damage areas A and C are spreading from area E to areas D and F. Therefore, the mutual support relationship of the auxiliary verification strategy requires first verifying that the source of vegetation damage A and C is indeed in suspicious area E, and then determining whether to continue verification in suspicious areas D and F based on the verification results.

[0138] The collaborative verification strategy for the collaborative verification factor constructed based on suspicious area E is as follows: first, visit suspicious area E to verify that the sources of damage A and C are indeed in suspicious area E, then interview residents in area E, and then visit the corresponding verification entities in suspicious areas D and F. The final planned local verification route between suspicious areas E, D, and F is the shortest route that passes through suspicious areas E, D, and F in that order. Similarly, all local verification routes are combined to form a final overall verification route. Verification is then performed based on this overall verification route, greatly improving offline verification efficiency.

[0139] The embodiment of the present invention provides a remote sensing real-time detection system based on improved RT-DETR, such as Figure 7 As shown, including:

[0140] Preprocessing module 1, used to obtain preprocessed remote sensing images;

[0141] The first training module 2 is used to input the remote sensing image into the improved RT-DETR model with the auxiliary algorithm branch closed for preliminary training of the model;

[0142] The second training module 3 is used to start the auxiliary algorithm branch and train;

[0143] The third training module 4 is used to synchronously train the model trunk and the auxiliary algorithm branch after the auxiliary algorithm training is completed;

[0144] Detection module 5 is used to close the auxiliary algorithm branch after the synchronous training is completed and select the optimal weight for remote sensing image target detection;

[0145] When improving the RT-DETR model in the first training module, the 3x3 standard convolution in the last layer of the original ResNet50 backbone network was replaced with a 3x3 adaptive rotation convolution. The model structure was simplified by removing the CCFF module from the Efficient Hybrid Encoder structure and reducing the number of layers in the decoder structure from 6 to 4. An additional auxiliary algorithm branch was set up after the Efficient Hybrid Encoder. The output features of the backbone network and the Efficient Hybrid Encoder were used as input to the branch. The anchor box coordinates and features output by the auxiliary algorithm branch were used to generate auxiliary queries, which were then used together with the basic queries generated by the backbone network for subsequent operations.

[0146] The remote sensing real-time detection system based on the improved RT-DETR also performs the following operations:

[0147] Determine the detection interval between target detection results and associated target detection results of remote sensing image target detection;

[0148] Determining the target type of the first detection target corresponding to the target detection result;

[0149] Determine the expected target detection results based on the evolution model, detection interval, and evolution environment corresponding to the target type;

[0150] Comparing the expected target detection result with the target detection result, and determining a second detection target whose result difference is greater than a preset difference threshold;

[0151] Screening suspicious areas based on the positional relationship and category relationship of the second detection target;

[0152] Determine the auxiliary verification strategy for the suspicious area based on the description of the result difference corresponding to each suspicious area;

[0153] Verify according to the auxiliary verification strategy and output abnormal warning;

[0154] The auxiliary verification strategy for determining the suspicious area according to the result difference description corresponding to each suspicious area includes:

[0155] Traverse each suspicious area in turn, and retrieve the knowledge graph knowledge containing the corresponding entity of the second detection target in the target suspicious area being traversed;

[0156] Determine the model based on the knowledge graph knowledge training auxiliary verification strategy;

[0157] Inputting the result difference description corresponding to the second detection target within the target suspicious area into the auxiliary verification strategy determination model to obtain the auxiliary verification strategy;

[0158] The verification according to the auxiliary verification strategy and output of abnormal warning include:

[0159] Determine the verification type of the auxiliary verification strategy, which includes online verification and offline verification;

[0160] If the verification type is offline verification, determine the verification features of the corresponding suspicious area, which include: the mutual support relationship between the verification subject and the auxiliary verification strategy;

[0161] Construct collaborative verification judgment factors based on verification characteristics;

[0162] Determine the collaborative verification strategy based on the collaborative verification determination factor;

[0163] Determine the verification route for suspicious areas of offline verification based on the collaborative verification strategy.

[0164] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A remote sensing real-time detection method based on improved RT-DETR, characterized in that: include: Obtain preprocessed remote sensing images; The remote sensing images were input into the improved RT-DETR model with the auxiliary algorithm branch closed for preliminary training of the model; Enable auxiliary algorithm branch and train; After the auxiliary algorithm training is completed, the model trunk and auxiliary algorithm branch are trained synchronously; After the synchronous training is completed, the auxiliary algorithm branch is closed and the optimal weight is selected for remote sensing image target detection.

2. A remote sensing real-time detection method based on improved RT-DETR as claimed in claim 1, characterized in that: The improvement steps to improve the RT-DETR model include: Adaptive rotation convolution is used to replace the standard convolution in the backbone network of the RT-DETR-R50 model, and an auxiliary algorithm branch is added after the Efficient Hybrid Encoder in the RT-DETR-R50 model.

3. A remote sensing real-time detection method based on improved RT-DETR as claimed in claim 2, characterized in that: The standard convolution in the backbone network of the RT-DETR-R50 model is replaced by adaptive rotation convolution, including: The 3*3 adaptive rotation convolution is used to replace the last layer of 3*3 standard convolution of the backbone network ResNet50 in the RT-DETR-R50 model.

4. A remote sensing real-time detection method based on improved RT-DETR as claimed in claim 2, characterized in that: Auxiliary algorithm branches include: The auxiliary algorithm corresponding to the auxiliary algorithm branch is a one-to-many matching supervision algorithm.

5. A remote sensing real-time detection method based on improved RT-DETR as claimed in claim 2, characterized in that: The improvement steps for improving the RT-DETR model also include: The CCFF module in the Efficient Hybrid Encoder structure is removed, and the decoder structure is changed from the original 6-layer to a 4-layer decoder.

6. A remote sensing real-time detection method based on improved RT-DETR as claimed in claim 4, characterized in that: Auxiliary algorithm branches also include: According to the output features of the backbone network and the Efficient Hybrid Encoder, the predicted anchor box coordinates are output; Determine the feature level of the target based on the size of the predicted anchor box; Perform RoIAlign operation on the corresponding coordinates of the predicted anchor box coordinates in the feature level to extract the required features; The anchor box coordinates and the required features are combined into an auxiliary query and sent to the Decoder for subsequent operations together with the initialization query generated by the model backbone.

7. A remote sensing real-time detection method based on improved RT-DETR as claimed in claim 1, characterized in that: Also includes: Determine the detection interval between target detection results and associated target detection results of remote sensing image target detection; Determining the target type of the first detection target corresponding to the target detection result; Determine the expected target detection results based on the evolution model, detection interval, and evolution environment corresponding to the target type; Comparing the expected target detection result with the target detection result, and determining a second detection target whose result difference is greater than a preset difference threshold; Screening suspicious areas based on the positional relationship and category relationship of the second detection target; Determine the auxiliary verification strategy for the suspicious area based on the description of the result difference corresponding to each suspicious area; Verify according to the auxiliary verification strategy and output abnormal warning.

8. A remote sensing real-time detection method based on improved RT-DETR as claimed in claim 7, characterized in that: Determine the auxiliary verification strategy for each suspicious area based on the description of the result difference, including: Traverse each suspicious area in turn, and retrieve the knowledge graph knowledge containing the corresponding entity of the second detection target in the target suspicious area being traversed; Determine the model based on the knowledge graph knowledge training auxiliary verification strategy; The result difference description corresponding to the second detection target in the target suspicious area is input into the auxiliary verification strategy determination model to obtain the auxiliary verification strategy.

9. A remote sensing real-time detection method based on improved RT-DETR according to claim 7, characterized in that: Verify and output abnormal warnings based on auxiliary verification strategies, including: Determine the verification type of the auxiliary verification strategy, which includes online verification and offline verification; If the verification type is offline verification, determine the verification features of the corresponding suspicious area, which include: the mutual support relationship between the verification subject and the auxiliary verification strategy; Construct collaborative verification judgment factors based on verification characteristics; Determine the collaborative verification strategy based on the collaborative verification determination factor; Determine the verification route for suspicious areas of offline verification based on the collaborative verification strategy.

10. A remote sensing real-time detection system based on improved RT-DETR, characterized in that: include: A preprocessing module is used to obtain preprocessed remote sensing images; The first training module is used to input remote sensing images into the improved RT-DETR model with the auxiliary algorithm branch closed for preliminary training of the model; The second training module is used to start the auxiliary algorithm branch and train it; The third training module is used to synchronously train the model trunk and auxiliary algorithm branches after the auxiliary algorithm training is completed; The detection module is used to close the auxiliary algorithm branch after the synchronous training is completed and select the optimal weight for remote sensing image target detection.

Images