A remote sensing real-time detection method and system based on improved RT-DETR

Abstract

The application provides a remote sensing real-time detection method and system based on an improved RT-DETR, wherein the method comprises the following steps: acquiring a preprocessed remote sensing image; inputting the remote sensing image into an improved RT-DETR model with a closed auxiliary algorithm branch to perform preliminary training of the model; opening the auxiliary algorithm branch and training; after the auxiliary algorithm training is completed, synchronously training the model backbone and the auxiliary algorithm branch; after the synchronous training is completed, closing the auxiliary algorithm branch, and selecting an optimal weight to perform remote sensing image target detection. The remote sensing real-time detection method and system based on the improved RT-DETR improve the RT-DETR and rely on an auxiliary algorithm branch and a training strategy to help the improved model to more accurately locate remote sensing detection targets, improve the speed and precision of model target detection, reduce the algorithm power consumption, and further improve the reliability of remote sensing image target detection.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a remote sensing real-time detection method and system based on an improved RT-DETR. Background Technology

[0002] Remote sensing image target detection technology plays a crucial role in numerous fields, including military reconnaissance, environmental monitoring, and urban planning. In recent years, with the continuous advancement of computer vision and artificial intelligence technologies, especially the rapid development of deep learning, image processing and target detection have ushered in unprecedented opportunities, and remote sensing image target detection technology has made significant progress. Deep learning technology, with its powerful feature learning and pattern recognition capabilities, can automatically learn rich feature representations from large amounts of remote sensing image data, thereby achieving efficient target detection.

[0003] The YOLO series has become a popular framework for real-time remote sensing due to its excellent balance between high accuracy and high speed. Following the emergence of the DETR framework, many excellent algorithms have also emerged, such as RT-DETR and D-FINE.

[0004] However, the models based on the current DETR framework algorithm are relatively large, resulting in high computational overhead and slow target detection speed for edge deployment tasks.

[0005] In view of this, there is an urgent need for a remote sensing real-time detection method and system based on improved RT-DETR to at least address the above-mentioned shortcomings. Summary of the Invention

[0006] One of the objectives of this invention is to provide a remote sensing real-time detection method and system based on an improved RT-DETR. The improved RT-DETR and the reliance on auxiliary algorithm branches and training strategies help the improved model to more accurately locate remote sensing targets, thereby improving the speed and accuracy of target detection, reducing computational overhead, and further enhancing the reliability of target detection in remote sensing images.

[0007] This invention provides a remote sensing real-time detection method based on an improved RT-DETR, comprising:

[0008] Acquire preprocessed remote sensing images;

[0009] The remote sensing image was input into the improved RT-DETR model with the auxiliary algorithm branch turned off for initial training of the model;

[0010] Start the auxiliary algorithm branch and train it;

[0011] After the auxiliary algorithm is trained, the main model and the auxiliary algorithm branches are trained simultaneously.

[0012] After synchronous training is completed, the auxiliary algorithm branch is closed, and the optimal weights are selected for target detection in remote sensing images.

[0013] Preferably, the improvement steps for the improved RT-DETR model include:

[0014] The standard convolutions in the backbone network of the RT-DETR-R50 model are replaced with adaptive rotational convolutions, and an auxiliary algorithm branch is added after the Efficient Hybrid Encoder in the RT-DETR-R50 model.

[0015] Preferably, adaptive rotational convolutions are used to replace standard convolutions in the backbone network of the RT-DETR-R50 model, including:

[0016] Replace the last 3x3 standard convolution of the ResNet50 backbone network in the RT-DETR-R50 model with a 3x3 adaptive rotational convolution.

[0017] Preferably, the auxiliary algorithm branches include:

[0018] The auxiliary algorithm branch corresponds to a one-to-many matching supervised algorithm.

[0019] Preferably, the improvement steps for the improved RT-DETR model further include:

[0020] Remove the CCFF module from the Efficient Hybrid Encoder structure and modify the Decoder structure from the original 6 layers to a 4-layer decoder.

[0021] Preferably, the auxiliary algorithm branch also includes:

[0022] Based on the output features of the backbone network and the Efficient Hybrid Encoder, the predicted anchor box coordinates are output;

[0023] Determine the feature level of the target based on the predicted anchor frame size;

[0024] Perform the RoIAlign operation on the coordinates corresponding to the predicted anchor box coordinates in the feature level to extract the required features;

[0025] The anchor box coordinates and the required features are merged into an auxiliary query and sent to the Decoder, where they are processed together with the initial query generated by the model backbone.

[0026] The remote sensing real-time detection method based on improved RT-DETR provided in this embodiment of the invention further includes:

[0027] Determine the detection interval between the target detection results and the associated target detection results in the remote sensing image;

[0028] Determine the target type corresponding to the first detection target based on the target detection result;

[0029] Based on the evolutionary model, detection interval, and evolutionary environment corresponding to the target type, determine the expected target detection results;

[0030] By comparing the expected target detection results with the target detection results, a second detection target whose result difference is greater than a preset difference threshold is identified.

[0031] Based on the location and category relationships of the second detection target, suspicious areas are screened;

[0032] Determine the auxiliary verification strategy for each suspicious area based on the description of the result differences corresponding to each suspicious area;

[0033] Verification is performed based on the auxiliary verification strategy, and anomaly warnings are output.

[0034] Preferably, the auxiliary verification strategy for determining suspicious regions based on the result difference descriptions corresponding to each suspicious region includes:

[0035] Each suspicious region is traversed sequentially, and the knowledge graph knowledge of the corresponding entity of the second detected target within the suspicious region being traversed is retrieved.

[0036] The model is determined based on a knowledge graph-based knowledge training and verification strategy.

[0037] The description of the difference in results corresponding to the second detected target within the suspicious target area is input into the auxiliary verification strategy to determine the model, thus obtaining the auxiliary verification strategy.

[0038] Preferably, verification is performed according to the auxiliary verification strategy and anomaly warnings are output, including:

[0039] Determine the verification type for the auxiliary verification strategy. Verification types include: online verification and offline verification.

[0040] If the verification type is offline verification, determine the verification characteristics of the corresponding suspicious areas. The verification characteristics include: the relationship between the verification subjects and the mutual support relationship of the auxiliary verification strategies.

[0041] Construct collaborative verification decision factors based on verification features;

[0042] Determine the collaborative verification strategy based on the collaborative verification decision factor;

[0043] The verification route for suspicious areas in offline verification is determined based on the collaborative verification strategy.

[0044] This invention provides a remote sensing real-time detection system based on an improved RT-DETR, comprising:

[0045] The preprocessing module is used to acquire preprocessed remote sensing images;

[0046] The first training module is used to input remote sensing images into an improved RT-DETR model with the auxiliary algorithm branch turned off for initial training of the model;

[0047] The second training module is used to enable and train the auxiliary algorithm branch;

[0048] The third training module is used to simultaneously train the model backbone and auxiliary algorithm branches after the auxiliary algorithm training is completed.

[0049] The detection module is used to close the auxiliary algorithm branch and select the optimal weights for target detection in remote sensing images after synchronous training is completed.

[0050] The beneficial effects of this invention are as follows:

[0051] This invention improves RT-DETR and relies on auxiliary algorithm branches and training strategies to help the model more accurately locate remote sensing targets, thereby improving the speed and accuracy of target detection, reducing computational overhead, and further enhancing the reliability of target detection in remote sensing images.

[0052] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in this application.

[0053] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0054] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0055] Figure 1 This is a schematic diagram of a remote sensing real-time detection method based on an improved RT-DETR in an embodiment of the present invention;

[0056] Figure 2 This is a diagram illustrating the adaptive rotational convolution function in an embodiment of the present invention;

[0057] Figure 3 This is a branch illustration in an embodiment of the present invention;

[0058] Figure 4This is a flowchart of the auxiliary algorithm in an embodiment of the present invention;

[0059] Figure 5 This is a diagram showing the proportions of each structure in the model in this embodiment of the invention;

[0060] Figure 6 This is a comparison chart of the quality of basic queries and auxiliary queries in an embodiment of the present invention;

[0061] Figure 7 This is a schematic diagram of a remote sensing real-time detection system based on an improved RT-DETR according to an embodiment of the present invention. Detailed Implementation

[0062] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0063] This invention provides a remote sensing real-time detection method based on an improved RT-DETR, such as... Figure 1 As shown, it includes:

[0064] Step 1: Acquire the preprocessed remote sensing image;

[0065] The preprocessed remote sensing images are those obtained by uniformly setting the image size in the public remote sensing dataset DIOR to 800×800 pixels and dividing them into training, validation, and test sets in a 1:1:2 ratio.

[0066] Step 2: Input the remote sensing image into the improved RT-DETR model with the auxiliary algorithm branch turned off for initial model training;

[0067] During the initial training, the main body of the model is trained with the auxiliary algorithm branch closed until convergence, and the optimal weights are retained.

[0068] In the improved RT-DETR model, a 3x3 adaptive rotation convolution was used to replace the standard 3x3 convolution in the last layer of the original backbone network ResNet50. The adaptive rotation convolution function is demonstrated as follows: Figure 2 As shown; the model structure is simplified by removing the CCFF module from the Efficient Hybrid Encoder structure and reducing the number of decoder layers from 6 to 4; an additional auxiliary algorithm branch is added after the Efficient Hybrid Encoder, as shown in the diagram. Figure 3As shown, the output features of the backbone network and the Efficient Hybrid Encoder serve as the input to the branches. The anchor box coordinates and features output by the auxiliary algorithm branches are used to generate auxiliary queries, which, along with the basic queries generated by the backbone, are then fed into subsequent operations. The workflow of the auxiliary algorithm is as follows: Figure 4 As shown;

[0069] Step 3: Start the auxiliary algorithm branch and train it;

[0070] When the auxiliary algorithm branch is enabled and training is performed, the main body of the model is frozen on the basis of the initial training of the model, the auxiliary algorithm branch is enabled, FCOS is used as a one-to-many matching supervision auxiliary algorithm, and the training is performed until convergence, while retaining the optimal weights.

[0071] Step 4: After the auxiliary algorithm is trained, train the main model and the auxiliary algorithm branch simultaneously;

[0072] During synchronous training, the auxiliary algorithm provides additional high-quality auxiliary queries to the backbone, enriches the positive sample supervision of the encoder, and provides the decoder with more accurate initialization queries, retaining the optimal weights until convergence.

[0073] Step 5: After synchronous training is completed, close the auxiliary algorithm branch and select the optimal weights for remote sensing image target detection.

[0074] Among them, selecting the optimal weight for remote sensing image target detection refers to the process of using the trained optimal weight to perform target detection on the input remote sensing image that needs target detection.

[0075] The working principle and beneficial effects of the above technical solution are as follows:

[0076] This invention introduces adaptive rotational convolution, enabling the model's backbone network to possess feature rotation invariance, better addressing the challenge of variable instance orientations in remote sensing. Furthermore, the simplified model structure improves real-time performance and makes it more lightweight. A better query generation method effectively enhances the model's localization capabilities, significantly improving accuracy, without introducing any additional computational overhead or latency during target detection.

[0077] Specifically, the experimental platform for measuring target detection latency in this embodiment is: CPU: Intel(R) Xeon(R) Gold6230R CPU@2.10GHz, GPU: a single NVIDIA GeForce RTX 3090.

[0078] After training under the above conditions, the algorithm's performance is shown in Table 1:

[0079] Table 1 Performance statistics of different algorithms

[0080]

[0081] In summary, compared with the RT-DETR-R50 benchmark algorithm, this invention demonstrates superior performance in terms of accuracy, speed, number of parameters, and computational cost. The AP of the model without model structure simplification... 50:95 The accuracy achieved is outstanding.

[0082] The results of the simplified expansion ablation experiment of the model are shown in Table 2:

[0083] Table 2 Simplified Ablation Experiment Table

[0084]

[0085] Figure 5 The impact of each module of the model on lightweight and real-time performance is presented. As can be seen from the experimental data, compared with the original model, the simplified model consumes 62% of the computational cost of the original model, achieving a 1.38-fold improvement in prediction speed.

[0086] The results of the ablation experiments on the better query generation method are shown in Table 3:

[0087] Table 3 Ablation experiment table with different auxiliary algorithms for query assistance

[0088]

[0089] Three auxiliary algorithms were also tested: ATSS, Faster R-CNN, and FCOS. Experiments showed that adding auxiliary algorithms improved AP (Ability to Process). 50:95 The significant improvement indicates that the generated high-quality queries enable the model to more accurately locate the target. Figure 6 This demonstrates a comparison of the location quality between auxiliary queries and basic queries.

[0090] Table 4. Complete Ablation Experiment Table

[0091]

[0092] The complete ablation experiments are shown in Table 4. In these experiments, it is clear that the enhanced query generation method effectively improves the model's accuracy without increasing computational cost or speed, especially in improving accuracy for small targets. Through observation...

[0093]

[0094] The significant comparison between Variant C and the baseline demonstrates the importance of feature rotation invariance for target detection tasks with variable instance orientations, such as remote sensing.

[0095] Meanwhile, the effectiveness of the algorithm was tested on the NWPU VHR-10, with the image size set to 800*800 pixels and the dataset divided into a 6:2:2 ratio. The dataset classification is shown in Table 5.

[0096] Table 5. Dataset Classification Table

[0097]

[0098] Table 6 shows the experimental data on the effectiveness of the algorithm on a small dataset:

[0099] Table 6. Algorithm effectiveness experiment on small datasets.

[0100]

[0101] As can be seen, this invention also achieves excellent performance on small datasets, demonstrating its good generalization ability. Therefore, this invention is not limited to large-scale datasets; even with limited data, it can still effectively learn key features, thereby achieving high-precision detection results.

[0102] This invention improves RT-DETR and relies on auxiliary algorithm branches and training strategies to help the model more accurately locate remote sensing targets, thereby improving the speed and accuracy of target detection, reducing computational overhead, and further enhancing the reliability of target detection in remote sensing images.

[0103] This invention provides a remote sensing real-time detection method based on an improved RT-DETR, and further includes:

[0104] Determine the detection interval between the target detection results and the associated target detection results in the remote sensing image;

[0105] The target detection result is: the local remote sensing image of the target (the first detected target) detected in the currently input remote sensing image; the associated target detection result is: the local remote sensing image of the first detected target that was most recently identified at the current time.

[0106] Determine the target type corresponding to the first detection target based on the target detection result;

[0107] Among them, the target type is: the type identifier of the first detection target;

[0108] Based on the evolutionary model, detection interval, and evolutionary environment corresponding to the target type, determine the expected target detection results;

[0109] Among them, the evolution model is: a model of how the target of a target type changes with its environment and time; the evolution environment is: the environmental changes of the first detected target within the detection interval; the expected target detection result is: the local remote sensing image of the first detected target at the current moment, output by the evolution model.

[0110] By comparing the expected target detection results with the target detection results, a second detection target whose result difference is greater than a preset difference threshold is identified.

[0111] The result difference is: the image difference between the local remote sensing image of the target (first detection target) detected in the currently input remote sensing image and the expected local remote sensing image of the first detection target at the current moment; the preset difference threshold is set manually; the second detection target is: the first detection target whose result difference is greater than the difference threshold;

[0112] Based on the location and category relationships of the second detection target, suspicious areas are screened;

[0113] The location relationship is the geographical distance between the second detection targets; the category relationship is the category association between the second detection targets; when screening suspicious areas, the second detection target clusters that meet the screening conditions are determined, including: geographical distance less than the distance threshold and category association greater than the association threshold; the area corresponding to the smallest enclosing circle containing the corresponding second detection target cluster is regarded as a suspicious area; the distance threshold and association threshold are both preset manually;

[0114] Determine the auxiliary verification strategy for each suspicious area based on the description of the result differences corresponding to each suspicious area;

[0115] The result difference description is: the semantic recognition result of the result difference; when determining the auxiliary verification strategy, each suspicious region is traversed in turn, and the suspicious region being traversed is taken as the target suspicious region. The knowledge graph knowledge containing the corresponding entity of the second detected target in the target suspicious region is retrieved. The auxiliary verification strategy determination model is trained using the knowledge graph knowledge. The result difference description corresponding to the second detected target in the target suspicious region is input into the auxiliary verification strategy determination model to obtain the output auxiliary verification strategy. The auxiliary verification strategy is: a strategy to help verify what causes the result difference;

[0116] Verification is performed based on the auxiliary verification strategy, and anomaly warnings are output.

[0117] The working principle and beneficial effects of the above technical solution are as follows:

[0118] Remote sensing targets evolve differently over time depending on their type. For example, buildings are relatively fixed, while vegetation changes with climate. Therefore, adaptive anomaly warnings are needed based on the type of target and the degree of change.

[0119] Specifically, for example: the target detection result is the local remote sensing image of vegetation A at the current time t1, and the associated target detection result is the local remote sensing image of vegetation A at time t2, the time when the remote sensing target detection of vegetation A was last performed at time t1. The target category is identified as "vegetation A". The evolutionary model is a model of the changes of vegetation A over time and in the environment. The climate information (evolutionary environment) of vegetation A during the detection interval, the detection interval, and the local remote sensing image of vegetation A at time t2 are input into the evolutionary model to obtain the expected local remote sensing image of vegetation A at time t1. By comparing the expected local remote sensing image of vegetation A at time t1 (expected target detection result) with the actual local remote sensing image (target detection result), if the image difference is greater than, for example, 15%, then vegetation A is selected as the second detection target.

[0120] Since there are various types of target detection, there may be situations where there is more than one second detection target. For example, based on the process of determining the second detection target as described above, vegetation B and C can also be second detection targets.

[0121] Meanwhile, the anomaly of the second detection target may not be an independent event. For example, if a certain influencing factor occurs in the area, the vegetation A and C in the area will be affected by the influencing factor. In order to save verification resources in the future, when determining the suspicious areas to be investigated, the suspicious areas can be determined based on the location and category relationships of the second detection targets. The number of suspicious areas can be 0, 1 or more.

[0122] When identifying suspicious areas, it is necessary to connect with other nodes to further verify the reasons for the differences in results. Therefore, an auxiliary verification strategy for suspicious areas is determined based on the description of the differences in results corresponding to each suspicious area. When determining the auxiliary verification strategy, knowledge graph knowledge containing entities of the second detection target corresponding to the target area being traversed is introduced, such as knowledge graph knowledge containing entities "vegetation A" and "vegetation C". The auxiliary verification strategy is trained based on the corresponding knowledge graph knowledge to determine the model. The model is determined based on the input description of the differences in results, such as "the areas of vegetation A and vegetation C are decreasing rapidly at an unconventional rate" and "vegetation A and vegetation C are close to residential areas" as input into the auxiliary verification strategy. The model outputs the auxiliary verification strategy "retrieve monitoring of vegetation areas to check whether there is illegal logging".

[0123] Finally, verification is performed according to the auxiliary verification strategy. If any anomalies are found, such as unauthorized logging by personnel, an anomaly warning is output.

[0124] This invention introduces an evolutionary model, detection interval, and evolutionary environment to determine the expected target detection result. By comparing the expected target detection result with the target detection result, a second detection target requiring anomaly verification is determined. The positional and category relationships of the second detection target are introduced to determine suspicious regions, saving subsequent verification resources. Furthermore, a knowledge graph containing the corresponding entities of the second detection target is used to train an auxiliary verification strategy determination model. Based on the result difference description and auxiliary verification strategy determination model corresponding to each suspicious region, the corresponding auxiliary verification strategy for the suspicious region is determined, improving the accuracy and efficiency of the auxiliary verification strategy determination.

[0125] In one embodiment, verification is performed according to an auxiliary verification strategy, and an anomaly warning is output, including:

[0126] Determine the verification type for the auxiliary verification strategy. Verification types include: online verification and offline verification.

[0127] Online verification refers to verification by connecting to online network nodes, such as monitoring nodes in vegetated areas; offline verification refers to on-site verification in the relevant suspicious areas, such as when no monitoring is set up in the vegetated areas, an on-site visit is required.

[0128] If the verification type is offline verification, determine the verification characteristics of the corresponding suspicious areas. The verification characteristics include: the relationship between the verification subjects and the mutual support relationship of the auxiliary verification strategies.

[0129] Among them, the relationship between the verification subjects is: the association between the verification subjects; the mutual support relationship of the auxiliary verification strategies refers to whether it is necessary to verify one auxiliary verification result before verifying another auxiliary verification result.

[0130] Construct collaborative verification decision factors based on verification features;

[0131] Among them, the collaborative verification judgment factor is: a feature description vector composed of the mutual support relationship between the verification subject relationship and the auxiliary verification strategy;

[0132] Determine the collaborative verification strategy based on the collaborative verification decision factor;

[0133] Specifically, when determining the collaborative verification strategy based on the collaborative verification judgment factor, the collaborative verification judgment factor is matched with the preset matching factors in the preset collaborative verification strategy library. The preset collaborative verification strategy library includes one-to-one matching factors and collaborative verification strategies. The collaborative verification strategy is: which verification subject to access first and which verification subject to access next, so as to achieve mutual support between the corresponding auxiliary verification strategies.

[0134] The verification route for suspicious areas in offline verification is determined based on the collaborative verification strategy.

[0135] The working principle and beneficial effects of the above technical solution are as follows:

[0136] There are two scenarios when performing anomaly verification: First, connecting to online nodes can directly assist remote sensing target detection managers in verifying anomalies; second, offline verification is required.

[0137] During offline verification, multiple suspicious areas need to be verified, requiring the planning of verification routes to these areas. However, simply planning the shortest route through these suspicious areas is not enough for efficient verification. For example, the shortest route might pass through suspicious areas D, E, and F in sequence, but the verification subjects in suspicious area E (e.g., residents of area E) and suspicious areas D and F (e.g., residents of areas D and F) are related (e.g., residents of area E can simultaneously observe the behavior of residents of areas D and F). Furthermore, remote sensing images show that vegetation destruction in areas A and C spreads from area E to areas D and F. Therefore, the supporting verification strategy requires first verifying from suspicious area E that the source of vegetation destruction in areas A and C is indeed in area E, and then determining whether to continue verification to suspicious areas D and F based on the verification results.

[0138] Therefore, the collaborative verification strategy based on the collaborative verification judgment factor constructed in suspicious area E is as follows: First, go to suspicious area E to check whether the sources of damage A and C are indeed in suspicious area E; then conduct interviews with residents in area E; then go to suspicious areas D and F to interview the corresponding verification entities. The final planned local verification route between suspicious areas E, D, and F is the shortest route passing through suspicious areas E, D, and F in sequence. Similarly, by performing a final overall route planning on all local verification routes, a total verification route is obtained. Verification is then carried out based on the total verification route, which greatly improves the efficiency of offline verification.

[0139] This invention provides a remote sensing real-time detection system based on an improved RT-DETR, such as... Figure 7 As shown, it includes:

[0140] Preprocessing module 1 is used to acquire preprocessed remote sensing images;

[0141] The first training module 2 is used to input remote sensing images into an improved RT-DETR model with the auxiliary algorithm branch turned off for initial training of the model;

[0142] The second training module 3 is used to enable and train the auxiliary algorithm branch;

[0143] The third training module 4 is used to simultaneously train the model backbone and auxiliary algorithm branches after the auxiliary algorithm training is completed.

[0144] Detection module 5 is used to close the auxiliary algorithm branch and select the optimal weights for target detection in remote sensing images after synchronous training is completed;

[0145] In the first training module, when improving the RT-DETR model, a 3*3 adaptive rotational convolution was used to replace the last layer of the 3*3 standard convolution in the original backbone network ResNet50. The model structure was simplified by removing the CCFF module in the Efficient Hybrid Encoder structure and reducing the number of layers in the Decoder structure from 6 layers to 4 layers. An additional auxiliary algorithm branch was set after the Efficient Hybrid Encoder. The output features of the backbone network and the Efficient Hybrid Encoder were used as the input of the branch. The anchor box coordinates and features output by the auxiliary algorithm branch were used to generate an auxiliary query, which was then entered into subsequent operations along with the basic query generated by the backbone.

[0146] The remote sensing real-time detection system based on the improved RT-DETR also performs the following operations:

[0147] Determine the detection interval between the target detection results and the associated target detection results in the remote sensing image;

[0148] Determine the target type corresponding to the first detection target based on the target detection result;

[0149] Based on the evolutionary model, detection interval, and evolutionary environment corresponding to the target type, determine the expected target detection results;

[0150] By comparing the expected target detection results with the target detection results, a second detection target whose result difference is greater than a preset difference threshold is identified.

[0151] Based on the location and category relationships of the second detection target, suspicious areas are screened;

[0152] Determine the auxiliary verification strategy for each suspicious area based on the description of the result differences corresponding to each suspicious area;

[0153] Perform verification based on the auxiliary verification strategy and output anomaly warnings;

[0154] The auxiliary verification strategy for determining suspicious regions based on the result difference description corresponding to each suspicious region includes:

[0155] Each suspicious region is traversed sequentially, and the knowledge graph knowledge of the corresponding entity of the second detected target within the suspicious region being traversed is retrieved.

[0156] The model is determined based on a knowledge graph-based knowledge training and verification strategy.

[0157] Input the description of the result difference corresponding to the second detected target within the suspicious target area into the auxiliary verification strategy to determine the model and obtain the auxiliary verification strategy;

[0158] The step of verifying according to the auxiliary verification strategy and outputting anomaly warnings includes:

[0159] Determine the verification type for the auxiliary verification strategy. Verification types include: online verification and offline verification.

[0160] If the verification type is offline verification, determine the verification characteristics of the corresponding suspicious areas. The verification characteristics include: the relationship between the verification subjects and the mutual support relationship of the auxiliary verification strategies.

[0161] Construct collaborative verification decision factors based on verification features;

[0162] Determine the collaborative verification strategy based on the collaborative verification decision factor;

[0163] The verification route for suspicious areas in offline verification is determined based on the collaborative verification strategy.

[0164] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A remote sensing real-time detection method based on improved RT-DETR, characterized in that, include: Acquire preprocessed remote sensing images; The remote sensing image was input into the improved RT-DETR model with the auxiliary algorithm branch turned off for initial training of the model; Start the auxiliary algorithm branch and train it; After the auxiliary algorithm is trained, the main model and the auxiliary algorithm branches are trained simultaneously. After synchronous training is completed, the auxiliary algorithm branch is closed, and the optimal weights are selected for target detection in remote sensing images. The improvement steps for the RT-DETR model include: The standard convolutions in the backbone network of the RT-DETR-R50 model are replaced with adaptive rotational convolutions, and an auxiliary algorithm branch is added after the Efficient Hybrid Encoder in the RT-DETR-R50 model. The auxiliary algorithm branches include: The auxiliary algorithm branch corresponds to a one-to-many matching supervised algorithm; The auxiliary algorithm branch also includes: Based on the output features of the backbone network and the Efficient Hybrid Encoder, the predicted anchor box coordinates are output; Determine the feature level of the target based on the predicted anchor frame size; Perform the RoIAlign operation on the coordinates corresponding to the predicted anchor box coordinates in the feature level to extract the required features; The anchor box coordinates and the required features are merged into an auxiliary query and sent to the Decoder, where they are processed together with the initial query generated by the model backbone.

2. The remote sensing real-time detection method based on improved RT-DETR as described in claim 1, characterized in that, Replacing the standard convolutions in the backbone network of the RT-DETR-R50 model with adaptive rotational convolutions includes: Replace the last 3x3 standard convolution of the ResNet50 backbone network in the RT-DETR-R50 model with a 3x3 adaptive rotational convolution.

3. The remote sensing real-time detection method based on improved RT-DETR as described in claim 1, characterized in that, The improvement steps for the RT-DETR model also include: Remove the CCFF module from the Efficient Hybrid Encoder structure and modify the Decoder structure from the original 6 layers to a 4-layer decoder.

4. The remote sensing real-time detection method based on improved RT-DETR as described in claim 1, characterized in that, Also includes: Determine the detection interval between the target detection results and the associated target detection results in the remote sensing image; Determine the target type corresponding to the first detection target based on the target detection result; Based on the evolutionary model, detection interval, and evolutionary environment corresponding to the target type, determine the expected target detection results; By comparing the expected target detection results with the target detection results, a second detection target whose result difference is greater than a preset difference threshold is identified. Based on the location and category relationships of the second detection target, suspicious areas are screened; Determine the auxiliary verification strategy for each suspicious area based on the description of the result differences corresponding to each suspicious area; Verification is performed based on the auxiliary verification strategy, and anomaly warnings are output.

5. The remote sensing real-time detection method based on improved RT-DETR as described in claim 4, characterized in that, Based on the description of the result differences corresponding to each suspicious region, an auxiliary verification strategy for the suspicious region is determined, including: Each suspicious region is traversed sequentially, and the knowledge graph knowledge of the corresponding entity of the second detected target within the suspicious region being traversed is retrieved. The model is determined based on a knowledge graph-based knowledge training and verification strategy. The description of the difference in results corresponding to the second detected target within the suspicious target area is input into the auxiliary verification strategy to determine the model, thus obtaining the auxiliary verification strategy.

6. The remote sensing real-time detection method based on improved RT-DETR as described in claim 4, characterized in that, Verification is performed based on the auxiliary verification strategy, and anomaly warnings are output, including: Determine the verification type for the auxiliary verification strategy. Verification types include: online verification and offline verification. If the verification type is offline verification, determine the verification characteristics of the corresponding suspicious areas. The verification characteristics include: the relationship between the verification subjects and the mutual support relationship of the auxiliary verification strategies. Construct collaborative verification decision factors based on verification features; Determine the collaborative verification strategy based on the collaborative verification decision factor; The verification route for suspicious areas in offline verification is determined based on the collaborative verification strategy.

7. A remote sensing real-time detection system based on an improved RT-DETR, characterized in that, include: The preprocessing module is used to acquire preprocessed remote sensing images; The first training module is used to input remote sensing images into an improved RT-DETR model with the auxiliary algorithm branch turned off for initial training of the model; The second training module is used to enable and train the auxiliary algorithm branch; The third training module is used to simultaneously train the model backbone and auxiliary algorithm branches after the auxiliary algorithm training is completed. The detection module is used to close the auxiliary algorithm branch and select the optimal weights for target detection in remote sensing images after synchronous training is completed. The improvement steps for the RT-DETR model include: The standard convolutions in the backbone network of the RT-DETR-R50 model are replaced with adaptive rotational convolutions, and an auxiliary algorithm branch is added after the Efficient Hybrid Encoder in the RT-DETR-R50 model. The auxiliary algorithm branches include: The auxiliary algorithm branch corresponds to a one-to-many matching supervised algorithm; The auxiliary algorithm branch also includes: Based on the output features of the backbone network and the Efficient Hybrid Encoder, the predicted anchor box coordinates are output; Determine the feature level of the target based on the predicted anchor frame size; Perform the RoIAlign operation on the coordinates corresponding to the predicted anchor box coordinates in the feature level to extract the required features; The anchor box coordinates and the required features are merged into an auxiliary query and sent to the Decoder, where they are processed together with the initial query generated by the model backbone.

Images