An aircraft change detection method based on deep learning and image registration algorithm

Abstract

The application relates to an airplane change detection method based on a deep learning and an image registration algorithm, and the method comprises the following steps: acquiring a first remote sensing image and a second remote sensing image of the same region at different time points; performing airport detection on the first remote sensing image and the second remote sensing image to obtain an image of a first airport region in the first remote sensing image, a first offset of the first airport region, an image of a second airport region in the second remote sensing image and a second offset of the second airport region; performing image registration on the image of the first airport region and the image of the second airport region according to a surf image registration algorithm to obtain a registration image; performing airplane detection on the registration image to obtain an airplane target; performing change detection on the result obtained by the airplane detection; and restoring a target frame corresponding to the airplane target to the first remote sensing image and the second remote sensing image. The application can solve the problems of low detection precision, slow detection process running speed and large hardware resource occupation in the prior art.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing and relates to an aircraft change detection method based on deep learning and image registration algorithms. Background Art

[0002] With the rapid development of space science and technology, my country has launched a series of civilian satellites, including the Gaofen, Ziyuan, and Gaojing series, equipped with advanced high-resolution observation capabilities. This has significantly enhanced the acquisition of high-resolution remote sensing imagery, and the volume of high-resolution remote sensing airport and aircraft imagery continues to grow. As a crucial component of national defense, airborne military forces, through dynamic monitoring of the status of military airports, have important applications in both military command and national defense. Remote sensing image change detection primarily analyzes remote sensing images of the same area at different times to identify differences between them. Using related techniques, it further identifies changes in ground feature status information. Typically, targets of interest include aircraft, ships, oil tanks, and nuclear test waste. For example, using continuous, dynamic airport observation data for change detection facilitates timely monitoring of target changes and identifying trends. This research has important implications for battlefield command and decision-making, as well as civil airport scheduling.

[0003] Currently, change detection can be categorized into three levels: pixel-level, feature-level, and object-level. Pixel-level change detection detects changes by calculating the grayscale values ​​of corresponding pixels in two images. It preserves as much original information as possible and includes details not available at other levels. However, it suffers from low efficiency, fails to consider variations in spatial features, and has poor anti-interference capabilities (due to natural factors such as solar angle and surface humidity). Feature-level change detection uses algorithms to extract feature information (edges, shapes, contours, and textures) from the original image, then performs comprehensive analysis and change detection on this information. It offers high efficiency, greater reliability and accuracy in determining feature attributes, and reduces the impact of external factors on the results. However, the feature extraction process can result in partial information loss, making it difficult to provide detailed information. It relies heavily on feature extraction results, which can be challenging. Object-level change detection primarily detects specific objects (such as houses, airplanes, and other well-defined objects). It builds on image understanding and recognition and is a high-level analysis method based on object models. It meets user needs, and the detection results can be directly applied. However, its target extraction is somewhat difficult.

[0004] In the published paper "Aircraft target change detection method under knowledge driving", Shi Shengwen et al. proposed an aircraft change detection method. For high-resolution optical remote sensing images, a knowledge-driven airport area aircraft target change detection idea and implementation method are proposed. First, according to the geographical location knowledge information of the airport, the spatial mask image of the airport is established, and the candidate area of the aircraft target to be detected is obtained on this basis. Then, the spatial position registration of the input image is carried out in combination with the control point knowledge information of the target area. Then, using the knowledge that the change of the aircraft target will cause the texture of the region to change significantly, the texture features of the target region are extracted for change detection, and the weak texture is removed and the edge is suppressed. Finally, the mathematical morphological operation method is used to remove part of the isolated points to obtain the final detection result. This method has three shortcomings. First, he carries out spatial position registration on the whole input image. The remote sensing image is large in size and complex in background, and full-image registration requires a large amount of time. Second, the texture information is used for change detection. Due to the complex background of remote sensing image, the background texture change caused by interference factors will cause a large number of false alarms in the detection result. The use of texture information often cannot get the best detection effect. Third, directly processing the whole remote sensing image will consume a large amount of hardware resources. SUMMARY

[0005] In order to solve the above problems existing in the prior art, the present application provides an aircraft change detection method based on deep learning and image registration algorithm. The technical problems to be solved by the present application are realized by the following technical scheme:

[0006] The embodiment of the present application provides an aircraft change detection method based on deep learning and image registration algorithm, comprising the following steps:

[0007] S1, acquiring a first remote sensing image and a second remote sensing image of the same area at different times;

[0008] S2, detecting the first remote sensing image and the second remote sensing image to obtain the image of the first airport area in the first remote sensing image, the first offset of the first airport area in the first remote sensing image, the image of the second airport area in the second remote sensing image, and the second offset of the second airport area in the second remote sensing image;

[0009] S3, image registration of the image of the first airport area and the image of the second airport area according to the surf image registration algorithm to obtain a registration image;

[0010] S4, performing aircraft detection on the registration image to obtain a first aircraft target and a second aircraft target, the first aircraft target comprising: an aircraft category in the image of the first airport area and coordinates of an aircraft target frame in the image of the first airport area; and the second aircraft target comprising: an aircraft category in the image of the second airport area and coordinates of an aircraft target frame in the image of the second airport area;

[0011] S5, performing change detection on the first aircraft target and the second aircraft target according to IOU to obtain a state of the aircraft target, the state of the aircraft target comprising: a newly added aircraft target, an unchanged aircraft target and a reduced aircraft target;

[0012] S6, restoring the coordinates of the aircraft target frame in the image of the first airport area to the first remote sensing image, and restoring the coordinates of the aircraft target frame in the image of the second airport area to the second remote sensing image.

[0013] In an embodiment of the present application, step S2 comprises:

[0014] Step S2.1, scaling the first remote sensing image into a first scaled image according to a preset scaling ratio, and scaling the second remote sensing image into a second scaled image according to the preset scaling ratio;

[0015] Step S2.2, performing airport detection on the first scaled image and the second scaled image according to an airport detection model to obtain an airport area in the first scaled image and an airport area in the second scaled image;

[0016] Step S2.3, mapping the airport area in the first scaled image back to the first remote sensing image according to the preset scaling ratio, cutting out the image of the mapped airport area in the first remote sensing image to obtain the image of the first airport area, and recording a first offset of the first airport area in the first remote sensing image;

[0017] Step S2.4, mapping the airport area in the second scaled image back to the second remote sensing image according to the preset scaling ratio, cutting out the image of the mapped airport area in the second remote sensing image to obtain the image of the second airport area, and recording a second offset of the second airport area in the second remote sensing image.

[0018] In an embodiment of the present application, before step S2, the method further comprises:

[0019] Obtaining a first training sample of a yolov5 model as an airport data set;

[0020] An airport.yaml file is newly created using the yolov5s6.pt file as the initialization weight file of the first initial model, wherein the nc of the airport.yaml file is set to 1, the names is set to airport, and the path parameter is set to the position corresponding to the airport data set;

[0021] The first initial model is trained to obtain the airport detection model, wherein the size of each first training sample is 1280*1280, the multi-scale training mode is enabled during training, the Mosica data enhancement is enabled, the adaptive anchor frame calculation is enabled, and 300 epochs are iterated.

[0022] In an embodiment of the present application, step S3 comprises:

[0023] Step S3.1, scaling the image of the first airport region into a first airport region scaled image according to the preset scaling ratio, and scaling the image of the second airport region into a second airport region scaled image according to the preset scaling ratio;

[0024] Step S3.2, extracting first Surf features of the first airport region scaled image and second Surf features of the second airport region scaled image;

[0025] Step S3.3, matching the first Surf features and the second Surf features to obtain a most matched feature point pair;

[0026] Step S3.4, extracting coordinates of the most matched feature point pair;

[0027] Step S3.5, restoring the coordinates of the most matched feature point pair to original coordinates of the image of the first airport region and the image of the second airport region according to the preset scaling ratio;

[0028] Step S3.6, generating a homography matrix of a first image relative to a second image according to the original coordinates, wherein the second image is a scaled image with a larger size in the first airport region scaled image and the second airport region scaled image, and the first image is another scaled image;

[0029] Step S3.7, performing perspective change on the first image according to the homography matrix to obtain a registration image.

[0030] In an embodiment of the present application, step S4 comprises:

[0031] Step S4.1, cutting the registration image into small images of a preset size;

[0032] Step S4.2, processing the small image using the torch Dataset class inherited by the aircraft data class;

[0033] Step S4.3, performing aircraft detection on the processed small image according to the aircraft detection model.

[0034] In an embodiment of the present application, before step S4, the method further comprises:

[0035] Obtaining a second training sample of a yolov5 model as an aircraft data set;

[0036] Using the yolov5s6.pt file as the initialization weight file of the second initial model, creating an airport.yaml file, wherein the nc of the airport.yaml file is set to 10, the names are set to a preset number of aircraft types, and the path parameter is set to the corresponding position of the aircraft data set;

[0037] Training the second initial model to obtain the aircraft detection model, wherein the size of each second training sample is 1024*1024, the multi-scale training mode is enabled during training, the Mosica data enhancement is enabled, the adaptive anchor frame calculation is enabled, and the iteration is 500 epochs.

[0038] In an embodiment of the present application, step S5 comprises:

[0039] If the IOU of the first aircraft target and the second aircraft target is greater than or equal to a first preset value and the class information is also the same, it is determined that the target is unchanged; wherein the first aircraft target is the aircraft target corresponding to the previous moment, and the second aircraft target is the aircraft target corresponding to the next moment;

[0040] If the IOU of the second aircraft target and the first aircraft target is less than the first preset value, the second aircraft target is determined to be a new target, and the first aircraft target is determined to be a reduced target; the aircraft target that is not matched in the next moment is determined to be a new target, and the aircraft target that is not matched in the previous moment is determined to be a reduced target.

[0041] In an embodiment of the present application, step S6 comprises:

[0042] Restoring the aircraft change detection target frame to the first remote sensing image and the second remote sensing image through inverse transformation of the perspective transformation matrix.

[0043] Compared with the prior art, the present application has the following advantages:

[0044] The aircraft change detection method based on the deep learning and the image registration algorithm can improve the change detection precision, greatly improves the aircraft change detection speed, and reduces the hardware resource consumption.

[0045] Other aspects and features of the present application will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the application. It should be understood, however, that the drawings solely are for the purpose of illustration and are not intended to limit the scope of the application as described by the appended claims. It should be further understood that the drawings are not necessarily drawn to scale and that, unless otherwise indicated, they are merely intended to conceptually illustrate the structures and procedures described herein. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 A flowchart of an aircraft change detection method based on deep learning and an image registration algorithm is provided for an embodiment of the application.

[0047] Figure 2 A flowchart of an aircraft change detection method based on deep learning and an image registration algorithm is provided for an embodiment of the application. DETAILED DESCRIPTION

[0048] The application will be described in further detail with reference to specific embodiments. The embodiments of the application are not limited to the specific embodiments described herein.

[0049] Embodiment one

[0050] Please refer to Figure 1 , Figure 1 and Figure 2 A flowchart of an aircraft change detection method based on deep learning and an image registration algorithm is provided for an embodiment of the application. The application provides an aircraft change detection method based on deep learning and an image registration algorithm. The analysis method comprises the following steps:

[0051] S1, acquiring a first remote sensing image and a second remote sensing image of the same region at different time points;

[0052] The remote sensing image size is not limited, but the scenes of the two remote sensing images must be consistent, which is a prerequisite for subsequent image registration and change detection.

[0053] As shown in Figure 2 , the acquired remote sensing images are remote sensing image A and remote sensing image B.

[0054] S2, performing airport detection on the first remote sensing image and the second remote sensing image to acquire an image of a first airport region in the first remote sensing image, a first offset of the first airport region in the first remote sensing image, an image of a second airport region in the second remote sensing image, and a second offset of the second airport region in the second remote sensing image;

[0055] Specifically, step S2 includes:

[0056] Step S2.1, scaling the first remote sensing image into a first scaled image according to a preset scaling ratio; scaling the second remote sensing image into a second scaled image according to the preset scaling ratio;

[0057] Step S2.2: Perform airport detection on the first zoom image and the second zoom image according to the airport detection model to obtain the airport area in the first zoom image and the airport area in the second zoom image;

[0058] In one embodiment, the aforementioned airport detection model needs to be trained, specifically including:

[0059] Get the training samples of the yolov5 model as the airport dataset;

[0060] Use the yolov5s6.pt file as the initial weight file for the initial model, create a new airport.yaml file, set the nc parameter of the airport.yaml file to 1, set names to airport, and set the path parameter to the corresponding location of the airport dataset;

[0061] The initial model is trained to obtain the airport detection model. During training, multi-scale training mode, Mosica data enhancement, and adaptive anchor box calculation are enabled, and 300 epochs are iterated.

[0062] Specifically, we use yolov5 to train an airport detection model and create an airport dataset with an image size of 1280*1280 and 110 images. We use the yolov5s6 model weights for initialization and enable multi-scale training of the model. The training images are randomly scaled to 0.5 to 1.5 times the original images. We also use the Mosica data enhancement method and the adaptive anchor frame calculation provided by yolov5. For different data sets, there will be an initial anchor frame with a set length and width. During network training, we will update the anchor frame until we find the anchor frame that best suits the airport data set and iterate for 300 epochs.

[0063] Step S2.3: Map the airport area in the second zoomed image back to the second remote sensing image according to a preset zoom ratio, crop the image of the airport area in the mapped second remote sensing image to obtain an image of the second airport area, and record the second offset of the second airport area in the second remote sensing image.

[0064] by Figure 2 Taking

[0014] as an example, perform airport detection on remote sensing image A and remote sensing image B according to the following steps.

[0065] Step 1, scale the original remote sensing image to 1280*1280 and record the scaling ratio, the training picture size used by the model of airport detection is 1280*1280, scaling the input remote sensing image to 1280*1280 is conducive to improving the detection accuracy and speed.

[0066] Step 2, map the detected airport back to the original image according to the scaling ratio, and then crop the airport area (that is, cut the detected airport area into a separate picture) and record the offset of the airport in the original image.

[0067] Since the aircraft target appears in the airport area, airport detection is performed before change detection, which can greatly improve the detection speed.

[0068] S3, image registration of the first airport area image and the second airport area image according to the surf image registration algorithm to obtain a registration image;

[0069] In one embodiment, step S3 comprises:

[0070] Step S3.1, scaling the first airport area image to a first airport area scaled image according to a preset scaling ratio; scaling the second airport area image to a second airport area scaled image according to the preset scaling ratio; wherein the scaled image with a larger size among the first airport area scaled image and the second airport area scaled image is taken as a reference image to register the other image;

[0071] Step S3.2, extracting a first Surf feature of the first airport area scaled image and a second Surf feature of the second airport area scaled image;

[0072] Step S3.3, matching the first Surf feature and the second Surf feature to obtain a most matched feature point pair;

[0073] Wherein, the most matched feature point pair is the content in the SURF algorithm, and the KNN algorithm is used to find the nearest two data points, if the ratio of the nearest and the second nearest is greater than a certain value, then the nearest value is reserved, and it is considered that the most matched feature point pair is matched with the point.

[0074] Step S3.4, extracting the coordinates of the most matched feature point pair;

[0075] Step S3.5, restoring the coordinates of the most matched feature point pair to the original coordinates of the first airport area image and the second airport area image according to the preset scaling ratio;

[0076] Step S3.6, generating a homography matrix according to the original coordinates;

[0077] Step S3.7, perspective transformation is performed on the image of the first airport area and the image of the second airport area according to the homography matrix, to obtain a registration image.

[0078] The registration image is obtained by using a Surf algorithm, x1, y1 are coordinates before perspective transformation, x2, y2 are coordinates after perspective transformation, H is a homography matrix obtained by the registration algorithm, and the perspective transformation formula is as follows:

[0079]

[0080]

[0081] Specifically, the cropped airport is subjected to image registration by using a surf image registration algorithm. First, Surf features of two airport images are extracted, the Surf features are matched, the most matched feature point pairs are found, coordinates of the most matched feature point pairs are extracted, a homography matrix is generated, and the cropped airport is subjected to perspective transformation to generate a registration image.

[0082] Before image registration, the airport image can be scaled again according to the recorded scaling ratio, and then the airport image with a larger size is used as a reference image to register the airport image with a smaller size. At this time, the coordinates of the most matched feature point pairs obtained by the Surf image registration algorithm are restored to the coordinates of the original airport image according to the scaling ratio, and the homography matrix after registration is calculated. In this way, the registration speed is greatly improved under the premise of ensuring the accuracy of registration.

[0083] The detected airport image is registered by using a Surf algorithm. Before registration, the airport image is scaled according to the recorded ratio, and then registration is performed, and finally the key point coordinates obtained by the registration algorithm are restored to the coordinate size of the original airport image according to the scaling ratio. In this way, the registration accuracy is ensured, and the registration speed is improved.

[0084] S4, performing plane detection on the registration image to obtain a first plane target and a second plane target, the first plane target including a plane category in the image of the first airport area and coordinates of a plane target frame in the image of the first airport area, and the second plane target including a plane category in the image of the second airport area and coordinates of a plane target frame in the image of the second airport area.

[0085] In one embodiment, step S4 includes:

[0086] Step S4.1, cutting the registration image into small images of a preset size;

[0087] Step S4.2, processing the small image using the aircraft data class inheriting the torch Dataset class;

[0088] Step S4.3, detecting the aircraft according to the aircraft detection model.

[0089] In one embodiment, the aircraft detection model described above also needs to be trained, specifically including:

[0090] Obtain the training sample of the yolov5 model as the aircraft dataset;

[0091] Use the yolov5s6.pt file as the initialization weight file of the second initial model, create a new airport.yaml file, where the nc of the airport.yaml file is set to 10, the names are set to the preset number of aircraft types, and the path parameter is set to the corresponding position of the aircraft dataset;

[0092] Train the second initial model to obtain the aircraft detection model, wherein during training, the multi-scale training mode is enabled, Mosica data augmentation is enabled, adaptive anchor box calculation is enabled, and iteration is performed for 500 epochs.

[0093] Specifically, a yolov5 is used to train an aircraft detection model, an aircraft dataset is made, the image size is 1024*1024, and the number of pictures is 10000; the yolov5s6 model weight is used for initialization, the multi-scale mode is enabled for model training, the training image is randomly scaled to 0.5 times to 1.5 times of the original image, Mosica data augmentation is also used, and the adaptive anchor box calculation of yolov5 is also used. For different datasets, there will be an initial anchor box with a set length and width. In network training, we will update the anchor box until we find the most suitable anchor box for the aircraft dataset, and iteration is performed for 500 epochs.

[0094] The registered airport image is cut into a small image 1024*1024 with an overlap rate of 300. Here, a Dataset class is written to inherit the torch Dataset class, and the small image processed using the aircraft data class is detected to improve the speed by one time compared with direct detection.

[0095] S5, detecting the first aircraft target and the second aircraft target according to IOU to obtain the state of the aircraft target, the state of the aircraft target including: new aircraft target, unchanged aircraft target and reduced aircraft target;

[0096] In one embodiment, step S5 includes:

[0097] If the IOU of the first aircraft target and the second aircraft target is greater than or equal to a first preset value and the category information is the same, then they are determined to be unchanged targets; wherein the first aircraft target is the aircraft target corresponding to the previous moment, and the second aircraft target is the aircraft target corresponding to the next moment;

[0098] If the IOU of the second aircraft target and the first aircraft target is less than the first preset value, the second aircraft target is determined to be a newly added target and the first aircraft target is determined to be a removed target; the aircraft target that is not matched at the latter moment is determined to be a newly added target, and the aircraft target that is not matched at the previous moment is determined to be a removed target.

[0099] That is, if the IOU of the aircraft target in the image of the airport area corresponding to the previous moment at the next moment is less than the first preset value, the aircraft target at the next moment is determined as a newly added target, and the aircraft target at the previous moment is determined as a reduced target; the aircraft target that is not matched at the next moment is determined as a newly added target, and the aircraft target that is not matched at the previous moment is determined as a reduced target.

[0100] Specifically, the aircraft detection results from the two airport images are subjected to change detection, with an IOU threshold of 0.5 set for matching. When the IOU of two aircraft targets is greater than or equal to 0.5 and their category information is the same, they are considered unchanged. When the IOU of an aircraft target in the airport image at a later moment is less than the threshold of 0.5, the aircraft target at the later moment is considered a new target, while the aircraft target at the previous moment is considered a removed target. Aircraft targets that are not matched at the later moment are considered new targets, while those that are not matched at the previous moment are considered removed targets. This method can handle object-level change detection at a speed that meets real-time requirements.

[0101] Using the intersection-over-union (IOU) threshold for change detection can achieve real-time speed requirements.

[0102] S6. Restoring the coordinates of the aircraft target frame in the image of the first airport area to the first remote sensing image, and restoring the coordinates of the aircraft target frame in the image of the second airport area to the second remote sensing image.

[0103] In one embodiment, step S6 includes:

[0104] The coordinates of the aircraft target frame in the first airport are restored to the first remote sensing image through the inverse transformation of the perspective transformation matrix and the first offset of the first airport area in the first remote sensing image; the coordinates of the aircraft target frame in the second airport are restored to the second remote sensing image through the inverse transformation of the perspective transformation matrix and the second offset of the second airport area in the second remote sensing image, and the final change detection result is displayed.

[0105] In summary:

[0106] Firstly, since the target detection model of the application selects a single-stage detection model yolov5, different resolution detection methods are adopted according to different targets, when detecting the airport, the entire image is scaled to 1280*1280, because most of the airports are large targets, the proportion of the entire image is large, even if the original image is scaled, the detection model can also well detect the airport area. When detecting the airplane, it is cut into small pictures, and cut into 1024*1024 with a step distance of 300. The above method speeds up the detection speed.

[0107] Secondly, when the airport image is registered, the airport image is again reduced to extract the Surf feature of the image, and after the optimal matching point is found, the coordinates are restored, which greatly speeds up the registration speed, and this registration method only needs about 0.6 seconds for the airport registration in a 12000*12000 image.

[0108] Thirdly, the airport is cut into 1024*1024 for airplane detection, which can reduce the consumption of video memory.

[0109] In summary, the application improves the accuracy of change detection, greatly improves the speed of airplane change detection, and reduces the hardware resource consumption.

[0110] In the description of the application, the terms "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined as "first", "second" can be explicitly or implicitly included one or more of the features. In the description of the application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0111] In the description of the application, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or specific data points described in conjunction with the embodiment or example are included in at least one embodiment or example of the application. In the description of the application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or specific data points described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in the specification.

[0112] The above is further detailed description of the present application in combination with specific preferred embodiments, and cannot be deemed as limitation of the specific implementation of the present application to these descriptions. For those skilled in the art to which the present application belongs, without departing from the concept of the present application, a number of simple deductions or substitutions can be made, and all of them shall be deemed as falling within the protection scope of the present application.

Claims

1. An aircraft change detection method based on deep learning and image registration algorithm, characterized in that: The following steps are involved: S1. Acquire a first remote sensing image and a second remote sensing image of the same area at different times; S2. Perform airport detection on the first remote sensing image and the second remote sensing image to obtain an image of a first airport area in the first remote sensing image, a first offset of the first airport area in the first remote sensing image, an image of a second airport area in the second remote sensing image, and a second offset of the second airport area in the second remote sensing image; S3. Performing image registration on the image of the first airport area and the image of the second airport area according to a SURF image registration algorithm to obtain a registered image; S4. Perform aircraft detection on the registered image to obtain a first aircraft target and a second aircraft target, where the first aircraft target includes: an aircraft type in the image of the first airport area and coordinates of an aircraft target frame in the image of the first airport area; and the second aircraft target includes: an aircraft type in the image of the second airport area and coordinates of an aircraft target frame in the image of the second airport area. S5. Perform change detection on the first aircraft target and the second aircraft target according to the IOU to obtain aircraft target states, where the aircraft target states include: a newly added aircraft target, an unchanged aircraft target, and a reduced aircraft target; S6. Restoring the coordinates of the aircraft target frame in the image of the first airport area to the first remote sensing image, and restoring the coordinates of the aircraft target frame in the image of the second airport area to the second remote sensing image.

2. The aircraft change detection method based on deep learning and image registration algorithm according to claim 1 is characterized in that: Step S2 includes: Step S2.1: scaling the first remote sensing image into a first scaled image according to a preset scaling ratio; scaling the second remote sensing image into a second scaled image according to a preset scaling ratio; Step S2.2: Perform airport detection on the first zoom image and the second zoom image according to the airport detection model to obtain an airport area in the first zoom image and an airport area in the second zoom image; Step S2.3: Mapping the airport area in the first zoomed image back to the first remote sensing image according to the preset zoom ratio, cropping the image of the airport area in the mapped first remote sensing image to obtain an image of the first airport area, and recording a first offset of the first airport area in the first remote sensing image; Step S2.4: Map the airport area in the second zoom image back to the second remote sensing image according to the preset zoom ratio, crop the image of the airport area in the mapped second remote sensing image to obtain an image of the second airport area, and record the second offset of the second airport area in the second remote sensing image.

3. The aircraft change detection method based on deep learning and image registration algorithm according to claim 2 is characterized in that: Before step S2, the method further includes: Get the first training sample of the yolov5 model as the airport dataset; Use the yolov5s6.pt file as the initialization weight file for the first initial model, create a new airport.yaml file, where the nc corresponding to the airport.yaml file is set to 1, the names is set to airport, and the path parameter is set to the corresponding location of the airport dataset; The first initial model is trained to obtain the airport detection model, wherein the size of each first training sample is 1280*1280, and the multi-scale training mode, Mosica data enhancement, and adaptive anchor box calculation are enabled during training, and 300 epochs are iterated.

4. The aircraft change detection method based on deep learning and image registration algorithm according to claim 1, characterized in that: Step S3 includes: Step S3.1: scaling the image of the first airport area into a first airport area zoom map according to a preset scaling ratio; scaling the image of the second airport area into a second airport area zoom map according to the preset scaling ratio; Step S3.2: extracting a first Surf feature of the first airport area zoom image, and extracting a second Surf feature of the second airport area zoom image; Step S3.3: Match the first Surf feature and the second Surf feature to obtain the most matching feature point pair; Step S3.4, extracting the coordinates of the best matching feature point pair; Step S3.5: Restoring the coordinates of the best-matched feature point pair to the original coordinates of the image of the first airport area and the image of the second airport area according to the preset scaling ratio; Step S3.6: Generate a homography matrix of the first image relative to the second image based on the original coordinates; wherein the second image is the larger scaled image of the first and second airport area scaled images, and the first image is the other scaled image; Step S3.7: Perform perspective transformation on the first image according to the homography matrix to obtain a registered image.

5. The aircraft change detection method based on deep learning and image registration algorithm according to claim 1, characterized in that: Step S4 includes: Step S4.1, cropping the registered image into small images of a preset size; Step S4.2: Use the aircraft data class to inherit the torch Dataset class to process the small image; Step S4.3: Perform aircraft detection on the processed thumbnail according to the aircraft detection model.

6. The aircraft change detection method based on deep learning and image registration algorithm according to claim 5 is characterized in that: Before step S4, the method further includes: Get the second training sample of the yolov5 model as the aircraft data set; Use the yolov5s6.pt file as the initialization weight file for the second initial model, create a new airport.yaml file, where the nc corresponding to the airport.yaml file is set to 10, the names is set to a preset number of aircraft types, and the path parameter is set to the corresponding location of the aircraft dataset; The second initial model is trained to obtain the aircraft detection model, where the size of each second training sample is 1024*1024. During training, multi-scale training mode, Mosica data enhancement, and adaptive anchor box calculation are enabled, and 500 epochs are iterated.

7. The aircraft change detection method based on deep learning and image registration algorithm according to claim 1, characterized in that: Step S5 includes: If the IOU of the first aircraft target and the second aircraft target is greater than or equal to a first preset value and the category information is the same, then the first aircraft target is determined to be an unchanged target; wherein the first aircraft target is the aircraft target corresponding to the previous moment, and the second aircraft target is the aircraft target corresponding to the next moment; If the IOU between the second aircraft target and the first aircraft target is less than the first preset value, the second aircraft target is determined as a newly added target and the first aircraft target is determined as a removed target; the aircraft target that was not matched at the latter moment is determined as a newly added target, and the aircraft target that was not matched at the previous moment is determined as a removed target.

8. The aircraft change detection method based on deep learning and image registration algorithm according to claim 1, characterized in that: Step S6 includes: The aircraft change detection target frame is restored to the first remote sensing image and the second remote sensing image through the inverse transformation of the perspective transformation matrix, wherein the perspective transformation matrix is ​​a homography matrix obtained by the registration algorithm.

Images