A method to reduce spatiotemporal registration error in multi-source remote sensing small target detection

Abstract

The present invention discloses a method for reducing the spatiotemporal registration error in multi-source remote sensing weak target detection, comprising: obtaining a first image of the weak target by a first sensor; obtaining a second image of the weak target by a second sensor; determining by a terminal whether the weak target in the first image and the weak target in the second image are located at the same position; if the terminal determines that the weak target in the first image and the weak target in the second image are located at the same position, the terminal continues to determine whether the weak target in the first image and the weak target in the second image are the same target; if the terminal fails to successfully determine whether the weak target in the first image and the weak target in the second image are the same target, the terminal provides the first image and the second image to the user. The method of the present invention introduces user intervention and predictive judgment methods, respectively solving the problems of temporal registration error and spatial registration error.

Description

Technical Field

[0001] The invention relates to the technical field of electrical data processing, in particular to a method for reducing the temporal and spatial registration error in multi-source remote sensing small target detection. Background Art

[0002] Multi-source remote sensing small target detection is a technology that combines multi-source remote sensing data (such as visible light, infrared light, multi-spectrum, etc.) with advanced algorithms to identify and locate targets with small size, low contrast, and susceptible to background interference. At present, there is a problem of spatiotemporal registration error in the process of multi-source remote sensing small target detection. In short, due to the limitations of various sensors, the sampling frame rates and image resolutions of various sensors are different. Due to the different sampling frame rates and image resolutions, the small target detection system may report errors. Summary of the invention

[0003] In order to solve the defects of the prior art, the present invention provides a method for reducing the spatiotemporal registration error in multi-source remote sensing weak target detection. The method of the present invention introduces user intervention and predictive judgment methods to respectively solve the problems of temporal registration error and spatial registration error.

[0004] To achieve the above object, the present invention provides a method for reducing the spatiotemporal registration error in multi-source remote sensing small target detection, comprising:

[0005] Acquiring a first image of a small target by a first sensor;

[0006] Acquiring a second image of the dim target by a second sensor, wherein a resolution of the first image is higher than a resolution of the second image;

[0007] The terminal determines whether the small and weak target in the first image and the small and weak target in the second image are located at the same position;

[0008] If the terminal determines that the small and weak target in the first image and the small and weak target in the second image are located at the same position, the terminal continues to determine whether the small and weak target in the first image and the small and weak target in the second image are the same target;

[0009] If the terminal fails to successfully determine whether the small target in the first image and the small target in the second image are the same target, the terminal provides the first image and the second image to the user, wherein the first image and the second image have marked areas.

[0010] In a preferred embodiment, the first sensor acquires the image of the small target at a first frame rate, and the second sensor acquires the image of the small target at a second frame rate, wherein the first frame rate is greater than the second frame rate;

[0011] The method further includes:

[0012] Receiving, by the terminal, a first set of image frames from a first sensor;

[0013] Receiving, by the terminal, a second set of image frames from a second sensor;

[0014] The terminal establishes a correspondence between each image frame in the first image frame set and each image frame in the second image frame set based on the time relationship;

[0015] The terminal determines whether the small and weak targets in the corresponding image frames are located at the same position.

[0016] In a preferred embodiment, the method further comprises:

[0017] If the terminal determines that the small and weak target in the first image frame and the small and weak target in the second image frame are not located at the same position, the terminal locates a third image frame immediately before the second image frame, wherein the first image frame and the second image frame correspond to each other, wherein the first image frame belongs to a first image frame set, and the second image frame belongs to a second image frame set;

[0018] The terminal determines a fourth image frame corresponding to the third image frame;

[0019] The terminal determines whether the small and weak target in the third image frame and the small and weak target in the fourth image frame are located at the same position;

[0020] If the terminal determines that the small and weak target in the third image frame and the small and weak target in the fourth image frame are located at the same position, the terminal locates the fifth image frame immediately after the second image frame;

[0021] The terminal determines a sixth image frame corresponding to the fifth image frame.

[0022] In a preferred embodiment, the method further comprises:

[0023] The terminal determines whether the small and weak target in the fifth image frame and the small and weak target in the sixth image frame are located at the same position;

[0024] If the terminal determines that the small and weak target in the fifth image frame and the small and weak target in the sixth image frame are located at the same position, the terminal determines whether the position of the small and weak target in each image frame before the sixth image frame and after the fourth image frame is the same as the position of the small and weak target in the second image frame;

[0025] If the terminal determines that the position of the small target in an image frame before the sixth image frame and after the fourth image frame is the same as the position of the small target in the second image frame, the terminal establishes a correspondence between the image frame and the second image frame.

[0026] In a preferred embodiment, the method further comprises:

[0027] If the terminal determines that the position of the weak target in more than one image frame before the sixth image frame and after the fourth image frame is the same as the position of the weak target in the second image frame, the terminal determines that the time registration has failed, and the terminal notifies the user of the time registration failure.

[0028] The present invention also provides a system for reducing the spatiotemporal registration error in multi-source remote sensing small target detection, the system comprising a module for performing the following operations:

[0029] Acquiring a first image of a small target by a first sensor;

[0030] Acquiring a second image of the dim target by a second sensor, wherein a resolution of the first image is higher than a resolution of the second image;

[0031] The terminal determines whether the small and weak target in the first image and the small and weak target in the second image are located at the same position;

[0032] If the terminal determines that the small and weak target in the first image and the small and weak target in the second image are located at the same position, the terminal continues to determine whether the small and weak target in the first image and the small and weak target in the second image are the same target;

[0033] If the terminal fails to successfully determine whether the small target in the first image and the small target in the second image are the same target, the terminal provides the first image and the second image to the user, wherein the first image and the second image have marked areas.

[0034] In a preferred embodiment, the first sensor acquires the image of the small target at a first frame rate, and the second sensor acquires the image of the small target at a second frame rate, wherein the first frame rate is greater than the second frame rate;

[0035] The system also includes modules for performing the following operations:

[0036] Receiving, by the terminal, a first set of image frames from a first sensor;

[0037] Receiving, by the terminal, a second set of image frames from a second sensor;

[0038] The terminal establishes a correspondence between each image frame in the first image frame set and each image frame in the second image frame set based on the time relationship;

[0039] The terminal determines whether the small and weak targets in the corresponding image frames are located at the same position.

[0040] In a preferred embodiment, the system further comprises a module for performing the following operations:

[0041] If the terminal determines that the small and weak target in the first image frame and the small and weak target in the second image frame are not located at the same position, the terminal locates a third image frame immediately before the second image frame, wherein the first image frame and the second image frame correspond to each other, wherein the first image frame belongs to a first image frame set, and the second image frame belongs to a second image frame set;

[0042] The terminal determines a fourth image frame corresponding to the third image frame;

[0043] The terminal determines whether the small and weak target in the third image frame and the small and weak target in the fourth image frame are located at the same position;

[0044] If the terminal determines that the small and weak target in the third image frame and the small and weak target in the fourth image frame are located at the same position, the terminal locates the fifth image frame immediately after the second image frame;

[0045] The terminal determines a sixth image frame corresponding to the fifth image frame.

[0046] In a preferred embodiment, the system further comprises a module for performing the following operations:

[0047] The terminal determines whether the small and weak target in the fifth image frame and the small and weak target in the sixth image frame are located at the same position;

[0048] If the terminal determines that the small and weak target in the fifth image frame and the small and weak target in the sixth image frame are located at the same position, the terminal determines whether the position of the small and weak target in each image frame before the sixth image frame and after the fourth image frame is the same as the position of the small and weak target in the second image frame;

[0049] If the terminal determines that the position of the small target in an image frame before the sixth image frame and after the fourth image frame is the same as the position of the small target in the second image frame, the terminal establishes a correspondence between the image frame and the second image frame.

[0050] In a preferred embodiment, the system further comprises a module for performing the following operations:

[0051] If the terminal determines that the position of the weak target in more than one image frame before the sixth image frame and after the fourth image frame is the same as the position of the weak target in the second image frame, the terminal determines that the time registration has failed, and the terminal notifies the user of the time registration failure.

[0052] Compared with the prior art, the present invention has the following advantages: due to different sampling frame rates and image resolutions, the small target detection system may report errors. The method of the present invention introduces user intervention and predictive judgment methods to solve the problems of temporal registration error and spatial registration error respectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a method flow chart of an embodiment of the present invention.

[0054] Figure 2 It is a schematic diagram of a computer screen interface according to an embodiment of the present invention.

[0055] Figure 3 It is a schematic diagram of the correspondence between image frames according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific implementation modes.

[0057] As mentioned above, due to different sampling frame rates and image resolutions, the weak target detection system may report errors. Specifically, after our research, the reasons for the weak target detection system reporting errors can be introduced from the perspective of different image resolutions and different sampling frame rates. First, different image resolutions may cause the computer (in the present invention, the computer is a specific example of a terminal) to be unable to automatically determine through machine vision whether the weak targets in different images collected by different sensors are the same target. The reasons are as follows: for example, the resolution of visible light sensors is generally higher, and the resolution of infrared sensors is generally lower. Assuming that a weak target occupies 20 pixels in the image collected by the visible light sensor, the same target may only occupy 5 pixels in the image collected by the infrared sensor at the same time. The fact that the weak target occupies fewer pixels in the image collected by the infrared sensor means that the image collected by the infrared sensor can provide less information to the computer. Therefore, the computer may not be able to collect sufficient feature information about the weak target in the image captured by the infrared sensor. Therefore, due to the lack of feature information, the computer cannot determine that the weak target in the image captured by the visible light sensor is the same target as the weak target in the image captured by the infrared sensor based solely on the automatic algorithm of machine vision, which will cause the weak target detection system to report an error (for example, reporting that the target is lost).

[0058] In addition, different sampling frame rates may cause the computer to misjudge the lost target for the following reasons: Generally speaking, the sampling frequency of visible light sensors is relatively high. The sampling frequency of some mid-range visible light cameras can reach 60 frames per second (i.e. 60fps), while the sampling frequency of infrared sensors is generally relatively low (this is because there are technical bottlenecks in the manufacturing process of infrared focal plane arrays, and high-density small pixel size devices are difficult to achieve and expensive. In addition, the structural limitations of the detector array lead to insufficient fill factor. According to the Nyquist sampling theorem, the sampling frequency is difficult to reach twice the highest frequency of natural scene images, which causes signal aliasing and image blur. This physical limitation directly restricts the sampling capability of the infrared system). The sampling frequency of general scientific research infrared sensors is about 15fps, which leads to mismatch problems between image frames. In order to achieve continuous detection of weak targets, it is necessary to continuously track weak targets. In order to ensure that the target is not lost, it is necessary to find weak targets in both the image frames collected by the visible light sensor and the image frames collected by the infrared sensor that correspond in time (corresponding in time means that the two image frames are taken at the same time t). However, due to the different sampling frequencies of the two sensors, the jitter of the timer may cause the computer to mistakenly believe that the visible light image frame taken at time t and the infrared image frame taken at time t+1 are two corresponding image frames in time at a certain moment, while the position of the small target in the visible light image frame taken at time t does not coincide with the position in the infrared image frame taken at time t+1, which will cause the computer to mistakenly believe that the small target has been lost, thereby causing misjudgment of the lost target. The method of the present invention is intended to solve the above technical problems.

[0059] Example 1

[0060] Figure 1 1 is a method flow chart of an embodiment of the present invention. As shown in the figure, the method of the present invention includes the following steps:

[0061] Step 1: A first image of a small target is acquired by a first sensor. It should be understood that the sensor in the present invention may be any type of image sensor, and the image sensor may be, for example, various cameras. For the convenience of discussion of the embodiments below, a visible light camera is selected as an example of the first sensor.

[0062] Step 2: Acquire a second image of the dim target by a second sensor, wherein the resolution of the first image is higher than the resolution of the second image; for the convenience of discussion of the embodiments below, an infrared camera is selected as an example of the first sensor;

[0063] Step 3: The terminal determines whether the weak target in the first image and the weak target in the second image are located at the same position. In one example, a weak target may appear in the first image taken by the visible light camera, but because the weak target is small or the infrared signal of the weak target itself is weak, the weak target may not appear in the second image taken by the infrared camera. At this time, the weak target in the first image and the weak target in the second image are not in the same position. Due to the particularity of multi-source remote sensing technology (that is, it is required to find the same weak target in multiple sources), the terminal must issue a detection failure reminder at this time so that the user can adjust the detection strategy as soon as possible. For example, if the infrared signal of the weak target is weak and it is impossible to require the weak target to send a strong infrared signal, the user can stop using the infrared camera.

[0064] Step 4: If the terminal determines that the small and weak targets in the first image and the small and weak targets in the second image are located at the same position, the terminal continues to determine whether the small and weak targets in the first image and the small and weak targets in the second image are the same target; in one example, although the resolution of the infrared camera is lower than that of the visible light camera, the second image cannot provide the computer with much feature information about the small and weak targets, but the computer can still determine that the small and weak targets in the first image and the small and weak targets in the second image are the same target based on the machine vision algorithm according to the less feature information about the small and weak targets provided by the second image (of course, the determination result may also be: the small and weak targets in the first image and the small and weak targets in the second image are not the same target);

[0065] Step 5: If the terminal fails to successfully determine whether the weak target in the first image and the weak target in the second image are the same target, the terminal provides the first image and the second image to the user, wherein the first image and the second image have annotated areas. In one example, if the second image cannot provide the computer with sufficient feature information about the weak target, the computer itself cannot determine whether the weak target in the first image and the weak target in the second image are the same target; at this time, the computer can display the first image and the second image on its screen so that the user can help the computer determine whether the weak target in the first image and the weak target in the second image are the same target. The interface for the computer to display the first image and the second image on its screen can be referred to. Figure 2 .like Figure 2As shown, the computer screen displays the first image on the left and the second image on the right. The lower left of the first image has an image of a weak target composed of 24 pixels (pixels are represented by small squares), and the lower left of the second image has an image of a weak target composed of 6 pixels. At this time, since the pixels of the weak target in the second image are too few, the terminal cannot successfully determine whether the weak target in the first image is the same as the weak target in the second image. Subsequently, the computer provides the first image and the second image to the user, and marks the location of the weak target that the user needs to judge and pay attention to (in Figure 2 In the 2nd image, the computer marks the position of the weak target with an ellipse outside the weak target). The user can judge whether the weak target in the first image is the same as the weak target in the second image by the existing features and details of the weak target itself, as well as the position and logical relationship between the weak target and other markers. Since users have rich experience and richer reasoning ability that current AI technology cannot achieve, it is possible for users to perform judgment actions that computers cannot complete, thereby avoiding errors reported by the weak target detection system. It should be understood that in Figure 2 In the figure, for the purpose of clear presentation, the small target is drawn larger. In the actual system, the area of ​​the small target is actually very small.

[0066] Example 2

[0067] In Embodiment 2, the first sensor acquires an image of a small target at a first frame rate, and the second sensor acquires an image of a small target at a second frame rate, wherein the first frame rate is greater than the second frame rate;

[0068] The method further includes:

[0069] A first image frame set is received by the terminal from the first sensor; it should be noted that, in the present invention, the terms image frame and image are respectively applied to different embodiments. In the embodiment 1 describing the solution to the spatial registration error, the term "image" is used to represent the information acquired by the sensor, and in other embodiments describing the solution to the temporal registration error, the term "image frame" is used to represent the information acquired by the sensor. This distinction is to make the description of the specification clearer;

[0070] Receiving, by the terminal, a second set of image frames from a second sensor;

[0071] The terminal establishes a correspondence between each image frame in the first image frame set and each image frame in the second image frame set based on the time relationship; Figure 3 Hereinafter, an example will be described in which a terminal establishes a correspondence relationship between each image frame in a first image frame set and each image frame in a second image frame set based on a time relationship. Figure 3As shown, the terminal receives a first image frame set from a visible light camera (assuming that its sampling frequency is 60fps) and a second image frame set from an infrared camera (assuming that its sampling frequency is 15fps). First, the terminal matches the first image frame in the first image frame set (i.e., the image frame received at time t) with the first image frame in the second image frame set. Then, the computer can calculate the position where the corresponding image frame appears next based on the sampling frequency of the camera. Figure 3 In , the fifth image frame in the first image frame set corresponding to time t+s (that is, the image frame received at time t+s) and the second image frame in the second image frame set correspond to each other, and the subsequent operations are similar. It should be noted that Figure 3 In the example, the system is in the initial stage of operation, and the possibility of timer jitter is small (timer jitter is generally caused by error accumulation after running for too long), so it can be ensured that the correspondence between the first image frame in the first image frame set and the first image frame in the second image frame set is correct;

[0072] The terminal determines whether the small and weak targets in the corresponding image frames are located at the same position.

[0073] Example 3

[0074] In embodiment 3, the method further comprises:

[0075] If the terminal determines that the small target in the first image frame and the small target in the second image frame are not located at the same position, the terminal locates the third image frame immediately before the second image frame, wherein the first image frame and the second image frame correspond to each other, wherein the first image frame belongs to the first image frame set, and the second image frame belongs to the second image frame set; in an example, as the timer runs longer, the computer may mistakenly correspond the image frames, so that Figure 3 For example, if the timer does not jitter, the computer should be able to correctly determine that the fifth image frame in the first image frame set and the second image frame in the second image frame set are corresponding image frames. At this time, the weak targets in the two images should be in the same position in the two image frames. However, if the timer jitters, the computer may mistakenly determine that the fourth image frame in the first image frame set and the second image frame in the second image frame set are corresponding image frames. At this time, since the fourth image frame in the first image frame set and the second image frame in the second image frame set are actually taken at different times, the positions of the weak targets in the two are different. At this time, the computer concludes that the positions of the weak targets are different in the corresponding image frames, which will cause the computer to erroneously conclude that the target is lost. In an example, assuming that the second image frame is Figure 3The second image frame in the second image frame set in , then the third image frame immediately before the second image frame is Figure 3 The first image frame in the second image frame set in ; our research found that timer jitter is a low-probability event, so it is almost impossible for timer jitter to occur when two frames of images are collected continuously. Therefore, it can be considered that the time correspondence between the third image frame and the fourth image frame is correct;

[0076] The terminal determines a fourth image frame corresponding to the third image frame; Figure 3 In the example, the fourth image frame is the first image frame in the first image frame set;

[0077] The terminal determines whether the small and weak target in the third image frame and the small and weak target in the fourth image frame are located at the same position;

[0078] If the terminal determines that the small target in the third image frame is located at the same position as the small target in the fourth image frame, the terminal locates the fifth image frame immediately after the second image frame; in one example, as long as the target is not really lost, the small target in the third image frame is definitely located at the same position as the small target in the fourth image frame; in one example, the fifth image frame should be Figure 3 The third image frame in the second image frame set (not shown in 3);

[0079] The terminal determines a sixth image frame corresponding to the fifth image frame. In one example, the sixth image frame should be Figure 3 The ninth image frame in the first image frame set (not shown in Figure 3).

[0080] In addition, the method also includes:

[0081] The terminal determines whether the small and weak target in the fifth image frame and the small and weak target in the sixth image frame are located at the same position;

[0082] If the terminal determines that the small and weak targets in the fifth image frame and the small and weak targets in the sixth image frame are located at the same position, the terminal determines whether the positions of the small and weak targets in the image frames before the sixth image frame and after the fourth image frame are the same as the positions of the small and weak targets in the second image frame; in one example, as long as the target is not really lost, the small and weak targets in the fifth image frame and the small and weak targets in the sixth image frame are definitely located at the same position; the purpose of performing the judgment step in this embodiment is mainly to determine whether the target is really lost. Our research has found that if the target is really lost, the positions of the small and weak targets in several consecutive corresponding image frames in the first image frame set and the second image frame set will be different;

[0083] If the terminal determines that the position of the small target in an image frame before the sixth image frame and after the fourth image frame is the same as the position of the small target in the second image frame, the terminal establishes a correspondence between the image frame and the second image frame.

[0084] Example 4

[0085] In embodiment 4, the method further comprises:

[0086] If the terminal determines that the position of the small target in more than one image frame before the sixth image frame and after the fourth image frame is the same as the position of the small target in the second image frame, the terminal determines that the time registration has failed, and the terminal notifies the user of the time registration failure. The purpose of designing this step is that if the small target remains stationary, the position of the small target in the second image frame may be consistent with the position of the small target in multiple image frames in the first image frame set. At this time, it is impossible to determine which image frame the second image frame specifically corresponds to, and the user should be reminded at this time.

[0087] Example 5

[0088] The present invention also provides a system for reducing the spatiotemporal registration error in multi-source remote sensing small target detection, the system comprising a module for performing the following operations:

[0089] Acquiring a first image of a small target by a first sensor;

[0090] Acquiring a second image of the dim target by a second sensor, wherein a resolution of the first image is higher than a resolution of the second image;

[0091] The terminal determines whether the small and weak target in the first image and the small and weak target in the second image are located at the same position;

[0092] If the terminal determines that the small and weak target in the first image and the small and weak target in the second image are located at the same position, the terminal continues to determine whether the small and weak target in the first image and the small and weak target in the second image are the same target;

[0093] If the terminal fails to successfully determine whether the small target in the first image and the small target in the second image are the same target, the terminal provides the first image and the second image to the user, wherein the first image and the second image have marked areas.

[0094] In a preferred embodiment, the first sensor acquires the image of the small target at a first frame rate, and the second sensor acquires the image of the small target at a second frame rate, wherein the first frame rate is greater than the second frame rate;

[0095] The system also includes modules for performing the following operations:

[0096] Receiving, by the terminal, a first set of image frames from a first sensor;

[0097] Receiving, by the terminal, a second set of image frames from a second sensor;

[0098] The terminal establishes a correspondence between each image frame in the first image frame set and each image frame in the second image frame set based on the time relationship;

[0099] The terminal determines whether the small and weak targets in the corresponding image frames are located at the same position.

[0100] In a preferred embodiment, the system further comprises a module for performing the following operations:

[0101] If the terminal determines that the small and weak target in the first image frame and the small and weak target in the second image frame are not located at the same position, the terminal locates a third image frame immediately before the second image frame, wherein the first image frame and the second image frame correspond to each other, wherein the first image frame belongs to a first image frame set, and the second image frame belongs to a second image frame set;

[0102] The terminal determines a fourth image frame corresponding to the third image frame;

[0103] The terminal determines whether the small and weak target in the third image frame and the small and weak target in the fourth image frame are located at the same position;

[0104] If the terminal determines that the small and weak target in the third image frame and the small and weak target in the fourth image frame are located at the same position, the terminal locates the fifth image frame immediately after the second image frame;

[0105] The terminal determines a sixth image frame corresponding to the fifth image frame.

[0106] In a preferred embodiment, the system further comprises a module for performing the following operations:

[0107] The terminal determines whether the small and weak target in the fifth image frame and the small and weak target in the sixth image frame are located at the same position;

[0108] If the terminal determines that the small and weak target in the fifth image frame and the small and weak target in the sixth image frame are located at the same position, the terminal determines whether the position of the small and weak target in each image frame before the sixth image frame and after the fourth image frame is the same as the position of the small and weak target in the second image frame;

[0109] If the terminal determines that the position of the small target in an image frame before the sixth image frame and after the fourth image frame is the same as the position of the small target in the second image frame, the terminal establishes a correspondence between the image frame and the second image frame.

[0110] In a preferred embodiment, the system further comprises a module for performing the following operations:

[0111] If the terminal determines that the position of the weak target in more than one image frame before the sixth image frame and after the fourth image frame is the same as the position of the weak target in the second image frame, the terminal determines that the time registration has failed, and the terminal notifies the user of the time registration failure.

Claims

1. A method for reducing spatiotemporal registration errors in multi-source remote sensing small target detection, comprising: Acquiring a first image of the small target by a first sensor; Acquiring a second image of the dim target by a second sensor, wherein a resolution of the first image is higher than a resolution of the second image; The terminal determines whether the small and weak target in the first image and the small and weak target in the second image are located at the same position; If the terminal determines that the small and weak target in the first image and the small and weak target in the second image are located at the same position, the terminal continues to determine whether the small and weak target in the first image and the small and weak target in the second image are the same target; If the terminal fails to successfully determine whether the small target in the first image and the small target in the second image are the same target, the terminal provides the first image and the second image to the user, wherein the first image and the second image have marked areas.

2. The method according to claim 1, wherein: The first sensor acquires the image of the small target at a first frame rate, and the second sensor acquires the image of the small target at a second frame rate, wherein the first frame rate is greater than the second frame rate; Wherein, the method further comprises: Receiving, by the terminal, a first set of image frames from the first sensor; Receiving, by the terminal, a second set of image frames from the second sensor; The terminal establishes a correspondence between each image frame in the first image frame set and each image frame in the second image frame set based on a time relationship; The terminal determines whether the small and weak targets in the corresponding image frames are located at the same position.

3. The method according to claim 2, wherein: The method further comprises: If the terminal determines that the small target in the first image frame and the small target in the second image frame are not located at the same position, the terminal locates a third image frame immediately before the second image frame, wherein the first image frame and the second image frame correspond to each other, wherein the first image frame belongs to the first image frame set, and the second image frame belongs to the second image frame set; The terminal determines a fourth image frame corresponding to the third image frame; The terminal determines whether the small and weak target in the third image frame and the small and weak target in the fourth image frame are located at the same position; If the terminal determines that the small and weak target in the third image frame and the small and weak target in the fourth image frame are located at the same position, the terminal locates a fifth image frame immediately after the second image frame; The terminal determines a sixth image frame corresponding to the fifth image frame.

4. The method according to claim 3, wherein: The method further comprises: The terminal determines whether the small and weak target in the fifth image frame and the small and weak target in the sixth image frame are located at the same position; If the terminal determines that the small and weak target in the fifth image frame and the small and weak target in the sixth image frame are located at the same position, the terminal determines whether the position of the small and weak target in each image frame before the sixth image frame and after the fourth image frame is the same as the position of the small and weak target in the second image frame; If the terminal determines that the position of the weak target in an image frame before the sixth image frame and after the fourth image frame is the same as the position of the weak target in the second image frame, the terminal establishes a corresponding relationship between the one image frame and the second image frame.

5. The method according to claim 4, wherein: The method further comprises: If the terminal determines that the position of the weak target in more than one image frame before the sixth image frame and after the fourth image frame is the same as the position of the weak target in the second image frame, the terminal determines that the time alignment has failed, and the terminal notifies the user of the time alignment failure.

6. A system for reducing spatiotemporal registration errors in multi-source remote sensing small target detection, the system comprising modules for performing the following operations: Acquiring a first image of the small target by a first sensor; Acquiring a second image of the dim target by a second sensor, wherein a resolution of the first image is higher than a resolution of the second image; The terminal determines whether the small and weak target in the first image and the small and weak target in the second image are located at the same position; If the terminal determines that the small and weak target in the first image and the small and weak target in the second image are located at the same position, the terminal continues to determine whether the small and weak target in the first image and the small and weak target in the second image are the same target; If the terminal fails to successfully determine whether the small target in the first image and the small target in the second image are the same target, the terminal provides the first image and the second image to the user, wherein the first image and the second image have marked areas.

7. The system according to claim 6, wherein: The first sensor acquires the image of the small target at a first frame rate, and the second sensor acquires the image of the small target at a second frame rate, wherein the first frame rate is greater than the second frame rate; The system further includes a module for performing the following operations: Receiving, by the terminal, a first set of image frames from the first sensor; Receiving, by the terminal, a second set of image frames from the second sensor; The terminal establishes a correspondence between each image frame in the first image frame set and each image frame in the second image frame set based on a time relationship; The terminal determines whether the small and weak targets in the corresponding image frames are located at the same position.

8. The system according to claim 7, wherein: The system also includes modules for performing the following operations: If the terminal determines that the small target in the first image frame and the small target in the second image frame are not located at the same position, the terminal locates a third image frame immediately before the second image frame, wherein the first image frame and the second image frame correspond to each other, wherein the first image frame belongs to the first image frame set, and the second image frame belongs to the second image frame set; The terminal determines a fourth image frame corresponding to the third image frame; The terminal determines whether the small and weak target in the third image frame and the small and weak target in the fourth image frame are located at the same position; If the terminal determines that the small and weak target in the third image frame and the small and weak target in the fourth image frame are located at the same position, the terminal locates a fifth image frame immediately after the second image frame; The terminal determines a sixth image frame corresponding to the fifth image frame.

9. The system according to claim 8, wherein: The system also includes modules for performing the following operations: The terminal determines whether the small and weak target in the fifth image frame and the small and weak target in the sixth image frame are located at the same position; If the terminal determines that the small and weak target in the fifth image frame and the small and weak target in the sixth image frame are located at the same position, the terminal determines whether the position of the small and weak target in each image frame before the sixth image frame and after the fourth image frame is the same as the position of the small and weak target in the second image frame; If the terminal determines that the position of the weak target in an image frame before the sixth image frame and after the fourth image frame is the same as the position of the weak target in the second image frame, the terminal establishes a corresponding relationship between the one image frame and the second image frame.

10. The system according to claim 9, wherein: The system also includes modules for performing the following operations: If the terminal determines that the position of the weak target in more than one image frame before the sixth image frame and after the fourth image frame is the same as the position of the weak target in the second image frame, the terminal determines that the time alignment has failed, and the terminal notifies the user of the time alignment failure.

Images