Target detection method and electronic device

Abstract

The present disclosure provides a target detection method and an electronic device, which can be applied to the technical field of computer vision. The method comprises: obtaining an object graph in a t-th image frame based on a target satellite video and historical detection results; obtaining a first detection result of a target object in the t-th image frame according to the t-th image frame, the object graph, a target historical image frame and a historical detection result of the target historical image frame, wherein the target historical image frame is one image frame in the target satellite video; determining a second detection result of the target object in the t-th image frame according to the historical detection result; and determining the first detection result as a target detection result in a case where the first detection result matches the second detection result.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision technology, and in particular to a target detection method and electronic device. Background Technology

[0002] In related technologies, when detecting target objects in satellite video, rectangular boxes are usually used to mark the target objects in the image frames of the satellite video. The target objects in satellite video mainly include ships, airplanes, vehicles, etc. Due to the limitation of the spatial resolution of remote sensing satellites, many target objects are small in size in satellite video, and the length or width of the bounding rectangle of the target object is only a few to a dozen pixels.

[0003] In the process of realizing the inventive concept disclosed herein, the inventors discovered that in the relevant technologies, when detecting target objects in satellite video, there are problems of missed detection and false detection of target objects. Summary of the Invention

[0004] In view of the above problems, this disclosure provides a target detection method and an electronic device.

[0005] According to a first aspect of this disclosure, a target detection method is provided, comprising: obtaining an object map in an image frame t based on a target satellite video and historical detection results; obtaining a first detection result of a target object in the image frame t based on the target satellite video image frame t, the object map, a target historical image frame, and historical detection results of the target historical image frame, wherein the target historical image frame is an image frame in the target satellite video; determining a second detection result of a target object in the image frame t based on the historical detection results; and determining the first detection result as the target detection result if the first detection result matches the second detection result.

[0006] According to an embodiment of this disclosure, the target historical image frame is the (t-1)th image frame; based on the target satellite video and historical detection results, the object map of the tth image frame is obtained, including: obtaining the background information of the (t-1)th image frame based on the target satellite video and historical detection results; processing the tth image frame using the inter-frame difference method based on the background information to obtain the object difference map of the tth image frame; and obtaining the object map based on the object difference map of the tth image frame.

[0007] According to an embodiment of this disclosure, background information of the (t-1)th image frame is obtained based on the target satellite video and historical detection results, including: initializing the target initial image frame of the target satellite video to obtain the initial background information of the target satellite video, wherein the target initial image frame is the image frame in the target satellite video before the (t-1)th image frame; updating the initial background information based on the target image frames in the target satellite video between the target initial image frame and the tth image frame and the historical detection results of the target image frames to obtain the background information.

[0008] According to embodiments of this disclosure, the initial background information is updated based on the target image frame between the initial image frame and the t-th image frame in the target satellite video and the historical detection results of the target image frame, to obtain background information, including: obtaining the background frame of the (t-1)-th image frame based on the size information in the historical detection results of the (t-1)-th image frame; and updating the initial background information using the background frame to obtain background information.

[0009] According to embodiments of this disclosure, a first detection result for the target object in the t-th image frame is obtained based on the t-th image frame of the target satellite video, the object map, the target historical image frame, and the historical detection results of the target historical image frame. This includes: performing convolution processing on the target historical image frame, the historical detection results of the target historical image frame, and the t-th image frame to obtain an image frame convolutional map; performing convolution processing on the object map to obtain a convolutional object map; fusing the convolutional object map and the image frame convolutional map to obtain a fused object map, wherein the fused object map includes features of the target object; extracting features from the fused object map to obtain first fused object features; performing channel attention processing on the convolutional object map and the first fused object features to obtain a channel attention map; performing planar spatial attention processing on the convolutional object map to obtain a spatial attention map; fusing the first fused object features, the channel attention map, and the spatial attention map to obtain a target fused object feature map; and detecting the target fused object feature map to obtain the first detection result.

[0010] According to embodiments of this disclosure, channel attention processing is performed on a convolutional object map and a first fused object feature to obtain a channel attention map, including: fusing the convolutional object map and the first fused object feature to obtain a first fused object feature map; performing convolution processing on the first fused object feature map to obtain a second fused object feature map; and performing channel attention processing on the second fused object feature map to obtain a channel attention map.

[0011] According to an embodiment of this disclosure, determining a second detection result for a target object in the t-th image frame based on historical detection results includes: obtaining first position information of the detected object in the t-th image frame based on multiple historical position information of the detected object in the historical detection results, and using the first position information and the historical size information and historical category information of the detected object as the second detection result.

[0012] According to an embodiment of this disclosure, the first detection result of the t-th image frame includes first category information and second location information of the target object; when the first detection result matches the second detection result, determining the first detection result as the target detection result includes: matching the first category information with historical category information to obtain a first matching result; matching the second location information with the first location information to obtain a second matching result; and when the first matching result and the second matching result respectively represent that the first detection result and the second detection result match, determining the first detection result as the target detection result.

[0013] According to embodiments of this disclosure, the above-described target detection method further includes: determining that the target object in the t-th image frame is an object different from the detected object when the first matching result indicates that the first detection result and the second detection result do not match, and the second matching result indicates that the first detection result and the second detection result match.

[0014] A second aspect of this disclosure provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the methods described above.

[0015] The target detection method provided in this embodiment enhances the features of the target object in the t-th image frame by using the object map of the t-th image frame, the target historical image frame, and the historical detection results of the target historical image frame. This avoids missing the target object in the t-th image frame due to weak or small target object features. Furthermore, the first detection result regarding the target object is matched with the second detection result regarding the detected object, further avoiding false detection of the target object. This ensures that the accuracy of the detection result in the t-th image frame meets the requirements for target detection in satellite video. Attached Figure Description

[0016] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0017] Figure 1 This diagram illustrates an application scenario of the target detection method according to an embodiment of the present disclosure.

[0018] Figure 2 A flowchart illustrating a target detection method according to an embodiment of the present disclosure is shown schematically.

[0019] Figure 3 A schematic diagram illustrating the structure of a target detection model according to an embodiment of the present disclosure is shown.

[0020] Figure 4 A schematic diagram illustrating the structure of a priori attention module according to an embodiment of the present disclosure is shown.

[0021] Figure 5 A flowchart illustrating a method for matching a first detection result and a second detection result according to an embodiment of the present disclosure is shown schematically.

[0022] Figure 6 A schematic block diagram of a target detection apparatus according to an embodiment of the present disclosure is shown.

[0023] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a target detection method according to an embodiment of the present disclosure. Detailed Implementation

[0024] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0025] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0026] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0027] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0028] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and there is no violation of public order and good morals.

[0029] In the technical solution disclosed herein, the acquisition, collection, storage, use, processing, transmission, provision, disclosure, and application of data all comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and they do not violate public order and good morals.

[0030] In realizing the inventive concept disclosed herein, the inventors discovered that changes in lighting and other conditions in satellite video can introduce noise and affect the detection of target objects. Therefore, the method of background modeling and foreground detection based on satellite video to detect target objects in satellite video may result in false detection of target objects.

[0031] Deep learning-based detection methods can reduce false positives for target objects, but they require multiple convolutions to obtain deep semantic features. However, due to the low resolution of satellite video, the features of target objects are weak and small. Therefore, performing detection after multiple convolutions may result in missing target object features, leading to missed detections.

[0032] In view of this, embodiments of the present disclosure provide a target detection method, comprising: obtaining an object map in an image frame t based on a target satellite video and historical detection results; obtaining a first detection result of a target object in the image frame t based on the target satellite video image frame t, the object map, a target historical image frame, and historical detection results of the target historical image frame, wherein the target historical image frame is an image frame in the target satellite video; determining a second detection result of a target object in the image frame t based on the historical detection results; and determining the first detection result as the target detection result if the first detection result matches the second detection result.

[0033] Figure 1 The diagram illustrates an application scenario of the target detection method according to an embodiment of the present disclosure.

[0034] like Figure 1As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0035] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 101, 102, and 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0036] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0037] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using terminal devices 101, 102, and 103 (for example only). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0038] It should be noted that the target detection method provided in this embodiment can generally be executed by server 105. Correspondingly, the target detection device provided in this embodiment can generally be located in server 105. The target detection method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the target detection device provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0039] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0040] The following will be based on Figure 1 The described scene, through Figures 2-5 The target detection method of the disclosed embodiments will be described in detail.

[0041] Figure 2 A flowchart illustrating a target detection method according to an embodiment of the present disclosure is shown schematically.

[0042] like Figure 2 As shown, the target detection method in this embodiment includes operations S210 to S240.

[0043] In operation S210, based on the target satellite video and historical detection results, the object map in the t-th image frame is obtained.

[0044] According to embodiments of this disclosure, the target satellite video may include image frames 1 through t. Therefore, image frame t can be obtained from the target satellite video.

[0045] According to embodiments of this disclosure, historical detection results may include detection results for image frames in the target satellite video preceding image frame t. For example, historical detection results may include detection results from image frame 1 to image frame (t-1). Each detection result may include category information, size information, and location information of the detected object.

[0046] According to embodiments of this disclosure, the t-th image frame may include background information and a target object to be detected. The target object may be a moving target object in the target satellite video. Therefore, background information can be obtained based on relatively static areas in the target satellite video.

[0047] According to embodiments of this disclosure, the object map may be, for example, a foreground mask of the target object in the t-th image frame. For example, the approximate area of ​​the target object in the t-th image frame can be determined based on the background information of the t-th image frame, and then the foreground mask can be generated based on that approximate area.

[0048] According to embodiments of this disclosure, an object map in the t-th image frame is obtained based on the target satellite video and historical detection results. For example, this may include: obtaining background information of the target satellite video based on the target satellite video and historical detection results; using the background information of the target satellite video, identifying the area of ​​the t-th image frame excluding the background as the location of the target object; and obtaining the object map of the target object in the t-th image frame based on the location of the target object.

[0049] Specifically, based on the region where the target object is located, the object map of the target object in the t-th image frame is obtained. For example, the region where the target object is located can be given more attention to reduce the chance of missing the target object; or a foreground mask can be generated for the region where the target object is located, and then the region where the target object is located can be detected based on the region corresponding to the foreground mask to reduce the chance of false detection of the target object.

[0050] According to embodiments of this disclosure, since the target objects in the target satellite video are small, generating an object map can enhance the target objects and reduce the chances of missing the target objects.

[0051] In operation S220, based on the t-th image frame of the target satellite video, the object image, the target historical image frame, and the historical detection results of the target historical image frame, the first detection result of the target object in the t-th image frame is obtained, where the target historical image frame is an image frame in the target satellite video.

[0052] According to embodiments of this disclosure, the target historical image frame may be, for example, the (t-1)th image frame of the target satellite video. The historical detection results of the (t-1)th image frame may include, for example, the location information, category information, and size information of the detected objects in the (t-1)th image frame. The first detection result may include the location information, category information, and size information of the target object in the tth image frame.

[0053] Since image frame t-1 and image frame t are adjacent image frames in the target satellite video, the detected object in image frame t-1 and the target object in image frame t may be the same object. If the detected object in image frame t-1 disappears, or if a new target object appears in image frame t, then the target object and the detected object are different objects.

[0054] Since the (t-1)th image frame and the tth image frame are adjacent image frames in the target satellite video, the position of the target object does not change much. Therefore, the corresponding area of ​​the tth image frame can be enhanced according to the area where the detected object is located in the (t-1)th image frame, and then the tth image frame can be detected again to reduce the situation of false detection and missed detection of the target object.

[0055] According to embodiments of this disclosure, a first detection result regarding the target object in the t-th image frame is obtained based on the t-th image frame, the object image, the target historical image frame, and the historical detection results of the target historical image frame. For example, this may include: processing the t-th image frame based on the category information, size information, and position information of the detected object in the historical detection results of the (t-1)-th image frame to enhance the corresponding position of the t-th image frame, thereby enhancing the target object; fusing the object image and the t-th image frame to obtain a fused object image, thereby further enhancing the target object; and detecting the fused object image to obtain the first detection result.

[0056] For example, it could include: inputting the historical detection results of the t-th image frame, the object map, the (t-1)-th image frame, and the (t-1)-th image frame into the object detection model, and outputting a first detection result. The object detection model can be a trained model, and the object detection model can include convolutional units. Specifically, inputting the historical detection results of the t-th image frame, the object map, the (t-1)-th image frame, and the (t-1)-th image frame into the object detection model and outputting a first detection result could, for example, include: inputting the historical detection results of the t-th image frame, the (t-1)-th image frame, and the (t-1)-th image frame into a convolutional unit to obtain a convolutional object map; fusing the object map with the convolutional object map to obtain a fused object map to enhance the target object; and detecting the fused object map to obtain the first detection result.

[0057] In operation S230, based on the historical detection results, a second detection result regarding the target object is determined in the t-th image frame.

[0058] According to embodiments of this disclosure, the second detection result may include the location information, category information, and size information of the target object predicted based on the detected object. If the detected object and the target object in image frame t are the same object, the obtained location information of the detected object in image frame t is the location information of the target object in image frame t. Therefore, the second detection result regarding the target object can be obtained using the size information, category information, and the aforementioned location information of the detected object.

[0059] According to embodiments of this disclosure, a second detection result regarding the target object in the t-th image frame is determined based on historical detection results. For example, this may include: obtaining multiple historical location information of the detected object based on the historical detection results; obtaining the historical trajectory of the detected object based on the multiple historical location information; determining the location information of the detected object in the t-th image frame based on the historical trajectory of the detected object; and obtaining the second detection result based on the location information, the category information of the detected object, and the size information of the detected object.

[0060] In operation S240, if the first detection result matches the second detection result, the first detection result is determined as the target detection result.

[0061] According to embodiments of this disclosure, by matching the first detection result and the second detection result, it can be determined whether the target object and the detected object are the same object.

[0062] If the target object is a new target object that does not exist in the (t-1)th image frame, or if the detected object in the (t-1)th image frame leaves the acquisition range of the target satellite video, the target object and the detected object will not match, which will in turn cause the first detection result to not match the second detection result.

[0063] If the first detection result and the second detection result match, it can be determined that the target object and the detected object are the same object. That is, relative to the (t-1)th image frame, the target object does not belong to the new target object that only appears in the tth image frame, and the detected object in the (t-1)th image frame has not left the acquisition range of the target satellite video.

[0064] According to embodiments of this disclosure, by matching the first detection result and the second detection, it can be determined whether there is a change in the object from the (t-1)th image frame to the tth image frame. If there is no change, it can be determined that there is no missed detection of the target object, and thus the accuracy of the first detection result can be determined to meet the requirements.

[0065] According to embodiments of this disclosure, by using the object map of the t-th image frame, the target historical image frame, and the historical detection results of the target historical image frame to enhance the features of the target object in the t-th image frame, the missed detection of the target object in the t-th image frame is avoided due to the weak or small features of the target object. Furthermore, by matching the first detection result of the target object with the second detection result of the detected object, false detection of the target object is further avoided, so that the accuracy of the detection result of the t-th image frame meets the requirements for target detection in satellite video.

[0066] According to an embodiment of this disclosure, the target historical image frame is the (t-1)th image frame; based on the target satellite video and historical detection results, the object map of the tth image frame is obtained, including: obtaining the background information of the (t-1)th image frame based on the target satellite video and historical detection results; processing the tth image frame using the inter-frame difference method based on the background information to obtain the object difference map of the tth image frame; and obtaining the object map based on the object difference map of the tth image frame.

[0067] According to embodiments of this disclosure, the background information of the (t-1)th image frame may include, for example, the background frame of the (t-1)th image frame. The size information of the detected objects in the (t-1)th image frame can be removed based on the historical detection results of the (t-1)th image frame to obtain the background frame of the (t-1)th image frame.

[0068] According to embodiments of this disclosure, background information of the (t-1)th image frame is obtained based on target satellite video and historical detection results. For example, it may include: obtaining the background frame of the aforementioned t-th image frame using the Vibe algorithm (background modeling and foreground detection algorithm) based on the (t-1)th image frame of the target satellite video and the historical detection results of the (t-1)th image frame.

[0069] According to embodiments of this disclosure, based on background information, the t-th image frame is processed using an inter-frame difference method to obtain an object difference map of the t-th image frame. For example, the t-th image frame and the aforementioned target background frame can be inter-frame differencing to obtain the object difference map of the t-th image frame.

[0070] According to embodiments of this disclosure, an object map is obtained based on the object difference map of the t-th image frame. For example, this may include: generating a foreground mask for the t-th image frame using an image processing operation of erosion and dilation based on the object difference map of the t-th image frame; and obtaining the object map of the target object based on the foreground mask.

[0071] According to embodiments of this disclosure, by obtaining an object map of the target object in the current image frame based on background information, and then enhancing the features of the target object through the object map, the accuracy of the detection results can be improved.

[0072] According to an embodiment of this disclosure, background information of the (t-1)th image frame is obtained based on the target satellite video and historical detection results, including: initializing the target initial image frame of the target satellite video to obtain the initial background information of the target satellite video, wherein the target initial image frame is the image frame in the target satellite video before the (t-1)th image frame; updating the initial background information based on the target image frames in the target satellite video between the target initial image frame and the tth image frame and the historical detection results of the target image frames to obtain the background information.

[0073] According to embodiments of this disclosure, the initial background information may be obtained, for example, using the Vibe algorithm based on the background frames of image frames preceding the target image frame. For instance, a background sample set for the Vibe algorithm can be constructed based on the background frames of the initial target image frame; the background frames of image frames preceding the target image frame can be added to the background sample set; and the initial background information can be output using the Vibe algorithm based on the background sample set.

[0074] According to embodiments of this disclosure, a target initial image frame is initialized in the target satellite video to obtain the initial background information of the target satellite video. For example, a target initial image frame can be selected from the first image frame to the (t-1)th image frame of the target satellite video. The first image frame can be used as the target initial image frame, and the Vibe algorithm can be used to initialize the first image frame to obtain the initial background information of the target satellite video.

[0075] According to embodiments of this disclosure, the background information may also be, for example, a background frame generated using the Vibe algorithm based on the background frame from the initial target image frame to the background frame of the target image frame. The target image frame may, for example, be the (t-1)th image frame.

[0076] According to embodiments of this disclosure, background information is obtained by updating the initial background information based on the target image frames between the initial target image frame and the t-th image frame in the target satellite video and the historical detection results of the target image frames. For example, this may include: obtaining the background frame of the target image frame based on the target image frame and the historical detection results of the target image frames; adding the background frame of each target image frame to a background sample set; and outputting the background information of the target image frame using the Vibe algorithm based on the background sample set. When the background sample set includes the background frames from the 1st image frame to the (t-1)th image frame, the Vibe algorithm can be used to output the background information of the (t-1)th image frame.

[0077] Based on the image frame and the detection result of each image frame that has been detected, the background frame of that image frame can be obtained and then added to the background sample set. Since a new background frame is added to the background sample set after each detection, the background sample set can be updated, thereby enabling frame-by-frame updating of background information.

[0078] Since the information about objects may change in each frame, the Vibe algorithm can be used to obtain a new background frame by combining the background information from each of the background frames that do not contain the detected objects from multiple image frames. This new background frame is then used to determine the background in the t-th image frame, thereby identifying the non-background region as the approximate region of the target object in the t-th image frame, which improves the accuracy of the generated object map.

[0079] According to embodiments of this disclosure, the background frame of the target image frame is obtained by using the target image frame and the historical detection results of the target image frame between the initial image frame and the t-th image frame in the target satellite video. Then, each obtained background frame can be used to update the initial background information, thereby improving the accuracy of the background information and thus improving the accuracy of the object image.

[0080] According to embodiments of this disclosure, the initial background information is updated based on the target image frame between the initial image frame and the t-th image frame in the target satellite video and the historical detection results of the target image frame, to obtain background information, including: obtaining the background frame of the (t-1)-th image frame based on the size information in the historical detection results of the (t-1)-th image frame; and updating the initial background information using the background frame to obtain background information.

[0081] According to embodiments of this disclosure, the background frame of the (t-1)th image frame is obtained based on the size information in the historical detection results of the (t-1)th image frame. For example, this may include: removing the area occupied by the detected object from the target image frame based on the size information in the historical detection results of the target image frame to obtain the background frame of the target image frame. Removing the detected object from the target image frame may, for example, involve removing the pixels of the detected object from the target image frame.

[0082] According to embodiments of this disclosure, background information is obtained by updating initial background information using background frames. For example, the background frames of the target image frame can be used as samples and added to a background sample set, and then the updated background information can be obtained based on the background sample set of the added background frames of the target image frame.

[0083] For example, if the initial target image frame is the first image frame and the t-th image frame is the third image frame, the target image frame can be the second image frame. Therefore, the background sample set constructed using the Vibe algorithm can include only the background frame of the first image frame. Thus, the initial background information can be obtained using the Vibe algorithm based on the background frame of the first image frame.

[0084] The background frame of the second image frame can be added to a background sample set, and then background information can be obtained based on this background sample set to update the initial background information. This updated background information can then be used to generate the object graph of the target object in the third image frame.

[0085] When the initial target image frame is image frame 1, image frame t is image frame 4, and the target image frame can be image frame 3, the background sample set can include the background frames of image frame 1 and image frame 2. The background information obtained from this background sample set can be used as the initial background information. By adding image frame 3 to this background sample set, and then using the background sample set after adding image frame 3, the background information is updated. This updated background information can be used to generate the object graph of the target object in image frame 4.

[0086] According to embodiments of this disclosure, based on historical detection results, detected objects are removed from the target image frame to obtain the background frame of the target image frame. The initial background information is then updated using the background frame, thereby achieving the update of the background information and obtaining an object image with the required accuracy.

[0087] According to embodiments of this disclosure, a first detection result for the target object in the t-th image frame is obtained based on the t-th image frame of the target satellite video, the object map, the target historical image frame, and the historical detection results of the target historical image frame. This includes: performing convolution processing on the target historical image frame, the historical detection results of the target historical image frame, and the t-th image frame to obtain an image frame convolutional map; performing convolution processing on the object map to obtain a convolutional object map; fusing the convolutional object map and the image frame convolutional map to obtain a fused object map, wherein the fused object map includes features of the target object; extracting features from the fused object map to obtain first fused object features; performing channel attention processing on the convolutional object map and the first fused object features to obtain a channel attention map; performing planar spatial attention processing on the convolutional object map to obtain a spatial attention map; fusing the first fused object features, the channel attention map, and the spatial attention map to obtain a target fused object feature map; and detecting the target fused object feature map to obtain the first detection result.

[0088] According to embodiments of this disclosure, a target historical image frame, its historical detection results, and the t-th image frame are convolved to obtain an image frame convolution map. For example, the (t-1)-th image frame, its historical detection results, and the t-th image frame can be input into a convolution unit to obtain the image frame convolution map of the t-th image frame. The historical detection results of the (t-1)-th image frame may include the location information, size information, and category information of the detected objects in the (t-1)-th image frame.

[0089] For example, it may include: obtaining the region where the detected object is located in the (t-1)th image frame based on the position and size information of the detected object in the (t-1)th image frame; and convolving the corresponding region of the tth image frame based on the region where the detected object is located in the (t-1)th image frame to obtain the image frame convolution map of the tth image frame.

[0090] Specifically, based on the location of the detected object in the (-1)th image frame, the corresponding region in the tth image frame is convolved to obtain the image frame convolution map of the tth image frame. For example, it may include: based on the location of the detected object in the (-1)th image frame, obtaining the approximate location of the target object in the tth image frame; and performing convolution processing on the approximate location of the target object to obtain the image frame convolution map of the tth image frame.

[0091] According to embodiments of this disclosure, an object graph is convolved to obtain a convolved object graph. For example, the object graph can be input into a convolution unit to obtain the convolved object graph.

[0092] According to embodiments of this disclosure, by fusing the image frame convolutional graph and the convolutional object graph, it is possible to enhance the target object in the image frame convolutional graph using the convolutional object graph.

[0093] According to embodiments of this disclosure, channel attention processing is performed on the convolutional object map and the first fused object feature to obtain a channel attention map. For example, this may include: fusing the convolutional object map and the first fused object feature to obtain a second fused object feature; and performing channel attention processing on the second fused object feature map according to the output channel to obtain a channel attention map. The channel attention processing may, for example, involve multiplying the feature value of each feature in the second fused image object feature map by a preset value.

[0094] According to embodiments of this disclosure, planar spatial attention processing may, for example, involve multiplying the width and height of each pixel corresponding to a feature in the convolutional object image by a preset value.

[0095] According to embodiments of this disclosure, the first fusion object features, the channel attention map, and the spatial attention map are fused. For example, the features of each of the channel attention map, the spatial attention map, and the first fusion object features may be multiplied together according to their corresponding positions.

[0096] According to embodiments of this disclosure, for example, the convolutional object map and the first fused object features can be input into a prior attention module, and the output channel attention map and spatial attention map can be obtained. The convolutional object map can be a feature prior of the object detection model, and the prior attention module can be a module that has pre-stored prior knowledge. By utilizing the prior knowledge of the prior attention module, the convolutional object map and the first fused object features can be processed to obtain the channel attention map and spatial attention map.

[0097] According to embodiments of this disclosure, the first detection result may, for example, include first size information, first category information, and second position information of the target object in the t-th image frame. The first category information may include a category heatmap, which can be used to identify the category of the target object. The first size information may include the length and width of the target object, which can be represented by an outer rectangle added during the detection process. The second position information may include the position of the target object in the t-th image frame, the offset correction information of the target object's center point due to downsampling, and the motion displacement information of the target object relative to the (t-1)-th image frame.

[0098] According to embodiments of this disclosure, the (t-1)th image frame, first position information, first size information, and first category information are convolved with the t-th image frame to enhance the features of the target object in the t-th image frame. Convolution is also performed on the object map to enhance the features of the target object in the object map. The convolutional image map and the convolutional object map are then fused to further enhance the features of the target object in the image frame. Feature extraction is then performed on the fused object map to obtain the enhanced first fused object feature. Channel attention processing is then applied to the convolutional object map and the first fused object feature to obtain a channel attention map. Planar spatial attention processing is then applied to the convolutional object map to obtain a spatial attention map. Finally, the first fused object feature, the channel attention map, and the spatial attention map are fused to obtain the target fused object feature map. This process enhances the features of the target object to meet the requirements of satellite video detection, improving the accuracy of the target satellite video detection results.

[0099] Figure 3 The schematic diagram illustrates the structure of a target detection model according to an embodiment of the present disclosure.

[0100] like Figure 3As shown, the object image 310, the t-th image frame 320, the (t-1)-th image frame 330, and the historical detection results 340 of the (t-1)-th image frame can be input into the target detection model. Specifically, the object image 310 can be input into the convolutional unit 3501, which outputs a convolutional object image; the historical detection results 340 of the t-th image frame 320, the (t-1)-th image frame 330, and the (t-1)-th image frame can be input into the convolutional unit 3502, which outputs an image frame convolutional image of the t-th image frame 320; the convolutional object image and the image frame convolutional image are fused to obtain a fused object image; the fused object image is input into the feature extraction unit 3504, which outputs a first fused object feature; the convolutional object image and the first fused object feature are input into the prior attention module 3503, which outputs a channel attention map and a spatial attention map, where the convolutional object image can serve as a feature prior for the target detection model; the first fused object feature, the spatial attention map, and the channel attention map are fused to obtain a target fused object feature map; the target fused object feature map is then detected to obtain the target fused object feature map. The system includes three data points: a first category information, a first size information, and a second position information. The first category information can be input to output unit 3505, and the first category information can be output. The first size information can be input to output unit 3506, and the first size information can be output. The second position information can be input to output unit 3507, and the second position information can be output. The first category information can include a category heatmap of the target object. The first size information can include the offset correction of the target object's center point due to downsampling and the displacement of the target object relative to the previous frame. The first size information can include the length and width of the target object's bounding rectangle. Convolution units 3501 and 3502 can both be 7×7 convolution units. Output units 3505, 3506, and 3507 can each include a 3×3 convolution kernel.

[0101] According to embodiments of this disclosure, channel attention processing is performed on a convolutional object map and a first fused object feature to obtain a channel attention map, including: fusing the convolutional object map and the first fused object feature to obtain a first fused object feature map; performing convolution processing on the first fused object feature map to obtain a second fused object feature map; and performing channel attention processing on the second fused object feature map to obtain a channel attention map.

[0102] According to embodiments of this disclosure, channel attention processing is performed on the second fused object feature map to obtain a channel attention map. For example, the feature value of each feature in the second fused object feature map may be multiplied by a preset value.

[0103] According to embodiments of this disclosure, by fusing the convolutional object graph and the first fused object features, the features of the target object in the convolutional object graph can be enhanced. Then, by performing convolution processing and channel attention processing on the second fused object features, the features of the target object can be enhanced.

[0104] Figure 4 A schematic diagram of the structure of a priori attention module according to an embodiment of the present disclosure is shown.

[0105] like Figure 4 As shown, the first fusion object feature 401 can be input into the convolutional unit 406 to output the first convolutional fusion object feature, wherein the first convolutional fusion object feature can be a three-dimensional image, i.e., C×H×W; the convolutional object map 403 can be input into the convolutional unit 405 to output the first enhanced convolutional object map; the first convolutional fusion object feature and the first enhanced convolutional object map can be fused to output the first fusion object feature map; the first fusion object feature map is input into the convolutional unit 407 to output the second fusion object feature map; channel attention processing is performed on the second fusion object feature map to obtain the intermediate channel attention map, wherein the channel attention processing can be performed on the C dimension; the intermediate channel attention map can be enhanced to obtain the channel attention map, wherein the enhancement processing... The process can involve multiplying the feature value of each feature in the intermediate channel attention map by a preset value; inputting the convolutional object map 403 into the convolutional unit 404 to output the second enhanced convolutional object map; performing planar spatial attention processing on the second enhanced convolutional object map to obtain an intermediate spatial attention map, wherein the planar spatial attention processing can be performed on the two dimensions H×W; performing enhancement processing on the intermediate spatial attention map to obtain a spatial attention map, wherein the enhancement processing can be multiplying the feature value of each feature in the intermediate spatial attention map by a preset value; fusing the spatial attention map and the first fusion object feature to obtain a third fusion object feature map; and fusing the third fusion object feature map and the channel attention map to obtain the target fusion object feature map 408.

[0106] Among them, convolutional units 404, 405, 406 and 407 can all be convolutional units with a kernel of 3×3; convolutional unit 404 can include a sigmoid activation function; convolutional unit 405 can include batch normalization and ReLU activation functions; convolutional unit 406 can include batch normalization and ReLU activation functions; convolutional unit 407 can include a sigmoid activation function.

[0107] According to an embodiment of this disclosure, determining a second detection result for a target object in the t-th image frame based on historical detection results includes: obtaining first position information of the detected object in the t-th image frame based on multiple historical position information of the detected object in the historical detection results, and using the first position information and the historical size information and historical category information of the detected object as the second detection result.

[0108] According to embodiments of this disclosure, for example, based on historical detection results, historical location information, historical size information, and historical category information of the detected object can be obtained.

[0109] According to embodiments of this disclosure, a first position information of the detected object in the t-th image frame is obtained based on multiple historical position information of the detected object in historical detection results. For example, this may include: obtaining the historical trajectory of the detected object based on multiple historical position information of the detected object in the target satellite video; and predicting the second position information of the detected object in the t-th image frame using a Kalman filter method based on the historical trajectory of the detected object.

[0110] According to embodiments of this disclosure, in some embodiments, a second detection result can be obtained by utilizing multiple historical category information and multiple historical size information of the detected object. When the detected object is located in different image frames, the pixels of the detected object may change, which may cause changes in the category heatmap and the size of the detected object. Therefore, obtaining the second detection result not only based on the first size information and the first category information, but also based on multiple historical category information and multiple historical size information of the detected object, can avoid the problem of decreased matching accuracy caused by changes in the pixels of the detected object, and helps to improve the accuracy of the subsequent matching result obtained by matching the first detection result and the second detection result.

[0111] According to an embodiment of this disclosure, the first detection result of the t-th image frame includes first category information and second location information of the target object; when the first detection result matches the second detection result, determining the first detection result as the target detection result includes: matching the first category information with historical category information to obtain a first matching result; matching the second location information with the first location information to obtain a second matching result; and when the first matching result and the second matching result respectively represent that the first detection result and the second detection result match, determining the first detection result as the target detection result.

[0112] According to embodiments of this disclosure, for example, the first category information can be matched with historical category information to obtain a first matching result; the second location information and the first location information can be matched using the Hungarian matching algorithm to obtain a second matching result.

[0113] According to embodiments of this disclosure, by matching the first category information and the historical category information, and then matching the first location information and the second location information, the change of the target object between the t-th image frame and the (t-1)-th image frame can be determined. Thus, if the target object is determined to be unchanged, the accuracy of the first detection result can be determined to meet the requirements.

[0114] According to an embodiment of this disclosure, the above-described target detection method further includes: determining that the target object in the t-th image frame is an object different from the detected object when the first matching result indicates that the first detection result and the second detection result do not match, and the second matching result indicates that the first detection result and the second detection result match.

[0115] According to embodiments of this disclosure, if a target object exists in the (t+m-1)th image frame, which is m frames apart from the (t-1)th image frame, and matches a detected object in the (t-1)th image frame, it can be determined that the detected object has reappeared, and the target object can be associated with the detected object, where m is an integer greater than 2. The target object is the object that has already been identified as a target object, meaning it has a response in the object graph of the (t+m-1)th image frame.

[0116] Before determining whether the detected object has reappeared, the process may further include: matching the category information of the target object appearing in image frames after image frame t with the historical category information of the detected object; and using a Hungarian matching algorithm to match the position of the target object with the historical trajectory of the detected object obtained using a Kalman filter. If both the category information and the historical trajectory match, it can be determined that the detected object has reappeared.

[0117] For example, to save memory, m can be set to be less than the preset frame number, which can be 30 frames. That is, if no target object matching the above-mentioned detected object appears within 30 frames, the above-mentioned matching of the detected object is abandoned.

[0118] In this process, the target object is associated with the detected object. For example, the historical trajectory of the detected object can be updated according to the location of the target object, and the category information and size information of the target object can be used as the category information and size information of the detected object.

[0119] According to embodiments of this disclosure, the Kalman filtering method can be used to supplement the missing trajectory between the detected object and the target object that matches the detected object in the (t-1)th image frame.

[0120] According to embodiments of this disclosure, when the first matching result indicates that the first detection result matches the second detection result, and the second matching result indicates that the first detection result does not match the second detection result, it is determined that the t-th image frame includes a new object other than the already detected object, and a new trajectory can be created for the new target object.

[0121] Determining that the t-th image frame includes a new object other than the already detected objects can include: determining whether the confidence level of the new object is greater than a preset threshold; if the confidence level is greater than the preset threshold, determining the new object as a new target object. The location information of the new target object can be saved to create a new trajectory for that new target object.

[0122] According to embodiments of this disclosure, the historical trajectory of a target object can be updated based on the second location information in the target detection result. The historical trajectory of the target object can be the historical trajectory of a detected object that matches the target object.

[0123] Figure 5 A flowchart illustrating a method for matching a first detection result and a second detection result according to an embodiment of the present disclosure is shown.

[0124] like Figure 5 As shown, the matching method for the first detection result and the second detection result in this embodiment includes operations S510 to S560.

[0125] In operation S510, the first category information is matched with the historical category information to obtain a first matching result; the first location information and the second location information are matched to obtain a second matching result. If the first matching result and the second matching result both indicate a match, operation S520 is executed; if the first matching result indicates a match but the second matching result indicates a mismatch, operation S530 is executed; if the second matching result indicates a match but the first matching result indicates a mismatch, operation S560 is executed.

[0126] When operating S520, the first detection result is used as the target detection result.

[0127] In operation S530, it is determined that a new object exists. If the confidence level of the new object is greater than a preset threshold, operation S540 is executed.

[0128] In operation S540, the new target object is identified.

[0129] According to embodiments of this disclosure, when the confidence level is less than a preset threshold, the new object feature can be processed as needed, without limitation.

[0130] According to embodiments of this disclosure, the preset threshold can be, for example, 0.3 or 0.4, and can be adjusted as needed.

[0131] In operation S550, the target object in the t-th image frame is determined to be an object different from the already detected objects.

[0132] Based on the above-described target detection method, this disclosure also provides a target detection device. The following will be combined with... Figure 6 The device is described in detail.

[0133] Figure 6 A schematic block diagram of a target detection apparatus according to an embodiment of the present disclosure is shown.

[0134] like Figure 6 As shown, the target detection device 600 in this embodiment includes a first acquisition module 610, a second acquisition module 620, a first determination module 630, and a second determination module 640.

[0135] The first acquisition module 610 is used to obtain the object map in the t-th image frame based on the target satellite video and historical detection results. In one embodiment, the first acquisition module 610 can be used to perform the operation S210 described above, which will not be repeated here.

[0136] The second acquisition module 620 is used to obtain a first detection result about the target object in the t-th image frame based on the t-th image frame of the target satellite video, the object image, the target historical image frame, and the historical detection results of the target historical image frame, wherein the target historical image frame is an image frame in the target satellite video. In one embodiment, the third acquisition module 620 can be used to perform the operation S220 described above, which will not be repeated here.

[0137] The first determining module 630 is used to determine a second detection result regarding the target object in the t-th image frame based on historical detection results. In one embodiment, the first determining module 630 can be used to perform the operation S230 described above, which will not be repeated here.

[0138] The second determining module 640 is used to determine the first detection result as the target detection result when the first detection result matches the second detection result. In one embodiment, the second determining module 640 can be used to perform the operation S240 described above, which will not be repeated here.

[0139] It should be noted that the target detection device mentioned above corresponds to the target detection method mentioned above. The target detection device may include modules, units, sub-units, etc., used to implement all the functions of the target detection method involved in the flowchart above. For the sake of brevity, these will not be elaborated here. For a detailed description, please refer to the description of the target detection method above.

[0140] According to embodiments of this disclosure, any plurality of modules among the first acquisition module 610, the second acquisition module 620, the first determination module 630, and the second determination module 640 may be combined into one module, or any one of these modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the first acquisition module 610, the second acquisition module 620, the first determination module 630, and the second determination module 640 may be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in a suitable combination of any of these. Alternatively, at least one of the first acquisition module 610, the second acquisition module 620, the first determination module 630, and the second determination module 640 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0141] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a target detection method according to an embodiment of the present disclosure.

[0142] like Figure 7 As shown, an electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage portion 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0143] RAM 703 stores various programs and data required for the operation of electronic device 700. Processor 701, ROM 702, and RAM 703 are interconnected via bus 704. Processor 701 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 702 and / or RAM 703. It should be noted that the programs may also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0144] According to embodiments of this disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to a bus 704. The electronic device 700 may also include one or more of the following components connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0145] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0146] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 702 and / or RAM 703 and / or one or more memories other than ROM 702 and RAM 703 described above.

[0147] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code enables the computer system to implement the target detection method provided in the embodiments of this disclosure.

[0148] When the computer program is executed by the processor 701, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0149] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 709, and / or installed from a removable medium 711. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0150] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0151] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0152] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0153] It should be noted that, unless it is explicitly stated that there is a sequential order of execution between different operations, or that there is a sequential order of execution between different operations in terms of technical implementation, the execution order between multiple operations may not be significant, and multiple operations may be executed simultaneously.

[0154] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0155] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A target detection method, comprising: Based on the target satellite video and historical detection results, the object map in the t-th image frame is obtained; Based on the t-th image frame of the target satellite video, the object image, the target historical image frame, and the historical detection results of the target historical image frame, a first detection result about the target object in the t-th image frame is obtained, wherein the target historical image frame is an image frame in the target satellite video; Based on the historical detection results, a second detection result regarding the target object is determined in the t-th image frame; If the first detection result matches the second detection result, the first detection result is determined to be the target detection result; The step of obtaining the first detection result of the target object in the t-th image frame based on the t-th image frame of the target satellite video, the object map, the target historical image frame, and the historical detection results of the target historical image frame includes: The target historical image frame, the historical detection result of the target historical image frame, and the t-th image frame are convolved to obtain an image frame convolution map. The object graph is convolved to obtain a convolved object graph; The convolutional object map and the image frame convolutional map are fused to obtain a fused object map, wherein the fused object map includes the features of the target object; Feature extraction is performed on the fused object graph to obtain the first fused object feature; Channel attention processing is performed on the convolutional object map and the features of the first fused object to obtain a channel attention map; Perform planar spatial attention processing on the convolutional object graph to obtain a spatial attention graph; The first fusion object feature, the channel attention map, and the spatial attention map are fused to obtain the target fusion object feature map; The feature map of the target fusion object is detected to obtain the first detection result.

2. The method according to claim 1, wherein, The target historical image frame is the (t-1)th image frame; The process of obtaining the object map of the t-th image frame based on the target satellite video and the historical detection results includes: Based on the target satellite video and the historical detection results, the background information of the (t-1)th image frame is obtained; Based on the background information, the t-th image frame is processed using the inter-frame difference method to obtain the object difference map of the t-th image frame; The object map is obtained based on the object difference map of the t-th image frame.

3. The method according to claim 2, wherein, The background information of the (t-1)th image frame obtained based on the target satellite video and the historical detection results includes: Initialize the target initial image frame of the target satellite video to obtain the initial background information of the target satellite video, wherein the target initial image frame is the image frame in the target satellite video before the (t-1)th image frame; Based on the target image frames in the target satellite video between the initial image frame and the t-th image frame, and the historical detection results of the target image frames, the initial background information is updated to obtain the background information.

4. The method according to claim 3, wherein, The step of updating the initial background information based on the target image frames in the target satellite video between the initial image frame and the t-th image frame, and the historical detection results of the target image frames, to obtain the background information includes: The background frame of the (t-1)th image frame is obtained based on the size information in the historical detection results of the (t-1)th image frame. The initial background information is updated using the background frame to obtain the background information.

5. The method according to claim 1, wherein, The step of performing channel attention processing on the convolutional object map and the first fused object features to obtain a channel attention map includes: The convolutional object map and the first fused object feature are fused to obtain the first fused object feature map; The first fused object feature map is convolved to obtain the second fused object feature map; The second fusion object feature map is subjected to channel attention processing to obtain the channel attention map.

6. The method according to claim 1, wherein, The step of determining the second detection result of the target object in the t-th image frame based on the historical detection results includes: Based on the historical location information of the detected object in the historical detection results, the first location information of the detected object in the t-th image frame is obtained, and the first location information and the historical size information and historical category information of the detected object are used as the second detection result.

7. The method according to claim 6, wherein, The first detection result of the t-th image frame includes the first category information and the second location information of the target object; The step of determining the first detection result as the target detection result when the first detection result matches the second detection result includes: The first category information is matched with the historical category information to obtain a first matching result; A second matching result is obtained by matching the second location information with the first location information; If the first matching result and the second matching result respectively indicate that the first detection result and the second detection result match, then the first detection result is determined to be the target detection result.

8. The method according to claim 7, further comprising: If the first matching result indicates that the first detection result and the second detection result do not match, and the second matching result indicates that the first detection result and the second detection result match, then the target object in the t-th image frame is determined to be an object different from the detected object.

9. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 8.

Images