Aircraft trajectory processing method and astronomical image restoration method, electronic device and computer program product

Abstract

The application provides an aircraft trajectory processing method and an astronomical image repairing method, an electronic device and a computer program product. The aircraft trajectory processing method comprises: acquiring a target image to be processed; inputting the target image to be processed into a multi-stage cascaded trajectory detection model to obtain an aircraft trajectory detection result output by the multi-stage cascaded trajectory detection model; if the result is an aircraft trajectory, inputting the target image to be processed into a trajectory extraction model to obtain a trajectory extraction image output by the trajectory extraction model; wherein the trajectory extraction image is an image only having the aircraft trajectory in the target image to be processed; and generating a target image without the aircraft trajectory according to the target image to be processed and the trajectory extraction image. Through the multi-stage cascaded trajectory detection mechanism, the false detection probability of the aircraft trajectory under a complex astronomical background is significantly reduced, the celestial body features such as star orbits and nebulae are avoided from being misjudged as the aircraft trajectory, and the accuracy of trajectory detection is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to a method for processing aircraft trajectories and astronomical image restoration, electronic equipment, and computer program products. Background Technology

[0002] Spacecraft tracks in astronomical images (including bright lines and condensation trails of the spacecraft itself) are one of the most common transient noises in ground-based astronomical observations, which can seriously affect the accuracy of astrophotometry, astrometry, and scientific analysis.

[0003] Currently, traditional methods for removing spacecraft tracks from astronomical images generally involve manually identifying the track location and then manually using professional image processing software to fill or replace the area covered by the track with surrounding uncontaminated pixel values. However, this manual track removal method suffers from problems such as long processing time and low repair quality. Summary of the Invention

[0004] The purpose of this application is to provide a method for processing aircraft trajectories, a method for restoring astronomical images, an electronic device, and a computer program product, in order to solve the problems of long time consumption and low restoration quality in current traditional methods for removing aircraft trajectories from astronomical images by manually identifying them.

[0005] In a first aspect, this application provides an aircraft trajectory processing method, which includes acquiring a target image to be processed; inputting the target image to be processed into a pre-trained multi-level cascaded trajectory detection model to obtain an aircraft trajectory detection result output by the multi-level cascaded trajectory detection model; if the aircraft trajectory detection result indicates the presence of an aircraft trajectory, then inputting the target image to be processed into a pre-trained trajectory extraction model to obtain a trajectory extraction image output by the trajectory extraction model; wherein, the trajectory extraction image is an image containing only the aircraft trajectory in the target image to be processed; and generating a target image with the aircraft trajectory removed based on the target image to be processed and the trajectory extraction image.

[0006] The spacecraft trajectory processing method provided in this solution first filters out trajectory-free images through a pre-stage multi-level cascaded trajectory detection model. Then, a pre-trained trajectory extraction model extracts the trajectory features that were originally mixed in with the starry background, achieving separation of the spacecraft trajectory from the starry background. Finally, the original image and an image containing only the spacecraft trajectory are used to generate a target image with the spacecraft trajectory removed. In this way, this solution significantly reduces the false detection probability of spacecraft trajectories in complex astronomical backgrounds through a multi-level cascaded trajectory detection mechanism, avoiding misidentification of celestial features such as star trails and nebulae as spacecraft trajectories, and improving the accuracy of trajectory detection. At the same time, the trajectory features extracted by the trajectory extraction model are used for precise restoration, rather than a complete reconstruction of the entire image. This ensures the thoroughness of trajectory removal while preserving the details of stars, nebulae, etc. in the original astronomical image to the greatest extent, thereby improving the restoration effect of the astronomical image.

[0007] In an optional implementation of the first aspect, the target image to be processed is input into a pre-trained multi-level cascaded trajectory detection model to obtain the aircraft trajectory detection result output by the multi-level cascaded trajectory detection model, including: sequentially inputting the target image to be processed into each level of the pre-trained multi-level cascaded trajectory detection model to obtain the trajectory detection result output by each level of the trajectory detection model; and determining that the aircraft trajectory detection result is that there is an aircraft trajectory when all the trajectory detection results output by the corresponding trajectory detection models are aircraft trajectories.

[0008] In the above implementation method, this solution only determines that there is an aircraft trajectory when all trajectory detection results output by the multi-level cascaded trajectory detection models show an aircraft trajectory. It adopts a method of independent verification of multiple trajectories, thereby avoiding misjudgments caused by noise, artifacts, and star trails and nebulae in astronomical images due to single models, and improving the accuracy of trajectory detection.

[0009] In an optional implementation of the first aspect, the target image to be processed is input into a pre-trained trajectory extraction model to obtain a trajectory extraction image output by the trajectory extraction model, including: performing layer-by-layer feature extraction and spatial resolution compression on the target image to be processed through the encoder of the trajectory extraction model, and capturing the global context information of the target image to be processed through dilated convolution to obtain a deep encoded feature map output by the encoder; performing upsampling reconstruction and fusing with the shallow detail feature map of the encoder at the corresponding level through the decoder of the trajectory extraction model layer by layer in the order from deep to shallow to obtain a fused feature map corresponding to each level; outputting a trajectory residual prediction map of the corresponding level based on the fused feature map of each level through the residual module corresponding to each level, until the output trajectory residual prediction map has the same resolution as the target image to be processed, wherein the trajectory residual prediction map is obtained by adding the residual brightness features of the trajectory pixels in the fused feature map and the original brightness features of each pixel; and performing noise removal on the trajectory residual prediction map with the same resolution as the image to be processed to obtain the trajectory extraction image output by the trajectory extraction model.

[0010] In the above implementation, the encoder of this scheme expands the receptive field of the network through dilated convolution, enabling the network to perceive the global linear features of the entire aircraft trajectory. This avoids perceptual misjudgment caused by insufficient receptive field, which may lead to confusion between local trajectory features and stars or nebulae, thus improving the trajectory extraction accuracy of the trajectory extraction model. Simultaneously, this scheme designs skip connections to supplement the shallow detail features obtained by the encoder during the downsampling stage, avoiding detail loss due to downsampling. This ensures that the original image details are not lost while achieving aircraft trajectory extraction. Furthermore, this scheme designs a residual learning mechanism that allows the neural network model to focus on learning the difference information of the aircraft trajectory region. This enables it to automatically focus on the aircraft trajectory region that needs repair, adjusting only the aircraft trajectory region while leaving the original star points, nebulae, and other detailed background regions unchanged. This effectively avoids the blurring or structural damage to the overall image caused by traditional methods, thus enabling the designed trajectory extraction model to accurately and completely extract the aircraft trajectory.

[0011] In an optional implementation of the first aspect, the training process of the trajectory extraction model includes: acquiring a target training image set; wherein the target training image set includes multiple training image sample pairs, each training image pair including an original image sample and a trajectory image sample corresponding to the original sample, the trajectory image sample being a binary label image corresponding to the original image sample, the binary label image representing an image in which all pixels covered by the aircraft trajectory in the image are labeled with a first preset value, and all pixels in the remaining part of the image are labeled with a second preset value; training the deep learning model to be trained according to the target training image set until a trained trajectory extraction model is obtained.

[0012] The above implementation method uses pixel-level binary labels for supervised training, which enables the model to learn the precise boundaries and shape of the trajectory. The output trajectory extraction image has accurate pixel-level positioning with small errors.

[0013] In an optional implementation of the first aspect, training the deep learning model to be trained based on the target training image set until a trained trajectory extraction model is obtained includes: extracting a batch of training image sample pairs from the target training sample set and inputting the training image sample pairs into the deep learning model to be trained; performing layer-by-layer feature extraction and spatial resolution compression on the input original image samples through the encoder of the deep learning model, and capturing the global context information of the input original image samples through dilated convolution to obtain the deep encoded feature map corresponding to the original image sample; performing upsampling reconstruction and fusing with the shallow detail feature map of the encoder of the corresponding layer through the decoder of the deep learning model layer by layer in the order from deep to shallow to obtain fused feature image samples; and outputting the trajectory residual prediction sample map of the corresponding layer based on the fused feature image sample of each layer through the residual module corresponding to each layer until the resolution of the output trajectory residual prediction sample map is consistent with that of the original image sample, wherein the trajectory residual prediction sample map is based on the fused feature image sample. The residual brightness features of trajectory pixels and the original brightness features of each pixel are added together to obtain the trajectory. Noise removal is performed on the trajectory residual prediction sample map with the same resolution as the original image sample to obtain the prediction segmentation map corresponding to the original image sample. The value of each pixel in the prediction segmentation map represents the probability that the pixel belongs to the aircraft trajectory. The prediction segmentation map and the trajectory image samples corresponding to the original image sample are input into a preset loss function to calculate the loss value between the prediction result and the true label. It is determined whether the loss value reaches the target loss value or whether the model iteration count reaches the target number. If the loss value does not reach the target loss value or the model iteration count does not reach the target number, backpropagation is performed based on the loss value between the prediction result and the true label, and the model parameters are updated. The process then returns to the step of extracting a batch of training image sample pairs from the target training sample set and inputting the training image sample pairs into the deep learning model. If the loss value reaches the target loss value or the model iteration count reaches the target number, the trained trajectory extraction model is obtained.

[0014] In the above implementation scheme, the encoder training method expands the receptive field of the network through dilated convolution, enabling the trained model to perceive the global linear features of the entire aircraft trajectory. This avoids perceptual misjudgment caused by insufficient receptive field, which may lead to confusion between local trajectory features and stars or nebulae, thus improving the trajectory extraction accuracy of the trained model. Simultaneously, this scheme designs skip connections to supplement the shallow detail features obtained by the encoder during the downsampling stage, allowing the model to learn how to accurately identify trajectory regions while preserving background details. Furthermore, this scheme designs a residual learning mechanism that allows the neural network model to focus on learning the difference information of the aircraft trajectory regions. This enables it to automatically focus on the aircraft trajectory regions that need repair, adjusting only the aircraft trajectory regions while leaving the original star points, nebulae, and other detailed background regions unchanged. This effectively avoids the blurring or structural damage to the overall image caused by traditional methods, thus enabling the designed trajectory extraction model to accurately and completely extract the aircraft trajectory. Moreover, this scheme uses dual termination conditions—loss value and iteration count—to precisely control the training accuracy of the model, avoiding underfitting and overfitting.

[0015] In an optional implementation of the first aspect, generating a target image with the aircraft trajectory removed based on the target image to be processed and the trajectory extraction image includes: if the trajectory extraction image is a binary trajectory mask, determining the trajectory region and effective background region of the target image to be processed based on the mask pixel values ​​of the trajectory extraction image; using the binary trajectory mask as a guiding mask, reconstructing the background of the trajectory region to generate a repaired background region corresponding to the trajectory region, wherein the repaired background region has the same size as the trajectory region; and stitching the repaired background region and the effective background region together to generate a target image with the aircraft trajectory removed.

[0016] The above implementation divides the target image to be processed into a trajectory region and an effective background region by using a mask value. Background reconstruction is performed only on the trajectory region, which has a small computational load and high processing efficiency. At the same time, the effective background region completely retains the pixel values ​​of the original image. In this way, the effective background completely retains the original data, and only the trajectory region is replaced with the repaired background region. This achieves both trajectory removal and maximum preservation of the original photometric and astronomical measurement information of the original astronomical image.

[0017] In an optional implementation of the first aspect, generating a target image with the aircraft trajectory removed based on the target image to be processed and the trajectory extraction image includes: if the trajectory extraction image is a trajectory probability map, wherein the value of each pixel in the trajectory probability map represents the confidence that the corresponding position belongs to the aircraft trajectory; determining the trajectory weight and background weight corresponding to each pixel value based on the confidence of each pixel value in the trajectory probability map, wherein the trajectory weight is the confidence that the corresponding pixel value belongs to the aircraft trajectory, and the background weight is 1 minus the calculated value of the trajectory weight; generating a target image with the trajectory removed based on the effective background pixels in the preset width on both sides of the trajectory in the target image to be processed. A background reconstruction image with the same size as the target image to be processed is generated. Valid background pixels represent pixels with a confidence level lower than a preset confidence threshold. The background reconstruction image contains the background reconstruction pixel value corresponding to each pixel position. The product of each background reconstruction pixel value and the trajectory weight corresponding to its position is calculated to obtain a first weighted value. The product of the original pixel value corresponding to each background reconstruction pixel value and the background weight corresponding to its position is calculated to obtain a second weighted value. The sum of the first weighted value and the second weighted value corresponding to each pixel position is calculated to obtain the fused pixel value of each pixel in the target image to be processed, generating a target image with the aircraft trajectory removed.

[0018] In the above implementation method, the probability map can accurately represent the proportion of trajectory components of each pixel. By weighting, different degrees of trajectory pollution can be removed proportionally. This further improves the effectiveness of aircraft trajectory removal for condensed trails with blurred edges and gradual brightness changes.

[0019] In an optional implementation of the first aspect, generating a target image with the aircraft trajectory removed based on the target image to be processed and the trajectory extraction image includes: if the trajectory extraction image is a trajectory residual map, the trajectory residual map represents the pixel brightness component of the aircraft trajectory, and the pixel value of the background area approaches zero; calculating the difference between each pixel value in the target image to be processed and the trajectory residual map to generate a target image with the aircraft trajectory removed.

[0020] In the above implementation method, the residual learning mechanism enables the model to output only the brightness difference of the trajectory region, with no residual in the background region. After subtraction, the background pixels are completely consistent with the original image, without any blurring or loss of detail. At the same time, the model learns the true brightness difference between the trajectory and the background. After subtraction, the original brightness of the background can be accurately restored. The model can adaptively remove trajectories with different brightness, without over-repairing or under-repairing.

[0021] Secondly, this application provides an astronomical image restoration method, which includes: acquiring an astronomical image to be processed; wherein the astronomical image to be processed is obtained by stretching and normalizing a preprocessing step on an original astronomical image; segmenting the astronomical image to be processed according to a preset segmentation size to obtain multiple image blocks; wherein each image block corresponds one-to-one with its spatial position in the astronomical image to be processed; sequentially selecting a target image block from the multiple image blocks according to the target scanning order and performing the spacecraft trajectory processing method of any optional embodiment of the first aspect until a target image corresponding to each image block is obtained; wherein if the spacecraft trajectory detection result is no spacecraft trajectory, the obtained target image is the corresponding image block itself; generating a stitched image corresponding to each image block based on the target image corresponding to each image block; stitching the stitched images corresponding to all image blocks according to their corresponding spatial positions to generate a target astronomical image corresponding to the astronomical image to be processed.

[0022] The astronomical image restoration method provided in this solution employs the spacecraft trajectory processing method described earlier to perform trajectory detection and removal on each image block. Therefore, the designed astronomical image restoration method can significantly reduce the false detection probability of spacecraft trajectories in complex astronomical backgrounds through a multi-level cascaded trajectory detection mechanism, avoiding misidentification of star trails, nebulae, and other celestial features as spacecraft trajectories, thus improving the accuracy of trajectory detection. Simultaneously, it uses trajectory features extracted by the trajectory extraction model for precise restoration, rather than a complete reconstruction of the entire image. This ensures thorough trajectory removal while preserving the details of stars, nebulae, etc., in the original astronomical image to the greatest extent, thereby improving the restoration effect. Furthermore, this solution uses block processing to break down large astronomical images into smaller blocks supported by the model. This allows the astronomical image restoration method designed in this solution to run on embedded devices with limited computing power (such as the built-in processor of an astronomical camera), without needing to upload to the cloud for processing, improving the real-time performance of the astronomical image restoration method.

[0023] In an optional embodiment of the second aspect, the astronomical image to be processed is segmented according to a preset segmentation size to obtain multiple image blocks, including: sliding sequentially along the horizontal and vertical directions on the astronomical image to be processed with a preset sliding step size of a number of segmentation pixels to generate multiple overlapping image blocks; wherein, the number of pixels of the overlap width of adjacent image blocks in the horizontal and vertical directions is 1 / n of the number of segmentation pixels of the sliding step size, where n is greater than or equal to 2.

[0024] In the above implementation method, an overlapping area is set during the segmentation of image blocks. The setting of the overlapping area ensures that after the cropping edge, adjacent blocks still have overlapping parts, which can be fused to achieve seamless stitching and avoid stitching gaps.

[0025] In an optional implementation of the second aspect, before generating multiple overlapping image blocks, the method further includes: determining whether the width and height of the astronomical image to be processed are both integer multiples of a preset number of segmented pixels; if it is determined that the width and height of the astronomical image to be processed are not both integer multiples of a preset number of segmented pixels, then pixel filling is performed on the edges of the astronomical image to be processed to obtain a pixel-filled astronomical image to be processed; wherein the height and width of the pixel-filled astronomical image to be processed are both integer multiples of a preset number of segmented pixels.

[0026] In the above implementation method, if the width and height of the astronomical image to be processed are not both integer multiples of a preset number of segmentation pixels, the edge of the astronomical image to be processed is filled with pixels so that the width and height of the filled image are both integer multiples of a preset number of segmentation pixels. By filling in the size, it can be ensured that all the image blocks obtained by segmentation can meet the image size requirements of the trajectory extraction model, thus avoiding the problem of poor processing effect of incomplete blocks.

[0027] In an optional implementation of the second aspect, generating a stitched image corresponding to each image block based on the target image corresponding to each image block includes: determining a target cropping width based on the number of pixels of the overlap width of adjacent image blocks, wherein the target cropping width is less than the overlap width; determining the edge region of the target cropping width around the target image corresponding to each image block as the target cropping region; and performing edge cropping on the target cropping region corresponding to each target image to obtain a stitched image corresponding to each image block.

[0028] In the above implementation method, the cropping width is smaller than the overlap width. On the one hand, most of the areas with severe artifacts at the model edges can be cropped out. On the other hand, it can ensure that the adjacent stitched images after cropping still have overlapping areas, so that there will be no image loss or stitching gaps.

[0029] In an optional implementation of the second aspect, the stitched images corresponding to all image blocks are stitched together according to their corresponding spatial positions to generate a target astronomical image corresponding to the astronomical image to be processed. This includes: obtaining a fusion weight corresponding to each stitched image for the overlapping region between adjacent stitched images, wherein the fusion weight corresponding to each stitched image decreases linearly from the image center to the edge; obtaining a calculated weight for each pixel in the overlapping region corresponding to each stitched image based on the fusion weight corresponding to each stitched image, wherein the calculated weight for each pixel in the overlapping region corresponding to each stitched image is obtained by weight normalization of the fusion weight corresponding to each stitched image; and calculating the product of the pixel value in each stitched image and the corresponding calculated weight for each pixel in the overlapping region to obtain the target pixel value of each pixel in the overlapping region, thereby generating the target astronomical image corresponding to the astronomical image to be processed.

[0030] In the above implementation method, this solution sets a fusion weight that decreases linearly from the center of the image to the edge in the overlapping area of ​​the stitched image, and normalizes the fusion weight of each pixel in each stitched image to avoid the pixel value being too high due to the superposition of weights in multiple overlapping areas, thus ensuring uniform brightness of the whole image and eliminating bright spots and light-dark boundaries in the stitching.

[0031] In an optional embodiment of the second aspect, the method further includes: obtaining an edge protection region of the astronomical image to be processed, wherein the edge protection region is a preset width region of the edge of the astronomical image to be processed; and when generating the target astronomical image, the pixel values ​​of the edge protection region adopt the original pixel values ​​of the astronomical image to be processed.

[0032] In the above implementation, the edge protection area directly retains the original image pixels, avoiding edge repair distortion and reducing the computational overhead of edge segmentation.

[0033] Thirdly, the present invention provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the methods described in the first aspect, any optional embodiment of the first aspect, the second aspect, and any optional embodiment of the second aspect.

[0034] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, performs the methods described in the first aspect, any optional embodiment of the first aspect, the second aspect, and any optional embodiment of the second aspect.

[0035] Fifthly, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, performs the methods described in the first aspect, any optional implementation of the first aspect, the second aspect, and any optional implementation of the second aspect.

[0036] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0037] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1This is a first flowchart illustrating the aircraft trajectory processing method provided in an embodiment of this application. Figure 2 This is a second flowchart illustrating the aircraft trajectory processing method provided in the embodiments of this application; Figure 3 A schematic diagram of the third process of the aircraft trajectory processing method provided in the embodiments of this application; Figure 4 A schematic diagram of the fourth process of the aircraft trajectory processing method provided in the embodiments of this application; Figure 5 A schematic diagram of the fifth process of the aircraft trajectory processing method provided in the embodiments of this application; Figure 6 A schematic diagram of the sixth process of the aircraft trajectory processing method provided in the embodiments of this application; Figure 7 A schematic diagram of the seventh process of the aircraft trajectory processing method provided in the embodiments of this application; Figure 8 The eighth flowchart of the aircraft trajectory processing method provided in the embodiments of this application is shown. Figure 9 This is a first flowchart illustrating the astronomical image restoration method provided in the embodiments of this application; Figure 10 This is a schematic diagram of the second process of the astronomical image restoration method provided in the embodiments of this application; Figure 11 A schematic diagram of the third process of the astronomical image restoration method provided in the embodiments of this application; Figure 12 This is a schematic diagram of the structure of the aircraft trajectory processing device provided in the embodiments of this application; Figure 13 This is a schematic diagram of the astronomical image restoration apparatus provided in the embodiments of this application; Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0039] Icons: 1200 - First acquisition module; 1210 - Trajectory detection module; 1220 - Trajectory extraction module; 1230 - First generation module; 1300 - Second acquisition module; 1310 - Image segmentation module; 1320 - Aircraft trajectory processing module; 1330 - Second generation module; 14 - Electronic device; 1401 - Processor; 1402 - Memory; 1403 - Communication bus. Detailed Implementation

[0040] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0042] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0043] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0044] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0045] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0046] In the description of the embodiments of this application, the technical terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0047] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0048] Spacecraft tracks in astronomical images (including bright lines and condensation trails of the spacecraft itself) are one of the most common transient noises in ground-based astronomical observations, which can seriously affect the accuracy of astrophotometry, astrometry, and scientific analysis.

[0049] Currently, traditional methods for removing spacecraft tracks from astronomical images generally involve manually identifying the track location and then manually using professional image processing software to fill or replace the area covered by the track with surrounding uncontaminated pixel values. However, this manual track removal method suffers from problems such as long processing time and low repair quality.

[0050] To address the aforementioned issues, this application provides a method for processing spacecraft trajectories, an astronomical image restoration method, an electronic device, and a computer program product. First, a multi-level cascaded trajectory detection model filters out images without trajectories. Then, a pre-trained trajectory extraction model extracts the trajectory features originally mixed with the starry background, achieving separation of the spacecraft trajectory from the starry background. Finally, the original image and an image containing only the spacecraft trajectory are used to generate a target image with the spacecraft trajectory removed. This solution, through a multi-level cascaded trajectory detection mechanism, significantly reduces the false detection probability of spacecraft trajectories against complex astronomical backgrounds, avoiding misidentification of star trails, nebulae, and other celestial features as spacecraft trajectories, thus improving the accuracy of trajectory detection. Simultaneously, precise restoration is achieved through trajectory features extracted by the trajectory extraction model, rather than a complete reconstruction of the entire image. This ensures thorough trajectory removal while preserving the details of stars, nebulae, and other elements in the original astronomical image to the greatest extent possible, thereby improving the restoration effect of the astronomical image.

[0051] Based on the above ideas, this application first provides a method for processing aircraft trajectory, such as... Figure 1 As shown, this aircraft trajectory processing method can be applied to computing devices, including but not limited to computers, servers, control chips, host computers, and microcontrollers. This aircraft trajectory processing method can be implemented in the following ways: Step S100: Obtain the target image to be processed.

[0052] Step S110: Input the target image to be processed into the pre-trained multi-level cascaded trajectory detection model to obtain the aircraft trajectory detection result output by the multi-level cascaded trajectory detection model.

[0053] Step S120: If the aircraft trajectory detection result is that there is an aircraft trajectory, then input the target image to be processed into the pre-trained trajectory extraction model to obtain the trajectory extraction image output by the trajectory extraction model.

[0054] Step S130: Generate a target image with the aircraft trajectory removed based on the target image to be processed and the trajectory extraction image.

[0055] In the above implementation, the target image to be processed represents the astronomical image data that needs to be processed. As one possible implementation, when the computing device has strong processing capabilities, the target image to be processed in this solution can be a whole image taken by an astronomical camera. As another possible implementation, the target image to be processed in this solution can also be any one of the multiple image blocks obtained after dividing the whole image taken by an astronomical camera according to the preset image block side length size.

[0056] For step S100, the target image to be processed in this scheme can be obtained by receiving the acquired image transmitted by the astronomical camera and preprocessing the acquired image. The preprocessing can be one or more of the following: image stretching, normalization, brightness enhancement, and image segmentation.

[0057] A multi-level cascaded trajectory detection model represents a detection system composed of two or more independent trajectory detection models arranged in a sequential hierarchy. The output of the previous level model serves as the trigger condition for the next level model. The number of levels in the multi-level cascaded trajectory detection model can be flexibly configured according to the computing power of the deployed equipment, with a minimum of two levels and the possibility of three or more levels when computing power is sufficient. As a specific example, assuming a two-level cascaded trajectory detection model, this solution can use a main model MobileNetv2 and a secondary model ResNet18 to jointly construct the two-level cascaded trajectory detection model.

[0058] The trajectory extraction model represents a neural network model trained based on a deep learning feature extraction network. It can be used to separate the pixel information corresponding to the trajectory of an aircraft from an input image and output an image containing only trajectory features.

[0059] The proposed aircraft trajectory processing method involves the computing device acquiring the target image and first inputting it into a trained multi-level cascaded trajectory detection model. This model detects the presence of an aircraft trajectory in the target image. Only when the model detects the presence of an aircraft trajectory is a pre-trained trajectory extraction model used to extract the trajectory features from the starry background, creating a trajectory extraction image containing only the aircraft trajectory features. This extracted image is then combined with the original image to generate the target image with the aircraft trajectory removed. The trajectory extraction image can be presented in various forms, such as a binary mask, a trajectory probability map, or a trajectory residual map.

[0060] The trajectory extraction model can be configured with different training objectives, label formats, and loss functions depending on the target output format. The corresponding training configurations for the three output formats are as follows: 1. Binary Mask Output: The model is trained as a pixel-level binary classification task, and the training labels are binary labeled images (trajectory regions are labeled as 1 and background regions are labeled as 0). The loss function is a weighted combination of cross-entropy loss and Dice loss, which is used to optimize the segmentation accuracy of the trajectory region. During the inference stage, the probability map output by the model is binarized with a preset threshold (such as 0.5), and the final output is a binary trajectory mask with pixel values ​​of only 0 or 1.

[0061] 2. Trajectory Probability Map Output: The model training objective, label format, loss function and binary trajectory mask map are completely consistent, both being pixel-level binary classification tasks; the difference is that thresholding is not performed during the inference stage, and the pixel probability value after sigmoid activation is directly output, with a value range of [0,1]. Each pixel value directly represents the confidence that the position belongs to the aircraft trajectory.

[0062] 3. Trajectory Residual Image Output: The model is trained as a pixel-level regression task. The training label is the trajectory brightness residual image (i.e., the pixel-wise brightness difference between the original image containing the trajectory and the corresponding clean background image). The loss function adopts mean squared error (MSE) loss to fit the true brightness increment of the trajectory. The model directly learns the brightness superposition of the trajectory on the background. The inference output is the trajectory residual image, in which the pixel value of the background area approaches 0, and the pixel value of the trajectory area corresponds to the brightness component of the trajectory at that position.

[0063] The proposed method for processing spacecraft trajectories first filters out trajectory-less images using a multi-level cascaded trajectory detection model. Then, a pre-trained trajectory extraction model extracts the trajectory features that were previously mixed with the starry background, separating the spacecraft trajectory from the background. Finally, the original image and an image containing only the spacecraft trajectory are compared to generate a target image with the trajectory removed. This multi-level cascaded trajectory detection mechanism significantly reduces the false detection probability of spacecraft trajectories against complex astronomical backgrounds, avoiding misidentification of star trails, nebulae, and other celestial features as spacecraft trajectories, thus improving the accuracy of trajectory detection. Furthermore, the precise restoration using trajectory features extracted by the trajectory extraction model, rather than a complete reconstruction of the entire image, ensures thorough trajectory removal while preserving the details of stars and nebulae in the original astronomical image to the greatest extent possible, thereby improving the restoration effect of the astronomical image.

[0064] In an optional implementation of this embodiment, the determination step S110 of the multi-level cascaded trajectory detection model is as follows: Figure 2 As shown, this solution can be implemented in the following ways, including: Step S200: Input the target image to be processed sequentially into each level of the pre-trained multi-level cascaded trajectory detection model to obtain the trajectory detection result output by each level of the trajectory detection model.

[0065] Step S210: If all trajectory detection models output trajectory detection results that indicate the presence of an aircraft trajectory, then determine that the aircraft trajectory detection result indicates the presence of an aircraft trajectory.

[0066] In the above implementation, this scheme inputs the target image to be processed into each level of the trajectory detection model in the hierarchical order of the multi-level cascaded trajectory model. Each level of the model independently outputs the trajectory detection result of that level. Only when all levels of the trajectory detection model output the determination that there is an aircraft trajectory is the final detection result determined to be that there is an aircraft trajectory. If any level determines that there is no trajectory, the final result is that there is no aircraft trajectory. In this way, the probability of aircraft trajectory detection is reduced by multiple independent verifications, avoiding the incorrect repair of the trajectory-free area due to false detection, which would damage the original astronomical data.

[0067] Furthermore, the multi-level cascaded trajectory model designed in this scheme can employ a lightweight model as the pre-stage detection model and a high-precision model as the post-stage detection model. By sequentially inputting each stage of the trajectory detection model, if the output of the previous stage is no trajectory, the detection can be terminated directly without proceeding to the next stage. This allows the lightweight pre-stage model to quickly filter out trajectory-less images, resulting in fast processing speed, while the high-precision post-stage model only processes a small number of suspected samples, ensuring final detection accuracy. For example, astronomical images have a low probability of trajectory occurrence; most trajectory-less images can be excluded in the first-stage trajectory detection model, eliminating the need to proceed to the subsequent high-precision model, significantly improving the overall average inference speed.

[0068] The above-described implementation scheme determines that an aircraft trajectory is present only when all trajectory detection models in the multi-level cascade output trajectory detection results. This scheme employs a method of independent verification of multiple trajectories, thereby avoiding misjudgments caused by noise, artifacts, and star trails and nebulae in astronomical images due to single models, and improving the accuracy of trajectory detection.

[0069] As a specific implementation method, in the scenario of a two-level cascaded trajectory detection model, the main model designed in this scheme can adopt a lightweight classification network such as MobileNetv2, which has a small number of parameters and fast inference speed, and is used to quickly exclude images without trajectory. The secondary auxiliary model can adopt a more accurate classification network such as ResNet18 to perform secondary confirmation on suspected samples.

[0070] Specifically, the target image to be processed can first be input into the MobileNetv2 trajectory detection model. If the MobileNetv2 trajectory detection model determines that there is no aircraft trajectory, then there is no need to input into the ResNet18 trajectory detection model, and the image without aircraft trajectory can be directly output. In this way, the target image without aircraft trajectory can be quickly excluded by the lightweight MobileNetv2 trajectory detection model.

[0071] If the MobileNetv2 trajectory detection model determines that the target image contains an aircraft trajectory, the target image is then input into the ResNet18 trajectory detection model for further detection. If the ResNet18 trajectory detection model determines that the target image contains an aircraft trajectory, then the image is finally confirmed to contain an aircraft trajectory. If the ResNet18 trajectory detection model determines that the target image does not contain an aircraft trajectory, then the image is finally confirmed to not contain an aircraft trajectory.

[0072] In an optional embodiment of this example, the step S120 for acquiring the trajectory extraction image of the trajectory extraction model is as follows: Figure 3As shown, this solution can be implemented in the following ways, including: Step S300: The encoder of the trajectory extraction model performs layer-by-layer feature extraction and spatial resolution compression on the target image to be processed, and captures the global context information of the target image to be processed through dilated convolution to obtain the deep coding feature map output by the encoder.

[0073] Step S310: The decoder of the trajectory extraction model performs upsampling reconstruction layer by layer in the order from deep to shallow, and fuses it with the shallow detail feature map of the encoder of the corresponding layer to obtain the fused feature map corresponding to each layer.

[0074] Step S320: Output the trajectory residual prediction map of the corresponding level based on the fusion feature map of each level through the residual module of each level, until the output trajectory residual prediction map has the same resolution as the target image to be processed.

[0075] Step S330: Perform noise removal on the trajectory residual prediction map with the same resolution as the image to be processed to obtain the trajectory extraction image output by the trajectory extraction model.

[0076] In the above implementation, for step S300, this scheme performs layer-by-layer convolution processing on the input image through the encoder of the trajectory extraction model, and simultaneously performs downsampling through stride convolution or pooling to gradually compress the spatial resolution and improve the feature semantic depth; a dilated convolution layer is embedded deep in the encoder to capture the global context information of the target image to be processed, and the receptive field is expanded without increasing the number of parameters, and finally the deep encoded feature map containing global semantic information is output.

[0077] The receptive field represents the range of the input image region that the neural network relies on when processing a certain pixel. In other words, it is the extent to which the judgment result of that pixel is affected by the range of image information. It can be understood as the "range of vision" of the neural network over the image.

[0078] The core function of the encoder's layer-by-layer downsampling in step S300 above is to gradually abstract semantic features from pixel-level details, enabling the trained neural network model to understand what an aircraft trajectory is rather than just seeing lines and pixels.

[0079] Furthermore, aircraft trajectories are typically long linear features spanning the entire image. Ordinary convolutional algorithms have a small receptive field, only able to perceive a small segment of the trajectory. This makes aircraft trajectories recognized by ordinary convolutional algorithms easily confused with star-shaped or short trailing lines. Therefore, this solution designs dilated convolutions to significantly expand the receptive field without increasing computational cost, allowing the neural network model to perceive the global shape of the entire trajectory and accurately distinguish between long trajectories and short trailing lines. Dilated convolution is a special convolutional operation that introduces interval sampling between convolutional kernel elements, expanding the receptive field without increasing the number of parameters or computational cost, enabling the network to capture a wider range of contextual information.

[0080] For step S310, the decoder performs upsampling operations on the feature map layer by layer from deep to shallow, gradually restoring the spatial resolution. After each layer of upsampling, the shallow detail feature map of the corresponding layer of the encoder is concatenated and fused with the feature map of the current layer of the decoder in the channel dimension through skip connections, forming a fused feature map of that layer. The role of skip connections is to directly transmit the detail information lost during the encoder downsampling process to the decoder, so that when the decoder reconstructs the image, it can use the deep semantic information to determine which region is the spacecraft trajectory, and use the shallow detail information to preserve the star edges and nebula textures, thereby ensuring that the details of the original image are not lost while extracting the spacecraft trajectory.

[0081] For step S320, a residual module is set up for each decoding layer to predict the trajectory residual map at the corresponding resolution based on the fused feature map of that layer. The residual module learns the brightness difference between the input image and the target image, rather than the complete image. Therefore, the residual in the background region is close to zero, and only the trajectory region has significant residuals. After upsampling prediction at each level, the final output is a trajectory residual prediction map with the same resolution as the input image. The trajectory residual prediction map is obtained by adding the residual brightness features of the trajectory pixels in the fused feature map and the original brightness features of each pixel.

[0082] In step S320 above, since most areas in the astronomical image, except for the spacecraft trajectory, remain consistent before and after repair, the residual signals corresponding to these areas are close to zero, while the residual signals of the spacecraft trajectory area are significant. Therefore, this scheme uses residual learning to allow the neural network model to focus on learning the difference information of the spacecraft trajectory area, so that it can automatically focus on the spacecraft trajectory area that needs to be repaired, only adjusting the spacecraft trajectory area, while keeping the original star points, nebulae and other detailed background areas unchanged, thereby effectively avoiding the blurring or structural damage to the overall image caused by traditional methods.

[0083] For step S330, this scheme performs noise removal processing on the trajectory residual prediction map of the final resolution. For example, it removes isolated small pixels and scattered noise, retains continuous trajectory lines, and thus obtains a clean trajectory extraction image, that is, the trajectory extraction image output by the trajectory extraction model.

[0084] In step S330 above, the output of the deep learning model inevitably contains scattered background noise points. This solution is designed with a noise elimination step to filter out such scattered noise, avoid the generation of dark spots and artifacts on the background, and ensure the purity of trajectory extraction.

[0085] The implementation scheme described above utilizes dilated convolutions to expand the receptive field of the network, enabling it to perceive the global linear features of the entire aircraft trajectory. This avoids misjudgments caused by insufficient receptive field, which could lead to confusion between local trajectory features and stars or nebulae, thus improving the accuracy of the trajectory extraction model. Simultaneously, the scheme incorporates skip connections during the downsampling stage to supplement the shallow detail features obtained by the encoder, preventing detail loss during downsampling and ensuring that the original image details are not lost while achieving aircraft trajectory extraction. Furthermore, the scheme employs a residual learning mechanism that allows the neural network model to focus on learning the difference information within the aircraft trajectory region. This enables it to automatically focus on the areas of the aircraft trajectory that require repair, adjusting only the aircraft trajectory region while leaving the original star points, nebulae, and other detailed background areas unchanged. This effectively avoids the blurring or structural damage to the overall image caused by traditional methods, resulting in a trajectory extraction model that can accurately and completely extract the aircraft trajectory.

[0086] In an optional implementation of this embodiment, such as Figure 4 As shown, the training process of the trajectory extraction model designed above in this scheme can be implemented in the following ways, including: Step S400: Obtain the target training image set.

[0087] Step S410: Train the deep learning model to be trained based on the target training image set until a trained trajectory extraction model is obtained.

[0088] In the above implementation, this scheme can collect a large number of original astronomical image samples that do not contain spacecraft trajectories. Then, each original astronomical image sample is manually annotated at the pixel level, i.e., a spacecraft trajectory is manually added to each original astronomical image sample, generating a corresponding binary label image. The binary label image represents all pixels covered by the spacecraft trajectory in the image, labeled with a first preset value, and all pixels in the remaining part of the image, labeled with a second preset value, forming a trajectory image sample. Then, training image sample pairs are created, with each astronomical image sample corresponding to its corresponding trajectory image sample. All sample pairs together form the target training image set. The multiple original astronomical images in the target training image set can include astronomical images from different sky regions, with different exposure times, and under different weather conditions. The trajectory forms in the trajectory image samples in the target training image set can adopt various different shapes, such as spacecraft trajectories of different thicknesses, brightnesses, and angles, thereby comprehensively improving the model's generalization ability.

[0089] In addition to manual annotation, this solution can also use simulation generation to overlay simulated spacecraft trajectories onto clean astronomical images without trajectories, automatically generating paired labeled images, which can quickly expand the size of the training dataset.

[0090] Based on the target training image set obtained above, this scheme takes the original image sample in each training image sample pair as input and the corresponding binary label image as the training ground value to supervise the training of the deep learning model to be trained, continuously optimize the model parameters until the model output accuracy meets the requirements, and obtain the trained trajectory extraction model.

[0091] The above implementation method of this scheme uses pixel-level binary labels for supervised training. The model can learn the precise boundaries and shape of the trajectory, and the output trajectory extraction image has accurate pixel-level positioning with small error.

[0092] In an optional implementation of this embodiment, for step S410, based on the obtained target training sample set, the following specific training methods can be used to train the model: Figure 5 As shown, it includes: Step S500: Extract a batch of training image sample pairs from the target training sample set and input the training image sample pairs into the deep learning model to be trained.

[0093] Step S510: Perform layer-by-layer feature extraction and spatial resolution compression on the input original image sample through the encoder of the deep learning model, and capture the global context information of the input original image sample through dilated convolution to obtain the deep encoded feature map corresponding to the original image sample.

[0094] Step S520: The decoder of the deep learning model performs upsampling reconstruction layer by layer in order from deep to shallow, and fuses it with the shallow detail feature map of the encoder at the corresponding level to obtain the fused feature image sample.

[0095] Step S530: Output the trajectory residual prediction sample map of the corresponding level based on the fused feature image sample of each level through the residual module of each level, until the resolution of the output trajectory residual prediction sample map is consistent with that of the original image sample.

[0096] Step S540: Perform noise removal on the trajectory residual prediction sample map with the same resolution as the original image sample to obtain the prediction segmentation map corresponding to the original image sample.

[0097] Step S550: Input the predicted segmentation map and the trajectory image samples corresponding to the original image samples into the preset loss function, and calculate the loss value between the prediction result and the true label.

[0098] Step S560: Determine whether the loss value has reached the target loss value or whether the number of model iterations has reached the target number. If yes, proceed to step S570; otherwise, proceed to step S580.

[0099] Step S570: Obtain the trained trajectory extraction model.

[0100] Step S580: Based on the loss value between the predicted result and the true label, perform backpropagation and update the model parameters, then proceed to step S500.

[0101] In the above implementation, for step S500, this scheme randomly selects a batch of training image sample pairs from the target training sample set, inputs the original image samples into the deep learning model to be trained, and uses the corresponding binary label images as ground values ​​for later use.

[0102] After inputting the original image samples into the deep learning model to be trained, step S510 is executed. Specifically, this scheme performs layer-by-layer feature extraction and spatial resolution compression on the input original image samples through the encoder of the deep learning model, captures global context information through dilated convolution, and outputs the deep encoded feature map corresponding to the original image samples. The execution method of the encoder is completely consistent with the execution method of the inference stage (step S300), and will not be described again here.

[0103] After obtaining the deep encoded feature map corresponding to the original image sample, this scheme executes steps S520 and S530. The decoder of the deep learning model performs upsampling reconstruction layer by layer from deep to shallow, fusing it with the shallow detail feature map of the encoder at the corresponding layer to obtain fused feature image samples. The residual module at each layer outputs the trajectory residual prediction sample map based on the fused feature image sample at each layer, until the output trajectory residual prediction sample map has the same resolution as the original image sample. The execution process of the decoder is completely consistent with the execution process of the inference stage (steps S310 and S320), and will not be described again here.

[0104] Given the obtained trajectory residual prediction sample map, this scheme performs noise removal on the trajectory residual prediction sample map at the final resolution and outputs a prediction segmentation map, where the value of each pixel in the prediction segmentation map represents the probability that the pixel belongs to the aircraft trajectory.

[0105] Given the predicted segmentation map obtained above, this scheme inputs the predicted segmentation map and the corresponding ground truth binary label image into a preset loss function to calculate the loss value between the prediction result and the ground truth label; it then determines whether the loss value reaches the target loss value or whether the number of iterations reaches the target number. Here, the loss value represents a quantitative indicator of the model's prediction error; the smaller the loss value, the more accurate the model's prediction.

[0106] If the loss value reaches the target loss value or the number of iterations reaches the target number, it means that the model training has reached the termination condition, and the model training is stopped, thus obtaining the trained trajectory extraction model.

[0107] If the loss value does not reach the target loss value and the number of iterations does not reach the target number, it means that the model has not yet reached the termination condition. Then, based on the loss value between the predicted result and the true label, backpropagation is performed and the model parameters are updated. The process then returns to the batch extraction step and enters the next round of training iterations until the loss value reaches the target loss value or the number of iterations reaches the target number.

[0108] The loss function designed in this scheme can be a weighted combination of cross-entropy loss and Dice loss, which improves the model's recognition accuracy for small targets and faint trajectories by addressing the sample imbalance problem where the trajectory region accounts for a small proportion. The optimizer can be the AdamW optimizer, which accelerates the model convergence speed and improves the final accuracy.

[0109] The above-described implementation scheme employs dilated convolutions to expand the receptive field of the encoder, enabling the trained model to perceive the global linear features of the entire aircraft trajectory. This avoids misjudgments caused by insufficient receptive field, which could lead to confusion between local trajectory features and stars or nebulae, thus improving the accuracy of trajectory extraction. Simultaneously, skip connections are incorporated into the downsampling phase to supplement the shallow detail features obtained by the encoder, allowing the model to learn how to accurately identify trajectory regions while preserving background details. Furthermore, a residual learning mechanism is designed to allow the neural network model to focus on learning the differential information of the aircraft trajectory regions. This enables it to automatically focus on the areas of the aircraft trajectory that need repair, adjusting only the aircraft trajectory regions while leaving the original star points, nebulae, and other detailed background areas unchanged. This effectively avoids the blurring or structural damage to the overall image caused by traditional methods, resulting in a trajectory extraction model that can accurately and completely extract the aircraft trajectory. Moreover, the scheme uses dual termination conditions—loss value and iteration count—to precisely control the training accuracy of the model, avoiding underfitting and overfitting.

[0110] In an optional implementation of this embodiment, the trajectory extraction image output by the trajectory extraction model described above can be any one of a binary trajectory mask, a trajectory probability map, or a trajectory residual map. Therefore, when the trajectory extraction image adopts different image output formats, different processing methods can be used to generate the target image with the aircraft trajectory removed.

[0111] As one possible implementation, when the trajectory extraction image is a binary trajectory mask, such as Figure 6 As shown, step S130 can be specifically implemented in the following ways, including: Step S600: Determine the trajectory region and effective background region of the target image to be processed based on the mask pixel values ​​of the trajectory extracted image.

[0112] Step S610: Using the binary trajectory mask as a guide mask, the background of the trajectory region is reconstructed to generate the repaired background region corresponding to the trajectory region.

[0113] Step S620: The repaired background area is stitched together with the effective background area to generate a target image with the aircraft trajectory removed.

[0114] In the above implementation, the binary trajectory mask is a single-channel binary image with pixel values ​​of only 0 or 1. The area with a value of 1 represents the area covered by the aircraft trajectory, and the area with a value of 0 represents the effective background area.

[0115] Therefore, this scheme can extract the mask pixel values ​​of the image based on the trajectory of the binary trajectory mask map, and divide the target image to be processed into the trajectory region (the region with a mask value of 1) and the effective background region (the region with a mask value of 0).

[0116] This scheme then uses a binary trajectory mask as a guiding mask to perform a background reconstruction algorithm only on the marked trajectory regions, generating a repaired background region with the exact same size as the trajectory region. The guiding mask limits the scope of the repair algorithm, preventing it from spreading into the background region and ensuring a precise correspondence between the repair boundary and the trajectory boundary.

[0117] The background reconstruction in this scheme can employ any existing background reconstruction algorithm. As a specific example, this scheme can be implemented in ways not limited to the following two: Method 1: This solution can use a deep learning inpainting network. Specifically, the target image to be processed and the binary trajectory mask are input into a pre-trained image inpainting network. The inpainting network takes the mask area as the inpainting range, extracts background features such as star distribution and nebula texture around the mask, and generates continuous and natural background content through an encoder and decoder structure. The output is an inpainted background area with the same size as the trajectory area.

[0118] Method 2: This solution can use traditional texture synthesis and interpolation methods. Specifically, the boundary of the binary trajectory mask can be used as the starting point, and effective background pixels within a preset width on both sides of the trajectory can be collected as samples along the trajectory normal direction. Bilinear interpolation or neighborhood texture synthesis algorithms are used to gradually fill the brightness and texture features of the sample pixels into the trajectory area to generate the repaired background area.

[0119] Given the restored background area, this solution can embed the reconstructed restored background area into the trajectory position of the original image and stitch it together with the original effective background area to obtain a complete target image with the aircraft trajectory removed.

[0120] It's important to note that the background reconstruction algorithm in this solution generates fill content based on the real background texture around the trajectory, allowing the repaired area to blend naturally with the surrounding starry sky. Specifically, background reconstruction can employ deep learning image inpainting algorithms or traditional texture synthesis and interpolation algorithms. For scenes with wide trajectories, a transition area of ​​1-2 pixels can be extended outward from the guiding mask for smoothing, avoiding hard edges at the repair boundary. After stitching, a slight feathering effect can be applied to the boundary area to further minimize stitching artifacts.

[0121] The above-described implementation scheme divides the target image to be processed into a trajectory region and an effective background region by using a mask value. Background reconstruction is performed only on the trajectory region, which has a small computational load and high processing efficiency. At the same time, the effective background region completely retains the pixel values ​​of the original image. In this way, the effective background completely retains the original data, and only the trajectory region is replaced with the repaired background region. This achieves both trajectory removal and maximum preservation of the original photometric and astronomical measurement information of the original astronomical image.

[0122] In an optional implementation of this embodiment, when the trajectory extraction image is a trajectory probability map, such as Figure 7 As shown, step S130 can be specifically implemented in the following ways, including: Step S700: Determine the trajectory weight and background weight corresponding to each pixel value based on the confidence level of each pixel value in the trajectory probability map.

[0123] Step S710: Generate a background reconstruction image with the same size as the target image based on the effective background pixels in the preset width on both sides of the trajectory in the target image to be processed.

[0124] Step S720: Calculate the product of the reconstructed background pixel value and the trajectory weight corresponding to its position to obtain the first weighted value.

[0125] Step S730: Calculate the product of the original pixel value corresponding to each background reconstructed pixel value and the background weight corresponding to its position to obtain the second weighted value.

[0126] Step S740: Calculate the sum of the first weighted value and the second weighted value of the pixel value corresponding to each position to obtain the fused pixel value of each pixel in the target image to be processed, and generate the target image with the aircraft trajectory removed.

[0127] In the above implementation, the value of each pixel in the trajectory probability map ranges from 0 to 1, representing the confidence level that the pixel belongs to the aircraft trajectory. The closer the value is to 1, the higher the probability that it belongs to the aircraft trajectory.

[0128] Based on the characteristics of the trajectory probability map, this scheme directly uses the confidence level of each pixel in the trajectory probability map as the trajectory weight of that pixel, and then calculates the corresponding background weight by subtracting the trajectory weight from 1.

[0129] Based on the obtained trajectory weights and background weights, this scheme generates a background reconstruction image with the same size as the target image by using the effective background pixels in the preset width on both sides of the trajectory in the target image to be processed. The effective background pixels represent pixels with a confidence level lower than a preset confidence threshold. The background reconstruction image contains the background reconstruction pixel values ​​corresponding to each pixel position.

[0130] Specifically, based on the trajectory area (or trajectory center line), a preset width is extended to both sides along the normal direction of the trajectory. For example, when the average width of the trajectory is 8 pixels, any pixel between 8 and 24 pixels is extended to both sides. Pixels that fall within this extension range and whose confidence level is lower than the preset confidence threshold are considered valid background pixels.

[0131] Then, this scheme creates a blank image with the same size as the target image to be processed as the background reconstruction base map; the original pixel values ​​of all effective background pixels are directly assigned to the corresponding positions in the base map, and the trajectory pollution areas with confidence scores higher than the threshold are reserved as unfilled states. For each trajectory pollution pixel to be filled, the two nearest effective background sample points are searched to both sides along the trajectory normal direction of its location; weighted interpolation calculation is performed with the reciprocal of the distance from the two points to the pixel as the weight to obtain the background reconstruction value of the pixel. Then, the boundary between the trajectory area and the original background area is feathered by 1~2 pixels to obtain the complete background reconstruction image.

[0132] Based on the above, this scheme calculates the product of each background reconstructed pixel value and the trajectory weight corresponding to its position to obtain a first weighted value; calculates the product of the original pixel value corresponding to each background reconstructed pixel value and the background weight corresponding to its position to obtain a second weighted value; and then calculates the sum of the first weighted value and the second weighted value of the pixel value corresponding to each position to obtain the fused pixel value of each pixel in the target image to be processed, and generates a target image with the aircraft trajectory removed.

[0133] In the above implementation method of this solution, the probability map can accurately represent the proportion of trajectory components of each pixel. By weighting, different degrees of trajectory pollution can be removed proportionally. Thus, for condensed trails with blurred edges and gradual brightness changes, the probability map weighting method fits the actual shape of the trajectory better than the binary mask, and removes the aircraft trajectory more thoroughly and without residue.

[0134] In an optional implementation of this embodiment, when the trajectory extraction image is a trajectory residual image, such as... Figure 8 As shown, step S130 can be specifically implemented in the following ways, including: Step S800: Calculate the difference between the target image to be processed and the trajectory residual image for each pixel value, and generate a target image with the aircraft trajectory removed.

[0135] In the above implementation, the value of each pixel in the trajectory residual map represents the brightness component of the aircraft trajectory for that pixel. The pixel values ​​in the background area are close to zero. Overall, the trajectory represents the amount of brightness superposition on the original image. Thus, the trajectory residual map represents the pixel brightness component of the aircraft trajectory, and the original image = the real background image + the trajectory residual map.

[0136] Based on the above principles, this solution only requires subtracting the target image to be processed from the trajectory residual image pixel by pixel. Since the residual in the background region is 0, the background pixels after the subtraction are completely consistent with the background pixels of the original target image to be processed, without any changes. However, the aircraft trajectory region has pixel brightness components. By subtracting the pixel brightness components of the trajectory residual image from the pixel values ​​of the aircraft trajectory region in the target image to be processed, the trajectory of the target image to be processed can be removed, thus obtaining the target image with the aircraft trajectory removed.

[0137] The above implementation method of this solution uses a residual learning mechanism to ensure that the model only outputs the brightness difference in the trajectory region, with no residual in the background region. After subtraction, the background pixels are completely consistent with the original image, without any blurring or loss of detail. At the same time, the model learns the true brightness difference between the trajectory and the background, and after subtraction, it can accurately restore the original brightness of the background. It can adaptively remove trajectories with different brightness, without over-repairing or under-repairing.

[0138] This application also provides a method for astronomical image restoration, which can also be applied to computing devices, including but not limited to computers, servers, host computers, and the built-in processor of astronomical cameras, etc. Figure 9 As shown, this astronomical image restoration method can be implemented in the following ways: Step S900: Acquire the astronomical image to be processed.

[0139] Step S910: Perform image segmentation on the astronomical image to be processed according to the preset segmentation size to obtain multiple image blocks.

[0140] Step S920: According to the target scanning order, select one target image block from multiple image blocks in sequence and execute the aircraft trajectory processing method until the target image corresponding to each image block is obtained.

[0141] Step S930: Generate a stitched image corresponding to each image block based on the target image corresponding to each image block.

[0142] Step S940: Stitch together the images corresponding to all image blocks according to their corresponding spatial positions to generate the target astronomical image corresponding to the astronomical image to be processed.

[0143] In the above implementation, for step S900, this scheme first acquires the original astronomical image captured by the astronomical camera, then performs stretching preprocessing on the original astronomical image to expand its dynamic range and enhance the contrast of faint trajectories, and then performs normalization processing to map the pixel values ​​to the range of 0~1 to obtain the astronomical image to be processed. The original astronomical image may have concentrated pixel values ​​in dark areas, making faint trajectories difficult to distinguish. Directly inputting the original astronomical image into a multi-level cascaded trajectory detection model in this way can easily lead to missed detections. Therefore, this scheme stretches the original astronomical image, which enhances the details in dark areas and improves the contrast between the trajectory and the background, significantly reducing the missed detection rate. Meanwhile, normalization is a standard input requirement for neural network models, mapping pixel values ​​to the numerical range used during model training, ensuring inference accuracy, and improving computational stability.

[0144] In step S910, given the astronomical image to be processed, this scheme divides the large image into multiple fixed-size small image blocks according to a preset segmentation size. Each image block records its spatial position in the original image. It should be noted that the memory usage of deep learning models is positively correlated with the input image size. Direct inference of high-resolution astronomical images requires a huge amount of memory, which ordinary edge devices cannot support. Therefore, this scheme breaks down the large image into small units that the model can handle through block processing, fundamentally solving the problem of edge deployment of high-resolution images and enabling deep learning algorithms to run on the built-in processor of the astronomical camera.

[0145] As a specific example, the preset segmentation size of this solution can be flexibly adjusted according to the video memory of the deployed device. The commonly used size is 256×256 blocks. This size balances the repair effect and the amount of computation. If the size is too large, the memory usage of a single block will be high, and if the size is too small, the number of blocks will be too large and the splicing overhead will be large.

[0146] When multiple image blocks are obtained, this scheme sequentially treats each image block as a target image block according to the target scanning order and executes the aircraft trajectory processing method described above. Specifically, if a trajectory is detected by the multi-level cascaded trajectory detection model, trajectory removal is achieved based on the trajectory extraction model, resulting in a target image with the trajectory removed. If no trajectory is detected by the multi-level cascaded trajectory detection model, the original image block is directly output; that is, the target image corresponding to the image block is the image block itself. By executing the aircraft trajectory processing method described above on all image blocks, the target image corresponding to each image block is obtained. When multiple image blocks are processed using the aircraft trajectory processing method described above, they must be processed block by block. That is, the next image block only enters the aircraft trajectory processing process after the previous image block has completed the aircraft trajectory processing method and output its corresponding target image.

[0147] Given the target image corresponding to each image patch, this scheme can perform adaptation processing on each target image to generate a stitched image corresponding to each image patch. Then, according to the spatial position of each stitched image, all the stitched images are combined and stitched together to form a complete target astronomical image. The core purpose of the stitching process is to eliminate block artifacts, ensure the visual consistency of the entire image, and avoid problems such as seams between blocks and uneven brightness.

[0148] This proposed astronomical image restoration method employs the previously described spacecraft trajectory processing approach for trajectory detection and removal in each image block. Therefore, through a multi-level cascaded trajectory detection mechanism, it significantly reduces the false detection probability of spacecraft trajectories in complex astronomical backgrounds, avoiding misidentification of star trails, nebulae, and other celestial features as spacecraft trajectories, thus improving trajectory detection accuracy. Furthermore, it uses trajectory features extracted by a trajectory extraction model for precise restoration, rather than a complete reconstruction of the entire image. This ensures thorough trajectory removal while preserving the details of stars and nebulae in the original astronomical image to the greatest extent possible, thereby improving the restoration effect. In addition, this scheme uses block processing to break down large astronomical images into smaller blocks supported by the model. This allows the proposed astronomical image restoration method to run on embedded devices with limited computing power (such as the built-in processor of an astronomical camera), eliminating the need for cloud-based processing and improving the real-time performance of the method.

[0149] In an optional implementation of this embodiment, for step S910 described above, as one possible implementation, such as... Figure 10 As shown, this scheme can achieve specific segmentation of the astronomical image to be processed in the following ways: Step S1000: With a preset sliding step size of a number of segmented pixels, slide sequentially along the horizontal and vertical directions on the astronomical image to be processed to generate multiple overlapping image blocks.

[0150] In the above implementation, this scheme uses a preset segmentation size as the window size and a fixed sliding step size to slide sequentially along the horizontal and vertical directions on the astronomical image to be processed. Each time it slides to a position, it captures an image block, and finally generates multiple overlapping image blocks.

[0151] In this scheme, the overlap width between adjacent image blocks is set to 1 / n of the sliding step, where n ≥ 2, ensuring that the overlap width is sufficient to support subsequent edge cropping and fusion. For example, as a concrete example, assuming the block size is 256×256 and the sliding step is set to 192 pixels, with the overlap width between adjacent image blocks being 1 / 3 of the block size (i.e., 64 pixels), in this case, the overlap width between adjacent image blocks in both the horizontal and vertical directions is 64 pixels during the image block segmentation process.

[0152] In the above implementation method of this solution, an overlapping area is set during the segmentation of image blocks. The setting of the overlapping area ensures that after the cropping edge, adjacent blocks still have overlapping parts, which can be fused to achieve seamless splicing and avoid splicing gaps.

[0153] In an optional implementation of this embodiment, the sliding window segmentation requires that the image size be divisible by the segmentation size; otherwise, the last row and last column of blocks will be insufficient in size, becoming incomplete image blocks. Incomplete image blocks cannot be directly input into a trajectory extraction model with a fixed input size, and forced scaling will lead to image distortion and decreased accuracy. Therefore, this solution can perform size adaptation in advance. Specifically, before generating multiple overlapping image blocks, this solution can also determine whether the width and height of the astronomical image to be processed are both integer multiples of a preset number of segmentation pixels. If it is determined that the width and height of the astronomical image to be processed are not both integer multiples of a preset number of segmentation pixels, then pixel filling is performed on the edges of the astronomical image to be processed so that the width and height of the filled image are both integer multiples of a preset number of segmentation pixels, thus obtaining a pixel-filled astronomical image to be processed. In this way, by filling to complete the size, it can be ensured that all segmented image blocks can meet the image size requirements of the trajectory extraction model, avoiding the problem of poor processing effect of incomplete blocks.

[0154] In addition, the filling method designed in this scheme can adopt mirror filling or edge copy filling, which can maintain the texture continuity of the image edge and reduce the impact of the filled area on model detection and repair.

[0155] In an optional implementation of this embodiment, for step S930 described above, as one possible implementation, such as... Figure 11 As shown, this solution can achieve the specific generation method of stitched images in the following ways: Step S1100: Determine the target cropping width based on the number of pixels of the overlap width of adjacent image blocks.

[0156] Step S1110: Determine the edge region of the target cropping width around the target image corresponding to each image block as the target cropping region.

[0157] Step S1120: Perform edge cropping on the target cropping region corresponding to each target image to obtain the stitched image corresponding to each image block.

[0158] In the above implementation, the number of pixels whose target cropping width is less than the overlap width is designed in this way. On the one hand, most of the areas with severe artifacts at the edge of the model can be cropped out, and on the other hand, it can ensure that the adjacent stitched images after cropping still have overlapping areas, so that there will be no image loss or stitching gaps.

[0159] Given a defined target cropping width, this scheme delineates the area within the target cropping width around each image block as the target cropping region, with the remaining area in the center being the valid region. Then, the target cropping regions around each target image are cropped, and the valid central region is retained to obtain the stitched image corresponding to the image block.

[0160] As a concrete example, assuming a block size of 256×256 and an overlap width of 64 pixels, the target crop width can be set to 16 pixels. This means cropping the edges of the image block by 16 pixels on each side, retaining the effective 224×224 area in the center. At this target crop width, most edge artifacts can be removed, while the remaining overlap area is sufficient for weighted fusion. The specific value of the target crop width designed in this scheme can be adjusted according to the actual performance of the model. If the model has a large range of edge artifacts, the crop width can be increased appropriately, and vice versa.

[0161] In the above implementation method of this solution, the cropping width is smaller than the overlap width. On the one hand, it can crop out most of the areas with severe artifacts at the edge of the model. On the other hand, it can ensure that the adjacent stitched images after cropping still have overlapping areas, so that there will be no image loss or stitching gaps.

[0162] In an optional embodiment of this example, the specific method for stitching the images in step S940 can be based on the original spatial position of each stitched image, placing it at the corresponding coordinates of the whole image to be stitched, and retaining overlapping areas between adjacent stitched images.

[0163] For the overlapping region between adjacent stitched images, this scheme first obtains the fusion weight corresponding to each stitched image, where the fusion weight of each stitched image decreases linearly from the image center to the edge; then, based on the fusion weight corresponding to each stitched image, the calculated weight of each pixel in the overlapping region corresponding to each stitched image is obtained, where the calculated weight of each pixel in each stitched image is obtained by weight normalization of the fusion weight corresponding to each stitched image; furthermore, for each pixel in the overlapping region, the product of its pixel value in each stitched image and its corresponding calculated weight is calculated to obtain the target pixel value of each pixel in the overlapping region, generating the target astronomical image corresponding to the astronomical image to be processed, thereby achieving a smooth transition between stitched images and ultimately achieving natural fusion of adjacent stitched images.

[0164] Additionally, it's important to note that for all overlapping regions, especially those where four blocks intersect, weight normalization is necessary. Without normalization, the pixel values ​​in the overlapping regions would be the sum of the pixel values ​​from the four stitched images. This would result in the overlapping regions being brighter than the non-overlapping regions, creating visible bright edges or spots. Therefore, weight normalization ensures that the total weight of each pixel in the overlapping regions is 1, preventing excessive brightness caused by multiple blocks overlapping and guaranteeing uniform brightness across the entire image. The specific weight normalization method can employ any known existing method to ensure that the total weight of each pixel in the overlapping regions is 1.

[0165] In the above implementation method of this scheme, a fusion weight is set in the overlapping area of ​​the stitched image, which decreases linearly from the image center to the edge. The fusion weight of each pixel in each stitched image is normalized to avoid the pixel value being too high due to the superposition of weights in multiple overlapping areas, so as to ensure that the brightness of the whole image is uniform and consistent, and to eliminate the bright spots and the boundary between light and dark areas in the stitching.

[0166] In an optional implementation of this embodiment, since the image edge region lacks sufficient neighborhood background information, edge artifacts are prone to occur during model restoration. Therefore, an edge protection mechanism can be set. Specifically, this solution can obtain the edge protection region of the astronomical image to be processed, wherein the edge protection region is a preset width region of the edge of the astronomical image to be processed; when generating the target astronomical image, the pixel values ​​of the edge protection region adopt the original pixel values ​​of the astronomical image to be processed. For example, a region with a preset width (such as 16 pixels) around the image is defined as the edge protection region. When generating the target astronomical image, this edge protection region directly retains the original image pixels, avoiding edge restoration distortion and reducing the computational overhead of edge segmentation.

[0167] Figure 12 A schematic structural block diagram of an aircraft trajectory processing device provided in this application is presented. It should be understood that this device is applied to the computing device described above, and this device is related to... Figures 1 to 8The method embodiments executed in this document correspond to the steps involved in the aforementioned method. The specific functions of the device can be found in the description above, and detailed descriptions are omitted here to avoid repetition. The device includes at least one software functional module that can be stored in memory or embedded in the device's operating system (OS) in the form of software or firmware. Specifically, the device includes: a first acquisition module 1200, a trajectory detection module 1210, a trajectory extraction module 1220, and a first generation module 1230. The first acquisition module 1200 is used to acquire a target image to be processed. The trajectory detection module 1210 is used to input the target image to be processed into a pre-trained multi-level cascaded trajectory detection model to obtain the aircraft trajectory detection result output by the multi-level cascaded trajectory detection model. The trajectory extraction module 1220 is used to input the target image to be processed into a pre-trained trajectory extraction model when the aircraft trajectory detection result of the trajectory detection module 1210 is that there is an aircraft trajectory, to obtain a trajectory extraction image output by the trajectory extraction model. The trajectory extraction image is an image that only contains the aircraft trajectory in the target image to be processed. The first generation module 1230 is used to generate a target image with the aircraft trajectory removed based on the target image to be processed and the trajectory extraction image.

[0168] The spacecraft trajectory processing device provided in this solution first filters out trajectory-free images through a pre-stage multi-level cascaded trajectory detection model. Then, it extracts the trajectory features that were originally mixed in with the starry background using a pre-trained trajectory extraction model, thus separating the spacecraft trajectory from the starry background. Finally, it generates a target image with the spacecraft trajectory removed by comparing the original image with an image containing only the spacecraft trajectory. In this way, this solution significantly reduces the false detection probability of spacecraft trajectories in complex astronomical backgrounds through a multi-level cascaded trajectory detection mechanism, avoiding misidentification of celestial features such as star trails and nebulae as spacecraft trajectories, and improving the accuracy of trajectory detection. At the same time, it performs precise restoration using trajectory features extracted by the trajectory extraction model, rather than reconstructing the entire image. This ensures the thoroughness of trajectory removal while preserving the details of stars, nebulae, etc., in the original astronomical image to the greatest extent, thereby improving the restoration effect of the astronomical image.

[0169] Figure 13 A schematic structural block diagram of an astronomical image restoration apparatus provided in this application is presented. It should be understood that the apparatus is applied to the computing device described above, and the apparatus is compatible with... Figures 9 to 11The method embodiments executed in this document correspond to the steps involved in the aforementioned method. The specific functions of the device can be found in the description above, and detailed descriptions are omitted here to avoid repetition. The device includes at least one software functional module that can be stored in memory or embedded in the device's operating system (OS) in the form of software or firmware. Specifically, the device includes: a second acquisition module 1300, an image segmentation module 1310, a spacecraft trajectory processing module 1320, and a second generation module 1330. The second acquisition module 1300 is used to acquire an astronomical image to be processed; wherein, the astronomical image to be processed is obtained by stretching and normalizing the original astronomical image as a preprocessing step; the image segmentation module 1310 is used to segment the astronomical image to be processed according to a preset segmentation size to obtain multiple image blocks; wherein, each image block corresponds one-to-one with its spatial position in the astronomical image to be processed; the spacecraft trajectory processing module 1320 is used to sequentially select a target image block from the multiple image blocks according to the target scanning order and execute the spacecraft trajectory processing method of any optional implementation until the target image corresponding to each image block is obtained; wherein, if the spacecraft trajectory detection result is no spacecraft trajectory, the obtained target image is the corresponding image block itself; the second generation module 1330 is used to generate a stitched image corresponding to each image block according to the target image corresponding to each image block; and to stitch together the stitched images corresponding to all image blocks according to their corresponding spatial positions to generate the target astronomical image corresponding to the astronomical image to be processed.

[0170] The astronomical image restoration device provided in this solution employs the spacecraft trajectory processing method described above to perform trajectory detection and removal on each image block. Therefore, the designed astronomical image restoration method can significantly reduce the false detection probability of spacecraft trajectories in complex astronomical backgrounds through a multi-level cascaded trajectory detection mechanism, avoiding misidentification of star trails, nebulae, and other celestial features as spacecraft trajectories, thus improving the accuracy of trajectory detection. Simultaneously, it uses trajectory features extracted by the trajectory extraction model for precise restoration, rather than reconstructing the entire image, ensuring thorough trajectory removal while preserving the details of stars, nebulae, etc., in the original astronomical image to the greatest extent, thereby improving the restoration effect. Furthermore, this solution uses block processing to break down large astronomical images into smaller blocks supported by the model, enabling the designed astronomical image restoration method to run on embedded devices with limited computing power (such as the built-in processor of an astronomical camera), without needing to upload to the cloud for processing, thus improving the real-time performance of the astronomical image restoration method.

[0171] According to some embodiments of this application, such as Figure 14As shown, this application provides an electronic device 14, including: a processor 1401 and a memory 1402. The processor 1401 and the memory 1402 are interconnected and communicate with each other through a communication bus 1403 and / or other forms of connection mechanism (not shown). The memory 1402 stores a computer program executable by the processor 1401. When the computing device is running, the processor 1401 executes the computer program to perform any optional implementation method, such as steps S100 to S130: acquiring a target image to be processed; inputting the target image to be processed into a pre-trained multi-level cascaded trajectory detection model to obtain the aircraft trajectory detection result output by the multi-level cascaded trajectory detection model; if the aircraft trajectory detection result is that there is an aircraft trajectory, then inputting the target image to be processed into a pre-trained trajectory extraction model to obtain the trajectory extraction image output by the trajectory extraction model; generating a target image with the aircraft trajectory removed based on the target image to be processed and the trajectory extraction image. Alternatively, steps S900 to S940 can be executed: acquiring the astronomical image to be processed; segmenting the astronomical image to be processed according to a preset segmentation size to obtain multiple image blocks; sequentially selecting a target image block from among the multiple image blocks according to the target scanning order and executing the spacecraft trajectory processing method until the target image corresponding to each image block is obtained; generating a stitched image corresponding to each image block based on the target image corresponding to each image block; stitching the stitched images corresponding to all image blocks according to their corresponding spatial positions to generate the target astronomical image corresponding to the astronomical image to be processed.

[0172] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the method in any of the aforementioned optional implementations.

[0173] The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0174] This application provides a computer program product that, when run on a computer, causes the computer to perform a method in any of the optional implementations.

[0175] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and they should all be covered within the scope of the claims and specification of this application. In particular, as long as there is no structural conflict, the various technical features mentioned in the embodiments can be combined in any way. This application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for processing aircraft trajectory, characterized in that, The method includes: Acquire the target image to be processed; The target image to be processed is input into a pre-trained multi-level cascaded trajectory detection model to obtain the aircraft trajectory detection result output by the multi-level cascaded trajectory detection model. If the aircraft trajectory detection result indicates that an aircraft trajectory exists, the target image to be processed is input into the pre-trained trajectory extraction model to obtain the trajectory extraction image output by the trajectory extraction model; wherein, the trajectory extraction image is an image containing only the aircraft trajectory in the target image to be processed; Based on the target image to be processed and the trajectory extraction image, a target image with the aircraft trajectory removed is generated.

2. The aircraft trajectory processing method according to claim 1, characterized in that, The step of inputting the target image to be processed into a pre-trained multi-level cascaded trajectory detection model to obtain the aircraft trajectory detection result output by the multi-level cascaded trajectory detection model includes: The target image to be processed is sequentially input into each level of the pre-trained multi-level cascaded trajectory detection model to obtain the trajectory detection result output by each level of the trajectory detection model. If all trajectory detection models output trajectory detection results indicating the presence of an aircraft trajectory, then the aircraft trajectory detection result is determined to be an aircraft trajectory.

3. The aircraft trajectory processing method according to claim 1, characterized in that, The step of inputting the target image to be processed into a pre-trained trajectory extraction model to obtain the trajectory extraction image output by the trajectory extraction model includes: The encoder of the trajectory extraction model performs layer-by-layer feature extraction and spatial resolution compression on the target image to be processed, and captures the global context information of the target image to be processed through dilated convolution to obtain the deep encoded feature map output by the encoder. The decoder of the trajectory extraction model performs upsampling reconstruction layer by layer from deep to shallow, and fuses it with the shallow detail feature map of the encoder at the corresponding level to obtain the fused feature map corresponding to each level. The residual module corresponding to each level outputs the trajectory residual prediction map of the corresponding level based on the fusion feature map of each level, until the output trajectory residual prediction map has the same resolution as the target image to be processed. The trajectory residual prediction map is obtained by adding the residual brightness features of the trajectory pixels in the fusion feature map and the original brightness features of each pixel. Noise is eliminated from the trajectory residual prediction map whose resolution is consistent with that of the image to be processed to obtain the trajectory extraction image output by the trajectory extraction model.

4. The aircraft trajectory processing method according to claim 1, characterized in that, The training process of the trajectory extraction model includes: Obtain a target training image set; wherein, the target training image set includes multiple training image sample pairs, each training image pair includes an original image sample and a trajectory image sample corresponding to the original sample, the trajectory image sample is a binary label image corresponding to the original image sample, the binary label image represents an image in which all pixels covered by the aircraft trajectory in the image are labeled with a first preset value, and all pixels in the remaining part of the image are labeled with a second preset value; The deep learning model to be trained is trained based on the target training image set until a trajectory extraction model is obtained after training.

5. The aircraft trajectory processing method according to claim 4, characterized in that, The step of training the deep learning model to be trained based on the target training image set until a trained trajectory extraction model is obtained includes: A batch of training image sample pairs is extracted from the target training sample set, and the training image sample pairs are input into the deep learning model to be trained. The encoder of the deep learning model performs layer-by-layer feature extraction and spatial resolution compression on the input original image sample, and captures the global context information of the input original image sample through dilated convolution to obtain the deep encoded feature map corresponding to the original image sample. The decoder of the deep learning model performs upsampling reconstruction layer by layer from deep to shallow, and fuses it with the shallow detail feature map of the encoder at the corresponding level to obtain fused feature image samples. The residual module corresponding to each level outputs the trajectory residual prediction sample map of the corresponding level based on the fusion feature image sample of each level, until the resolution of the output trajectory residual prediction sample map is consistent with that of the original image sample. The trajectory residual prediction sample map is obtained by adding the residual brightness features of the trajectory pixels in the fusion feature image sample and the original brightness features of each pixel. Noise removal is performed on the trajectory residual prediction sample map with the same resolution as the original image sample to obtain the prediction segmentation map corresponding to the original image sample; wherein, the value of each pixel in the prediction segmentation map represents the probability that the pixel belongs to the aircraft trajectory; The predicted segmentation map and the trajectory image samples corresponding to the original image samples are input into a preset loss function to calculate the loss value between the prediction result and the true label; Determine whether the loss value reaches the target loss value or whether the number of model iterations reaches the target number; If it is determined that the loss value has not reached the target loss value or the number of model iterations has not reached the target number of iterations, then backpropagation is performed and the model parameters are updated based on the loss value between the prediction result and the true label. Then, the process returns to the step of extracting a batch of training image sample pairs from the target training sample set and inputting the training image sample pairs into the deep learning model. If it is determined that the loss value reaches the target loss value or the number of model iterations reaches the target number, then the trained trajectory extraction model is obtained.

6. The aircraft trajectory processing method according to claim 1, characterized in that, The step of generating a target image with the aircraft trajectory removed based on the target image to be processed and the trajectory extraction image includes: If the trajectory extraction image is a binary trajectory mask image, then the trajectory region and effective background region of the target image to be processed are determined based on the mask pixel values ​​of the trajectory extraction image; Using the binary trajectory mask as a guide mask, the trajectory region is reconstructed to generate a repaired background region corresponding to the trajectory region, wherein the repaired background region has the same size as the trajectory region; The repaired background area is stitched together with the effective background area to generate the target image with the aircraft trajectory removed.

7. The aircraft trajectory processing method according to claim 1, characterized in that, The step of generating a target image with the aircraft trajectory removed based on the target image to be processed and the trajectory extraction image includes: If the trajectory extraction image is a trajectory probability map, the value of each pixel in the trajectory probability map represents the confidence that the corresponding position belongs to the trajectory of the aircraft. Based on the confidence level of each pixel value in the trajectory probability map, the trajectory weight and background weight corresponding to each pixel value are determined. The trajectory weight is the confidence level of the corresponding pixel value belonging to the aircraft trajectory, and the background weight is 1 minus the calculated value of the trajectory weight. Based on the effective background pixels in the preset width on both sides of the trajectory in the target image to be processed, a background reconstruction image with the same size as the target image to be processed is generated. The effective background pixels represent pixels with a confidence level lower than a preset confidence threshold. The background reconstruction image contains the background reconstruction pixel value corresponding to each pixel position. Calculate the product of each background reconstructed pixel value and the trajectory weight corresponding to its position to obtain the first weighting value; The second weighted value is obtained by multiplying the original pixel value corresponding to each reconstructed background pixel value with the background weight corresponding to its position. The sum of the first weighted value and the second weighted value of the pixel value corresponding to each position is calculated to obtain the fused pixel value of each pixel in the target image to be processed, and the target image with the aircraft trajectory removed is generated.

8. The aircraft trajectory processing method according to claim 1, characterized in that, The step of generating a target image with the aircraft trajectory removed based on the target image to be processed and the trajectory extraction image includes: If the trajectory extraction image is a trajectory residual map, the trajectory residual map represents the pixel brightness component of the aircraft trajectory, and the pixel value of the background area is close to zero; The difference between each pixel value in the target image to be processed and the trajectory residual image is calculated to generate a target image with the aircraft trajectory removed.

9. A method for astronomical image restoration, characterized in that, The method includes: Acquire an astronomical image to be processed; wherein the astronomical image to be processed is obtained by stretching and normalizing the original astronomical image as a preprocessing step; The astronomical image to be processed is segmented according to a preset segmentation size to obtain multiple image blocks; wherein each image block corresponds one-to-one with its spatial position in the astronomical image to be processed. According to the target scanning order, a target image block is selected sequentially from the plurality of image blocks to perform the aircraft trajectory processing method according to any one of claims 1-8 until the target image corresponding to each image block is obtained; wherein, if the aircraft trajectory detection result is no aircraft trajectory, the obtained target image is the corresponding image block itself; Generate a stitched image corresponding to each image block based on the target image corresponding to each image block; All image blocks are stitched together according to their corresponding spatial positions to generate the target astronomical image corresponding to the astronomical image to be processed.

10. The astronomical image restoration method according to claim 9, characterized in that, The process of segmenting the astronomical image to be processed according to a preset segmentation size to obtain multiple image blocks includes: With a preset sliding step size of a number of segmented pixels, the image is slid sequentially along the horizontal and vertical directions on the astronomical image to be processed to generate multiple overlapping image blocks; The number of pixels with the overlap width of adjacent image blocks in the horizontal and vertical directions is 1 / n of the number of segmented pixels of the sliding step, where n is greater than or equal to 2.

11. The astronomical image restoration method according to claim 10, characterized in that, Before generating multiple overlapping image patches, the method further includes: Determine whether the width and height of the astronomical image to be processed are both integer multiples of the preset number of segmented pixels; If it is determined that the width and height of the astronomical image to be processed are not both integer multiples of the preset number of segmented pixels, then pixel filling is performed on the edges of the astronomical image to be processed to obtain a pixel-filled astronomical image to be processed. In this process, the height and width of the astronomical image to be processed after pixel filling are both integer multiples of the preset number of segmented pixels.

12. The astronomical image restoration method according to claim 10, characterized in that, The step of generating a stitched image corresponding to each image block based on the target image corresponding to each image block includes: The target cropping width is determined based on the number of pixels of the overlap width of adjacent image blocks, wherein the target cropping width is smaller than the overlap width; The edge region of the target image corresponding to each image block with the target cropping width is defined as the target cropping region. Edge cropping is performed on the target cropping region corresponding to each target image to obtain the stitched image corresponding to each image block.

13. The astronomical image restoration method according to claim 12, characterized in that, The step of stitching together the images corresponding to all image blocks according to their corresponding spatial positions to generate the target astronomical image corresponding to the astronomical image to be processed includes: Based on the corresponding spatial position of each stitched image, place the stitched image at the corresponding position in the astronomical image to be processed; For the overlapping area between adjacent stitched images, obtain the fusion weight corresponding to each stitched image, wherein the fusion weight corresponding to each stitched image decreases linearly from the image center to the edge; Based on the fusion weight corresponding to each stitched image, the calculated weight of each pixel in the overlapping area is obtained in each stitched image. The calculated weight of each pixel in each stitched image is obtained by weight normalization of the fusion weight corresponding to each stitched image. For each pixel in the overlapping region, the product of its pixel value in each stitched image and its corresponding calculated weight is calculated to obtain the target pixel value of each pixel in the overlapping region, and the target astronomical image corresponding to the astronomical image to be processed is generated.

14. The astronomical image restoration method according to claim 13, characterized in that, The method further includes: Obtain the edge protection region of the astronomical image to be processed, wherein the edge protection region is a preset width region of the edge of the astronomical image to be processed; When generating the target astronomical image, the pixel values ​​of the edge protection region are taken from the original pixel values ​​of the astronomical image to be processed.

15. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 8 or 9 to 14.

16. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 8 or 9 to 14.

Images