Image processing methods, electronic devices and software products

Abstract

This application provides an image processing method, electronic device, and program product, relating to the field of astrophotography technology. The image processing method includes: performing background target removal processing on an acquired image containing a high-speed moving target using a depth model to obtain a background target image and a moving target image; overlaying multiple moving target images to obtain a first overlay image; overlaying multiple background target images to obtain a second overlay image; and fusing the first and second overlay images to obtain a target image. This application can separate an original acquired image containing a high-speed moving target into a background target image and a moving target image, perform overlay processing on each separately, and then fuse the two overlay images to obtain the final target image, effectively improving the quality of the target image and meeting the user's needs for capturing and imaging high-speed moving targets.

Description

Technical Field

[0001] This application relates to the field of astrophotography technology, and more specifically, to an image processing method, electronic device, and program product. Background Technology

[0002] Currently, in the field of astrophotography, image overlay is typically only used when photographing deep-sky targets (overlaying consecutive deep-sky images taken within a given time period). In this process, the deep-sky target can generally be considered relatively fixed (on the one hand, the equipment can track it using theodolites and equatorial mounts; on the other hand, even without tracking capabilities, the composition of the image is essentially static, so there are usually no relatively moving objects in the overlay image based on star alignment). This star alignment-based overlay method typically ensures that the target is essentially stationary relative to the background stars (or that the target is part of the background stars). Therefore, after aligning multiple images, the target can also be aligned for further overlaying.

[0003] However, when there is a rapidly moving object in the image (relative to the background stars, or relative to a deep-space target), the image of that object taken at different times will appear in different positions in the superimposed image, forming a continuous, blurred trajectory. Take a comet as an example: a comet consists of a center and a "tail," occupying a much larger relative area in the image than regular stars, and is also much brighter. Therefore, when superimposing comet images from different times, the bright center of the comet will form a straight line (possibly with a certain width). Because the comet's trajectory is difficult to predict, the direction of this line is also difficult to estimate. In the final image, the comet's structure is almost unrecognizable or at least distorted (due to slower speed or shorter shooting time). If superimposing the comet is abandoned and only single shots are taken, the exposure time for each shot cannot be too long because the comet moves relative to the background, resulting in less comet detail and poorer image quality. Simultaneously, because superimposition is impossible, the background stars will be relatively less clear, affecting the overall image quality.

[0004] Therefore, the image quality obtained from photographing fast-moving celestial bodies is currently poor and cannot meet the actual shooting needs of users. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide an image processing method, electronic device and program product to improve the problem of poor image quality obtained by photographing fast-moving celestial bodies in the prior art.

[0006] To address the aforementioned problems, in a first aspect, embodiments of this application provide an image processing method, the method comprising: Background target removal processing is performed on the acquired images containing high-speed moving targets using a deep model to obtain background target images and moving target images; Multiple images of the moving target are superimposed to obtain a first superimposed image; Multiple background target images are superimposed to obtain a second superimposed image; The target image is obtained by fusing the first and second overlaid images.

[0007] In the above implementation process, when processing the acquired image containing the high-speed moving target, considering the significant difference between the high-speed moving target and other targets in the background, the acquired image can first be processed to remove the background target using a depth model, resulting in a background target image containing the background target and a moving target image containing the high-speed moving target. To optimize the processing effect of the two types of images, they can be independently superimposed to obtain a first superimposed image and a second superimposed image, and then the two superimposed images are fused to obtain the final target image. It can separate the original captured image containing a high-speed moving target into a background target image and a moving target image for separate overlay processing. This avoids the unfavorable situation that occurs when directly overlaying, such as the high-speed moving target appearing as a bright straight line, a blurry bright block, or a gray shadow, making it impossible to effectively observe the morphology of the high-speed moving target structure. It also avoids the problem of insufficient image information caused by short exposure time in single shooting. Furthermore, it can use different overlay strategies to process the two types of images according to their characteristics, which can better show the structural details of different types of targets. Then, the two overlay images are fused to obtain the final target image. By fusion and post-processing of the two types of overlay images, the sense of disjointedness at the stitching boundary is eliminated, and the quality of the target image is improved in a way that is imperceptible to the user, meeting the user's needs for shooting and imaging high-speed moving targets.

[0008] Optionally, the step of overlaying multiple moving target images to obtain a first overlaid image includes: Each of the moving target images is preprocessed to obtain a preprocessed image; wherein the preprocessing includes at least one of grayscale processing, noise reduction processing, contrast processing, and binarization processing; Contour detection is performed on each of the preprocessed images to determine celestial candidate regions in the preprocessed images; Based on preset screening criteria, a moving target region is determined from the candidate celestial bodies. A first reference image is determined from among the plurality of preprocessed images; Alignment processing is performed on the reference moving target region and the moving target region in the first reference image, and multiple preprocessed images are superimposed on the first reference image to obtain the first superimposed image.

[0009] In the above implementation process, when overlaying multiple images containing high-speed moving targets, each moving target image can be preprocessed first to obtain a high-quality preprocessed image. Then, contour detection is performed on each preprocessed image to determine celestial candidate regions where high-speed moving targets may exist. Based on preset screening conditions, the moving target region containing the high-speed moving target is determined within the celestial candidate region. A first reference image can be selected from the multiple preprocessed images as the basis for overlay. The moving target region in the first reference image is used as the reference moving target region for overlay. The moving target regions in other preprocessed images are aligned with the reference moving target region, thereby overlaying the other preprocessed images onto the first reference image to obtain the corresponding first overlay image. This ability to identify the moving target region in the moving target image before overlay processing effectively improves the image quality of the final first overlay image.

[0010] Optionally, the screening conditions include at least one of the following: area conditions, morphological feature conditions, brightness distribution conditions, and orientation conditions; The process of determining the moving target region based on preset screening criteria within the celestial candidate region includes: Based on the area condition, multiple bounding boxes are determined in the candidate celestial body region; Based on the aforementioned morphological features, aspect ratio testing and shape fitting analysis are performed on multiple bounding boxes to determine the moving target region. And / or, based on the brightness distribution conditions, determine the brightness distribution of the region within the bounding box, and determine the moving target region based on the brightness distribution; And / or, based on the direction conditions, determine the direction angle within the bounding box, and based on the direction angle, determine the direction consistency to determine the moving target area.

[0011] In the above implementation process, the screening criteria used to determine the moving target region within the celestial candidate region can include one or more different types of criteria. First, area criteria can be used to determine multiple bounding boxes that may contain high-speed moving targets within the celestial candidate region. Then, based on morphological feature criteria, aspect ratio verification and shape fitting analysis are performed on these bounding boxes to determine the corresponding moving target region. Alternatively, due to the brightness differences between different types of targets, brightness distribution criteria can be used to determine the brightness distribution within the bounding box, thereby determining the moving target region. Or, because high-speed moving targets exhibit obvious directional characteristics, directional criteria can be used to determine the directional angle within the bounding box, and directional consistency can be judged based on the directional angle to determine the moving target region. This approach allows for the judgment of moving target regions from multiple different perspectives, such as area, morphological features, brightness distribution, and directional characteristics, thereby improving the effectiveness of identifying moving target regions.

[0012] Optionally, the step of overlaying multiple background target images to obtain a second overlaid image includes: Determine a second reference image associated with the first reference image from among the plurality of background target images; Identify multiple background targets in each of the background target images; Identify multiple reference background targets in the second reference image; Alignment processing is performed on multiple reference background targets and multiple background targets, and the images of multiple background targets are superimposed on the second reference image to obtain the second superimposed image.

[0013] In the above implementation process, when overlaying multiple background target images, to facilitate subsequent fusion processing, a second reference image associated with the first reference image can be determined among the multiple background target images, so that the two different overlay processes have associated overlay reference data. Multiple background templates are determined in each background target image, and multiple reference background targets are determined in the second reference image. Using the reference background targets in the second reference image as the reference for overlay, the background targets in other background target images are aligned with the reference background targets, thereby overlaying the other background target images onto the second reference image to obtain the corresponding second overlay image. This method enables the association and overlay of background target images with moving target images, and performs overlay processing on multiple background targets, effectively improving the image quality of the final second overlay image and the correlation between the first and second overlay images.

[0014] Optionally, determining the first reference image among the plurality of preprocessed images includes: Determine the user's selection time; The first reference image is determined based on the selection time and the acquisition time of the multiple moving target images.

[0015] In the above implementation process, when determining the first reference image, the acquisition time of multiple moving target images can be compared with the user's selection time to determine the first reference image among multiple preprocessed images, based on either the acquisition time being the same as or closest to the selection time. This allows for the selection of the appropriate first reference image according to the user's actual needs, satisfying various overlay requirements.

[0016] Optionally, the process of constructing the deep model includes: Construct an encoder with multiple downsampling stages; Construct a decoder with multiple upsampling paths; The depth model is obtained based on the input channel, the encoder, the decoder, and the output channel.

[0017] In the above implementation process, in order to achieve efficient background object removal processing, an encoder with multiple downsampling stages can be constructed to accelerate the process using depthwise separable convolutions through multiple downsampling stages, and an upsampling path decoder can be constructed to use transposed convolutions + skip connections, thereby combining the input and output channels to obtain the corresponding depth model capable of background object removal processing.

[0018] Optionally, the method further includes: Identify a first class of training datasets with background targets and a second class of training datasets without background targets; The deep model is trained for background target recognition based on the first type of training dataset and the second type of training dataset.

[0019] In the above implementation process, in order to further improve the effect of background target removal processing of deep models, a first type of training dataset with background targets and a second type of training dataset without background targets can be determined. The deep model is trained for background target recognition based on multi-class training datasets of different types, so that the deep model can achieve high-precision background target recognition and remove the background targets after recognition.

[0020] Optionally, the step of fusing the first overlay image and the second overlay image to obtain the target image includes: The second superimposed image is subjected to attenuation processing to obtain an attenuated image; The high-speed moving target in the first superimposed image is superimposed with the target boundary in the attenuated image by coordinate alignment, and other areas are superimposed by edge superposition to obtain the initial image; The initial image is post-processed to obtain the target image.

[0021] In the above implementation process, to highlight the high-speed moving target, the second overlay image can be attenuated to obtain an attenuated image. Then, the high-speed moving target in the first overlay image is superimposed with the target boundary in the attenuated image using coordinates. Edge superposition is performed on other areas to obtain an initial image after preliminary superposition. Further post-processing adjustments are then made to the initial image to improve its naturalness and realism, resulting in a target image of higher quality. During the fusion process of the first and second overlay images, partitioned superposition processing can be performed, effectively reducing the adverse interference caused by uniform superposition between different regions and improving the quality of the final target image.

[0022] Optionally, the post-processing of the initial image to obtain the target image includes: The parameters of the first region of the high-speed moving target and the second region of the background in the initial image are adjusted respectively to obtain an adjusted image; The stitching edges of the first region and the second region in the adjusted image are processed to obtain the target image.

[0023] In the above implementation process, to highlight the difference between the high-speed moving target and the background, the parameters of the first region of the high-speed moving target and the second region of the background can be adjusted separately during post-processing to obtain an adjusted image. To eliminate stitching artifacts between different regions, the stitching edges of the first and second regions in the initial image can be processed to obtain a target image with high naturalness, good realism, and a more prominent high-speed moving target.

[0024] Optionally, the step of performing background target removal processing on the acquired image containing the high-speed moving target using a depth model to obtain a background target image and a moving target image includes: Using the depth model, background target recognition is performed on the acquired image containing the high-speed moving target to obtain the identified target; Based on the identified target, the acquired image is processed to remove the background target, thus obtaining the moving target image; The moving target image is removed from the acquired image to obtain the background target image.

[0025] In the above implementation process, when performing background target removal processing, a deep model can first be used to identify background targets in the acquired image containing the high-speed moving target. This identifies background targets in the acquired image that differ from the high-speed moving target as the identification targets. Then, based on these identified targets, the acquired image is processed to remove background targets, resulting in a moving target image. Finally, the moving target image is removed from the acquired image, thus obtaining the corresponding background target image. This approach, based on a deep model, first identifies and processes background targets in the acquired image, and then combines this with the removal of the obtained moving target image to obtain the background target image. This effectively improves the separation effect of the acquired image, thereby enhancing the effectiveness of both the moving target image and the background target image.

[0026] Secondly, embodiments of this application also provide an electronic device, which includes a memory and a processor. The memory stores program instructions, and when the processor executes the program instructions, it performs the steps in any of the methods described in the first aspect above.

[0027] Thirdly, embodiments of this application also provide a computer program product, which includes a computer program / instruction that, when executed by a processor, implements the steps of the method described in any of the first aspects above.

[0028] In summary, the embodiments of this application provide an image processing method, electronic device, and program product that can separate an original acquired image containing a high-speed moving target into a background target image and a moving target image, perform superposition processing on each image, and then perform fusion processing on the two superimposed images to obtain the final target image. This effectively improves the quality of the target image and meets the user's needs for shooting and imaging high-speed moving targets. Attached Figure Description

[0029] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 A block diagram illustrating an electronic device provided in an embodiment of this application; Figure 2 A schematic flowchart of an image processing method provided in an embodiment of this application; Figure 3 A detailed flowchart of step S300 provided for an embodiment of this application; Figure 4A detailed flowchart of step S330 provided for an embodiment of this application; Figure 5 A detailed flowchart of step S400 provided for an embodiment of this application; Figure 6 A detailed flowchart of step S340 provided for an embodiment of this application; Figure 7 A flowchart illustrating another image processing method provided in an embodiment of this application; Figure 8 A schematic flowchart illustrating another image processing method provided in an embodiment of this application; Figure 9 A detailed flowchart of step S500 provided for an embodiment of this application; Figure 10 A detailed flowchart of step S530 provided for an embodiment of this application; Figure 11 This is a detailed flowchart of step S200 provided in an embodiment of this application.

[0031] Icons: 100 - Electronic device; 111 - Memory; 112 - Memory controller; 113 - Processor; 114 - Peripheral interface; 115 - Input / output unit; 116 - Display unit. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of the embodiments of this application.

[0033] In existing image capture methods, when a rapidly moving object (relative to the background stars or deep-space targets) is present in the frame, the image of that object captured at different times will appear in different positions in the superimposed image, creating a continuous, blurred trajectory. Take a comet as an example: a comet consists of a center and a "tail," occupying a much larger relative area in the image than regular stars and is also much brighter. Therefore, when superimposing comet images from different times, the bright comet center will form a straight line (possibly with a certain width). Since the comet's trajectory is difficult to predict, the direction of this line is also difficult to estimate. In the final image, the comet's structure is almost unrecognizable or at least distorted (due to slower speed or shorter capture time). If superimposing the comet is abandoned and only single shots are used, the exposure time for each shot cannot be too long because the comet moves relative to the background, resulting in less comet detail and poorer image quality. Simultaneously, because superimposition is impossible, the background stars will be relatively less clear, affecting the overall image quality. Therefore, the image quality obtained from photographing fast-moving celestial bodies is currently poor and cannot meet the actual shooting needs of users.

[0034] To address the aforementioned issues, embodiments of this application provide an image processing method, electronic device, and program product that can separate an original acquired image containing a high-speed moving target into a background target image and a moving target image, perform overlay processing on each, and then fuse the two overlay images to obtain the final target image. This effectively improves the quality of the target image and meets the user's needs for capturing and imaging high-speed moving targets.

[0035] This application provides an image processing method applied to an electronic device, which can be a server, personal computer (PC), tablet computer, smartphone, personal digital assistant (PDA), or other electronic device with logical computing functions. The method can process the original acquired image to obtain a high-quality target image containing a high-speed moving target.

[0036] Optionally, please refer to Figure 1 , Figure 1 This is a block diagram illustrating an electronic device according to an embodiment of this application. The electronic device 100 may include a memory 111, a memory controller 112, a processor 113, a peripheral interface 114, an input / output unit 115, and a display unit 116. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device 100. For example, the electronic device 100 may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1The different configurations shown.

[0037] The aforementioned memory 111, memory controller 112, processor 113, peripheral interface 114, input / output unit 115, and display unit 116 are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The aforementioned processor 113 is used to execute executable modules stored in the memory.

[0038] The memory 111 can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 111 stores programs. After receiving execution instructions, the processor 113 executes the programs. The methods executed by the electronic device 100 as defined in any embodiment of this application can be applied to the processor 113, or implemented by the processor 113.

[0039] The aforementioned processor 113 may be an integrated circuit chip with signal processing capabilities. The processor 113 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a digital signal processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor.

[0040] The peripheral interface 114 described above couples various input / output devices to the processor 113 and the memory 111. In some embodiments, the peripheral interface 114, the processor 113, and the memory controller 112 can be implemented on a single chip. In other instances, they can be implemented on separate chips.

[0041] The input / output unit 115 described above is used to provide user input data. The input / output unit 115 can be, but is not limited to, a mouse and a keyboard.

[0042] The aforementioned display unit 116 provides an interactive interface (e.g., a user interface) between the electronic device 100 and the user, or displays image data for the user's reference. In this embodiment, the display unit can be a liquid crystal display (LCD) or a touch display. If it is a touch display, it can be a capacitive touchscreen or a resistive touchscreen that supports single-point and multi-point touch operations. Supporting single-point and multi-point touch operations means that the touch display can sense touch operations generated simultaneously from one or more locations on the touch display and pass the sensed touch operations to the processor for calculation and processing. In this embodiment, the display unit 116 can display various types of images, such as a background target image, a moving target image, a first overlay image, a second overlay image, and a target image.

[0043] The electronic device in this embodiment can be used to execute various steps in the various image processing methods provided in the embodiments of this application. The implementation process of the image processing methods is described in detail below through several embodiments.

[0044] Please see Figure 2 , Figure 2 This is a flowchart illustrating an image processing method provided in an embodiment of this application. The method may include steps S200-S500.

[0045] Step S200: The acquired image containing the high-speed moving target is processed by a depth model to remove the background target, resulting in a background target image and a moving target image.

[0046] When processing images containing high-speed moving targets, considering the significant differences between high-speed moving targets and other targets in the background, the images can first be processed using a depth model to remove background targets, resulting in a background target image containing the background target and a moving target image containing the high-speed moving target.

[0047] Optionally, taking astrophotography as an example, the acquired images may include acquired sky images, high-speed moving targets may include targets such as comets, and background targets may include celestial objects such as stars in the background.

[0048] Optionally, the deep model can be a lightweight U-Net model or other fast deep learning models that can identify background targets in the acquired image for background target removal processing.

[0049] It should be noted that in shooting scenarios without a landscape (such as buildings, people, or other non-celestial targets), the support structure of the shooting equipment for acquiring images can be an equatorial mount, or a theodolite that supports equatorial mount mode, to eliminate the influence of field rotation on the shooting, allowing for longer single-shot exposure times and improving the quality of the acquired images. Unlike deep-space photography, when using an equatorial mount / equatorial mount mode, the position of deep-space targets in the frame is ideally always fixed. However, the position and size of high-speed moving targets will change (although the changes are small). Therefore, a longer single-shot exposure time does not necessarily yield better results; after selecting a certain exposure time, using interval shooting can obtain better final results.

[0050] Step S300: Overlay multiple moving target images to obtain a first overlay image.

[0051] Step S400: Multiple background target images are superimposed to obtain a second superimposed image.

[0052] In order to optimize the processing effect of the two types of images, the two images can be independently superimposed to obtain the corresponding first superimposed image and second superimposed image.

[0053] Optionally, due to the difference in target types between the moving target image and the background target image, different overlay methods can be used to further optimize the quality of the first and second overlay images.

[0054] It should be noted that superimposing multiple background target images can produce a second superimposed image with lower noise, while superimposing multiple moving target images can produce a first superimposed image with a higher signal-to-noise ratio.

[0055] Step S500: Perform fusion processing based on the first superimposed image and the second superimposed image to obtain the target image.

[0056] In order to obtain the final image of a high-speed moving target, the two superimposed images can be fused to obtain the final target image.

[0057] Optionally, the fusion process may include various processing methods such as coordinate alignment and fusion to obtain a complete target image that includes both high-speed moving targets and background targets.

[0058] It should be noted that during the shooting process using the shooting equipment, each captured image can be separated and superimposed. The image is then superimposed on the latest captured image or the image obtained from the previous superposition, and the image is output so that the user can observe the latest shooting results in real time.

[0059] exist Figure 2 In the illustrated embodiment, the original acquired image containing the high-speed moving target can be separated into a background target image and a moving target image, which are then overlaid separately. This avoids the unfavorable situation that occurs when directly overlaying, where the high-speed moving target appears as a bright straight line, a blurry bright block, or a gray shadow, making it impossible to effectively observe the morphology of the high-speed moving target structure. It also avoids the problem of insufficient image information caused by short exposure time when shooting a single image. Furthermore, different overlay strategies can be used to process the two types of images according to their characteristics, which can better display the structural details of different types of targets. The two overlaid images are then fused to obtain the final target image. By fusing the two types of overlaid images in post-processing, the sense of disjointedness at the stitching boundary is eliminated, and the quality of the target image is improved in a way that is imperceptible to the user, thus meeting the user's needs for shooting and imaging high-speed moving targets.

[0060] Optionally, please refer to Figure 3 , Figure 3 The following is a detailed flowchart of step S300 provided in an embodiment of this application. Step S300 may include steps S310-S350.

[0061] Step S310: Preprocess each moving target image to obtain a preprocessed image.

[0062] When overlaying multiple images of moving targets containing high-speed moving targets, each moving target image can be preprocessed to obtain a high-quality preprocessed image. Preprocessing can include one or more of the following methods: grayscale processing, noise reduction, contrast enhancement, and binarization.

[0063] Optionally, the preprocessing steps may include: Grayscale conversion: converting the color moving target image to a grayscale image to simplify subsequent processing. Gaussian filtering denoising: smoothing the grayscale image using a Gaussian blur filter (typically a 5×5 or 7×7 kernel) to reduce the impact of random noise. Contrast enhancement: enhancing the overall contrast of the image through histogram equalization or linear stretching to make the features of the dim, high-speed moving target more prominent. Adaptive binarization: due to the significant brightness difference between the high-speed moving target and the background, a threshold can be set based on the typical brightness of the high-speed moving target (e.g., using 3 times the standard deviation) to convert the image into a binary image, turning bright areas white and dark backgrounds black.

[0064] Step S320: Perform contour detection on each preprocessed image to determine celestial candidate regions in the preprocessed image.

[0065] Since the preprocessed images still contain the overall situation of the captured sky region, in order to determine the location of high-speed moving targets in the images, contour detection can be performed on each preprocessed image to identify celestial candidate regions in which high-speed moving targets may exist.

[0066] Optionally, the celestial candidate region can be a rectangular region, and contour detection can be performed using edge detection or connected components to find all white region contours in the binary preprocessed image as the corresponding celestial candidate region.

[0067] Step S330: Based on preset screening conditions, determine the moving target region in the celestial candidate region.

[0068] Since there will inevitably be other undetected background targets in the preprocessed image, in order to improve the detection accuracy of high-speed moving targets, the moving target area containing high-speed moving targets can be determined in the celestial candidate area according to the preset screening conditions.

[0069] Optionally, the filtering criteria can be based on the characteristics of high-speed moving targets, such as different types of criteria related to brightness, size, etc.

[0070] Step S340: Determine a first reference image among multiple preprocessed images.

[0071] Step S350: Alignment processing is performed based on the reference moving target region and the moving target region in the first reference image, and multiple preprocessed images are superimposed on the first reference image to obtain the first superimposed image.

[0072] Among them, a first reference image can be selected as the reference for superposition among multiple preprocessed images. The moving target region in the first reference image is used as the reference moving target region for superposition. The moving target regions in other preprocessed images are aligned with the reference moving target region, so that the other preprocessed images are superimposed on the first reference image to obtain the corresponding first superimposed image.

[0073] Optionally, the first reference image can be selected according to the user's actual needs, or the first preprocessed image can be used as the default reference image.

[0074] Optionally, since the target area to be moved can be a corresponding rectangle, the target area to be moved can be aligned to achieve overlay processing.

[0075] Optionally, multiple preprocessed images can be superimposed one by one to obtain the final first superimposed image. During the image-by-image superposition process, the correspondence between the pixel coordinates of the superimposed image and the pixel coordinates of the first reference image can be preserved, or the same set of coordinates can be used directly for superposition.

[0076] exist Figure 3 In the illustrated embodiment, the moving target region in the moving target image can be identified before being overlaid, which effectively improves the image quality of the final overlaid image.

[0077] It should be noted that the screening criteria may include one or more of the following: area criteria, morphological feature criteria, brightness distribution criteria, and orientation criteria.

[0078] Optionally, please refer to Figure 4 , Figure 4 The following is a detailed flowchart of step S330 provided in an embodiment of this application. Step S330 may include steps S331-S334.

[0079] Step S331: Based on the area condition, determine multiple bounding boxes in the celestial candidate region.

[0080] Since high-speed moving targets, such as comets, usually occupy a large area in the image, multiple bounding boxes that may contain high-speed moving targets can be determined in the celestial candidate region first by using the area condition.

[0081] Optionally, the area condition can be a relevant threshold for the area. For example, a preliminary screening can be performed based on the set area threshold: excluding areas that are too small (<50 pixels): which may be noise points; excluding areas that are too large (>10000 pixels): which may be galaxies or image defects, to determine the corresponding bounding box. The bounding box is the smallest bounding rectangle calculated for each celestial candidate region retained after screening.

[0082] Step S332: Based on morphological features, perform aspect ratio checks and shape fitting analyses on multiple bounding boxes to determine the target region for movement.

[0083] Since high-speed moving targets, such as comets, have long tails and differ in shape from other targets, aspect ratio verification and shape fitting analysis can be performed on multiple bounding boxes based on morphological characteristics to determine the corresponding moving target region.

[0084] Optionally, morphological feature conditions may include: Aspect ratio test: Calculate the aspect ratio of the bounding box for each candidate region. Comets are typically elongated, with an aspect ratio generally greater than 1.3-1.8; stars are nearly circular, with an aspect ratio close to 1.0. Ellipse fitting analysis: Perform ellipse fitting on the contour of each bounding box and calculate the ratio of the major axis to the minor axis of the fitted ellipse. Due to the elongation characteristics of comets, the ratio of the major axis to the minor axis is usually greater than 1.2. If the morphological feature conditions are met, the region of the bounding box is considered the target region for movement.

[0085] And / or, in step S333, based on the brightness distribution conditions, determine the brightness distribution of the region within the bounding box, and based on the brightness distribution conditions, determine the moving target region.

[0086] Since there are differences in brightness between different types of targets, high-speed moving targets, such as comets, have high brightness. Therefore, the brightness distribution of the region within the bounding box can be determined based on the brightness distribution conditions, so as to determine the region of the moving target based on the brightness distribution.

[0087] Optionally, since comets typically have asymmetrical brightness distributions and significant brightness gradients (decreasing from the nucleus to the tail), brightness distribution conditions can include: calculating the brightness centroid (weighted center) of each region; brightness gradient analysis: analyzing brightness variations along the possible principal axis directions of the comet to determine the brightness distribution. The calculation method for the brightness distribution can include: determining the principal axis direction of the bounding box through second-order moment calculations, sampling brightness values ​​at multiple points along the principal axis direction, and calculating the average brightness difference between adjacent sampling points to obtain the corresponding brightness distribution. Since comets typically have high brightness gradient values, the condition value set in the brightness distribution conditions can be greater than 3 times the variance threshold; that is, if the brightness gradient value in the brightness distribution conditions is greater than the condition value, then the region within the bounding box is considered a moving target region.

[0088] And / or, in step S334, based on the orientation condition, determine the orientation angle within the bounding box, and based on the orientation angle, determine the orientation consistency to determine the target area for movement.

[0089] Since high-speed moving targets have obvious directional characteristics, the directional angles within the bounding box can be determined based on directional conditions. The consistency of direction can be judged based on the directional angles, thereby determining the moving target area.

[0090] Optionally, the orientation condition may include an orientation consistency check, which may involve moment analysis: calculating the central moment and standard moment of the bounding box region, and calculating the orientation angle of the region using the second-order central moment to verify whether the brightness distribution is consistent with the calculated orientation.

[0091] It should be noted that multiple different screening conditions can be combined for comprehensive decision-making and verification to achieve multi-feature fusion judgment. A comprehensive score is calculated for each bounding box. Judgment conditions may include: whether the aspect ratio meets the standard, whether the ellipse elongation meets the standard, whether the brightness gradient meets the standard, whether the orientation consistency meets the standard, etc., to further improve the effectiveness of moving target regions.

[0092] exist Figure 4 In the illustrated embodiment, the moving target area can be judged from multiple different perspectives such as area, morphological features, brightness distribution, and direction, thereby improving the effectiveness of the moving target area.

[0093] Optionally, please refer to Figure 5 , Figure 5 This is a detailed flowchart of step S400 provided in an embodiment of the present application. Step S400 may include steps S410-S440.

[0094] Step S410: Determine a second reference image associated with the first reference image among multiple background target images.

[0095] In the process of overlaying multiple background target images, in order to facilitate subsequent fusion processing, a second reference image associated with the first reference image can be determined among the multiple background target images, so that the two different overlay processes have associated overlay reference data.

[0096] For example, when the first reference image is the first preprocessed image, the corresponding second reference image is the background target image associated with the moving target image corresponding to the first preprocessed image.

[0097] Step S420: Identify multiple background targets in each background target image.

[0098] Step S430: Identify multiple reference background targets in the second reference image.

[0099] In this process, multiple background templates can be determined in each background target image, and multiple reference background targets can be determined in the second reference image.

[0100] Step S440: Alignment processing is performed based on multiple reference background targets and multiple background targets, and the multiple background target images are superimposed on the second reference image to obtain the second superimposed image.

[0101] In this process, the reference background target in the second reference image is used as the reference for overlay. The background targets in other background target images are aligned with the reference background target, so that the other background target images are overlaid on the second reference image to obtain the corresponding second overlay image.

[0102] Optionally, alignment can be performed based on the coordinates and positional relationships of multiple background targets. Taking stars as an example, image information can be superimposed after star point matching and alignment. Since the position of stars in the sky is fixed, while the Earth's rotation causes stars to move, alignment needs to be performed according to the position of the stars. For shape differences existing in different background target images, overlapping of repeated parts and retention of individual parts (newly added parts of the image) can be performed. The maximum range of image information retained is based on the second reference image, and the expanded parts will be cropped to maintain consistency.

[0103] Optionally, multiple background target images can be superimposed one by one to obtain the final second superimposed image. During the image-by-image superposition process, the correspondence between the pixel coordinates of the superimposed image and the pixel coordinates of the second reference image can be preserved, or the same set of coordinates can be used directly for superposition.

[0104] exist Figure 5 In the illustrated embodiment, the background target image and the moving target image can be correlated and superimposed, and multiple background targets can be superimposed, which effectively improves the image quality of the final second superimposed image and the correlation between the first superimposed image and the second superimposed image.

[0105] Optionally, please refer to Figure 6 , Figure 6 The following is a detailed flowchart of step S340 provided in an embodiment of this application. Step S340 may include steps S341-S342.

[0106] Step S341: Determine the user's selected time.

[0107] In determining the first reference image, the user-defined selection time can be obtained and used as the data basis for selecting the first reference image.

[0108] Step S342: Determine the first reference image based on the selection time and the acquisition time of multiple moving target images.

[0109] Specifically, the acquisition time of multiple moving target images can be compared with the user's selected time to determine the first reference image among multiple preprocessed images, which has the same acquisition time as the selected time or the acquisition time is closest to the selected time.

[0110] exist Figure 6 In the illustrated embodiment, the corresponding first reference image can be selected according to the user's actual needs to meet the user's various overlay requirements.

[0111] Optionally, please refer to Figure 7 , Figure 7This is a flowchart illustrating another image processing method provided in an embodiment of this application. The process of constructing a depth model may include steps S610-S630.

[0112] Step S610: Construct an encoder with multiple downsampling stages.

[0113] In order to achieve efficient background object removal processing, an encoder with multiple downsampling stages can be constructed to accelerate the process using depthwise separable convolutions through multiple downsampling stages.

[0114] Optionally, the encoder design process may include: designing four downsampling stages, using depthwise separable convolutions for acceleration: Stage1:Conv3x3(3→64)+ReLU→MaxPool2x2; Stage2:DepthwiseSepConv(64→128)→MaxPool2x2; Stage3:DepthwiseSepConv(128→256)→MaxPool2x2; Stage 4: DepthwiseSepConv(256→512)→bottleneck layer.

[0115] Step S620: Construct a decoder with multiple upsampling paths.

[0116] Among these, a decoder for upsampling paths can be constructed to use transposed convolutions with skip connections.

[0117] Optionally, the decoder design process may include: using transposed convolutions + skip connections: Upsampling 4x → Convolution → Concatenation with encoder features → Convolution; Upsampling 2x → Convolution → Concatenation with encoder features → Convolution; Upsample 2x → Convolution → Concatenate with encoder features → Output layer.

[0118] Step S630: Obtain the depth model based on the input channel, encoder, decoder and output channel.

[0119] By combining the input and output channels, a corresponding depth model capable of removing background targets can be obtained.

[0120] Optionally, the input channel can be 3-channel RGB, and the output channel can be a 3-channel moving target image. Background removal is performed using the constructed deep model. Due to depthwise separable convolution, computation can be reduced by 70-80%. The deep model has controlled the number of channels, with a maximum of 512 channels. Speed ​​has been optimized by incorporating early-skip connections in shallow layers to avoid information loss. Furthermore, processing is based on lightweight upsampling, using bilinear interpolation + convolution instead of pure transposed convolution, further optimizing processing speed.

[0121] exist Figure 7 In the illustrated embodiment, a lightweight deep model can be constructed for processing, effectively improving the processing speed.

[0122] Optionally, please refer to Figure 8 , Figure 8 This is a flowchart illustrating another image processing method provided in an embodiment of this application. The process of constructing a depth model may further include steps S640-S650.

[0123] Step S640: Determine the first type of training dataset with background targets and the second type of training dataset without background targets.

[0124] In order to further improve the effect of background object removal processing of deep models, a first type of training dataset with background objects and a second type of training dataset without background objects can be identified.

[0125] Optionally, the first type of training dataset may contain image data of multiple stars, while the second type of training dataset may contain image data of only high-speed moving targets.

[0126] Step S650: Based on the first type of training dataset and the second type of training dataset, train the deep model for background target recognition.

[0127] Among them, deep models can be trained for background target recognition based on different types of multi-class training datasets, so that deep models can achieve high-precision background target recognition and remove the background targets after they are recognized.

[0128] exist Figure 8 In the illustrated embodiment, the processing performance of deep models can be effectively improved based on data training.

[0129] Optionally, please refer to Figure 9 , Figure 9 The following is a detailed flowchart of step S500 provided in an embodiment of this application. Step S500 may include steps S510-S530.

[0130] Step S510: Perform attenuation processing on the second superimposed image to obtain an attenuated image.

[0131] In order to highlight high-speed moving targets, the second superimposed image can be attenuated to obtain an attenuated image.

[0132] Optionally, when performing attenuation processing, an appropriate brightness attenuation coefficient can be set according to actual needs, such as 0.75-0.85. The second superimposed image is then multiplied by the attenuation coefficient to obtain the corresponding attenuated image.

[0133] Step S520: Align the high-speed moving target in the first superimposed image with the target boundary in the attenuated image and superimpose them with coordinates, and perform edge superposition on other areas to obtain the initial image.

[0134] Specifically, the high-speed moving target in the first superimposed image can be superimposed with the target boundary in the attenuated image using coordinates, and other areas can be edge superimposed to obtain the initial superimposed image.

[0135] Optionally, the high-speed moving target can be the final moving target area in the first overlay image. The boundary of the moving target area can be aligned with the boundary coordinates of the background image. For other areas outside the boundaries of the moving target area and the background image, overlay processing can be performed using feathering, blending modes, layer masks, and gradients.

[0136] Step S530: Post-process the initial image to obtain the target image.

[0137] Further post-processing adjustments can be made to the initial image to improve its naturalness and realism, resulting in a target image with higher quality.

[0138] exist Figure 9 In the embodiment shown, during the fusion process of the first and second superimposed images, partition superimposition processing can be performed, which effectively reduces the adverse interference caused by uniform superimposition between different regions and improves the quality of the final target image.

[0139] Optionally, please refer to Figure 10 , Figure 10 The following is a detailed flowchart of step S530 provided in an embodiment of this application. Step S530 may include steps S531-S532.

[0140] Step S531: Adjust the parameters of the first region of the high-speed moving target and the second region of the background in the initial image to obtain the adjusted image.

[0141] In order to highlight the difference between the high-speed moving target and the background, the parameters of the first region of the high-speed moving target and the second region of the background can be adjusted separately during post-processing to obtain an adjusted image.

[0142] Optionally, parameters such as white balance, exposure, and color temperature can be adjusted for the images of the first and / or second regions respectively.

[0143] Step S532: Process the splicing edge of the first region and the second region in the adjusted image to obtain the target image.

[0144] In order to eliminate stitching marks between different regions, the stitching edges of the first and second regions in the initial image can be processed to obtain a target image with high naturalness, good realism, and more obvious high-speed moving targets.

[0145] Optionally, the splicing edge of the first and second regions can be processed through physical transition, brightness adjustment, color adjustment, texture blending, and environment blending.

[0146] exist Figure 10 In the illustrated embodiment, the quality of the target image can be further improved through post-processing.

[0147] Optionally, please refer to Figure 11 , Figure 11 The following is a detailed flowchart of step S200 provided in an embodiment of this application. Step S200 may include steps S210-S230.

[0148] Step S210: Using a depth model, background target recognition is performed on the acquired image containing high-speed moving targets to obtain the identified targets.

[0149] In the background target removal process, a deep model can be used to identify background targets in the acquired image containing high-speed moving targets, so as to identify background targets in the acquired image that are different from high-speed moving targets as the identification targets.

[0150] Step S220: Based on the identified target, the acquired image is processed to remove the background target, resulting in a moving target image.

[0151] Step S230: Remove the moving target image from the acquired image to obtain the background target image.

[0152] Among these methods, background target removal processing can be performed on the acquired image based on the identified target to obtain a moving target image. The obtained moving target image can then be removed from the acquired image to obtain the corresponding background target image.

[0153] Alternatively, the corresponding background target image can be obtained by subtracting the moving target image from the acquired image.

[0154] exist Figure 11 In the illustrated embodiment, the background target in the acquired image can be identified and processed based on the depth model, and then the background target image can be removed by combining the obtained moving target image. This effectively improves the separation effect of the acquired image, thereby improving the effectiveness of the moving target image and the background target image.

[0155] This application also provides a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, they implement the steps in any of the image processing methods described above.

[0156] In the several embodiments provided in this application, it should be understood that the disclosed device can also be implemented in other ways. The device embodiments described above are merely illustrative; for example, the block diagrams in the accompanying drawings illustrate the possible architecture, functions, and operations of the device according to various embodiments of this application. In this regard, each block in the block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram, and combinations of block diagrams, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0157] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0158] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0159] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0160] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

[0161] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. An image processing method, characterized by, The method includes: Background target removal processing is performed on the acquired images containing high-speed moving targets using a deep model to obtain background target images and moving target images; Multiple images of the moving target are superimposed to obtain a first superimposed image; Multiple background target images are superimposed to obtain a second superimposed image; The target image is obtained by fusing the first and second overlaid images.

2. The method of claim 1, wherein, The process of overlaying multiple moving target images to obtain a first overlaid image includes: Each of the moving target images is preprocessed to obtain a preprocessed image; wherein the preprocessing includes at least one of grayscale processing, noise reduction processing, contrast processing, and binarization processing; Contour detection is performed on each of the preprocessed images to determine celestial candidate regions in the preprocessed images; Based on preset screening criteria, a moving target region is determined from the candidate celestial bodies. A first reference image is determined from among the plurality of preprocessed images; Alignment processing is performed on the reference moving target region and the moving target region in the first reference image, and multiple preprocessed images are superimposed on the first reference image to obtain the first superimposed image.

3. The method of claim 2, wherein, in, The screening criteria include at least one of the following: area criteria, morphological feature criteria, brightness distribution criteria, and orientation criteria; The process of determining the moving target region based on preset screening criteria within the celestial candidate region includes: Based on the area condition, multiple bounding boxes are determined in the candidate celestial body region; Based on the aforementioned morphological features, aspect ratio testing and shape fitting analysis are performed on multiple bounding boxes to determine the moving target region. And / or, based on the brightness distribution conditions, determine the brightness distribution of the region within the bounding box, and determine the moving target region based on the brightness distribution; And / or, based on the direction conditions, determine the direction angle within the bounding box, and based on the direction angle, determine the direction consistency to determine the moving target area.

4. The method of claim 2, wherein, The step of overlaying multiple background target images to obtain a second overlaid image includes: Determine a second reference image associated with the first reference image from among the plurality of background target images; Identify multiple background targets in each of the background target images; Identify multiple reference background targets in the second reference image; Alignment processing is performed on multiple reference background targets and multiple background targets, and the images of multiple background targets are superimposed on the second reference image to obtain the second superimposed image.

5. The method of claim 2, wherein, Determining the first reference image among the plurality of preprocessed images includes: Determine the user's selection time; The first reference image is determined based on the selection time and the acquisition time of the multiple moving target images.

6. The method according to any one of claims 1-5, characterized in that, in, The process of constructing the deep model includes: Construct an encoder with multiple downsampling stages; Construct a decoder with multiple upsampling paths; The depth model is obtained based on the input channel, the encoder, the decoder, and the output channel.

7. The method of claim 6, wherein, The method further includes: Identify a first class of training datasets with background targets and a second class of training datasets without background targets; The deep model is trained for background target recognition based on the first type of training dataset and the second type of training dataset.

8. The method according to any one of claims 1-5, characterized in that, The step of fusing the first overlay image and the second overlay image to obtain the target image includes: The second superimposed image is subjected to attenuation processing to obtain an attenuated image; The high-speed moving target in the first superimposed image is superimposed with the target boundary in the attenuated image by coordinate alignment, and other areas are superimposed by edge superposition to obtain the initial image; The initial image is post-processed to obtain the target image.

9. The method of claim 8, wherein, The post-processing of the initial image to obtain the target image includes: The parameters of the first region of the high-speed moving target and the second region of the background in the initial image are adjusted respectively to obtain an adjusted image; The stitching edges of the first region and the second region in the adjusted image are processed to obtain the target image.

10. The method according to any one of claims 1-5, characterized in that, The process of removing background targets from acquired images containing high-speed moving targets using a depth model to obtain background target images and moving target images includes: Using the depth model, background target recognition is performed on the acquired image containing the high-speed moving target to obtain the identified target; Based on the identified target, the acquired image is processed to remove the background target, thus obtaining the moving target image; The moving target image is removed from the acquired image to obtain the background target image.

11. An electronic device, comprising: The electronic device includes a memory and a processor. The memory stores program instructions, and when the processor executes the program instructions, it performs the steps of the method according to any one of claims 1-10.

12. A computer program product, characterised in that, The computer program product includes a computer program / instruction that, when executed by a processor, implements the steps of the method according to any one of claims 1-10.

Images