Image processing method and device

Abstract

The embodiments of the present application provide an image processing method and device. The method includes: based on a cross-attention mechanism, capturing a moving object between a first image feature and a second image feature, and determining an intermediate feature between the first image feature and the second image feature; determining, based on the intermediate feature between the first image feature and the second image feature, a position offset of the moving object between the first image feature and the second image feature, and a position offset feature of the position offset; based on the position offset feature of the moving object between the first image feature and the second image feature, performing a deformable alignment process on the second image feature to the first image feature to obtain an alignment feature; obtaining a merged image feature based on the alignment feature and the first image feature; and determining a target image based on the merged image feature. This method is used to improve image quality.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method and device. Background Art

[0002] With the advancement of digital imaging technology, the demand for image quality is increasing across various fields. Traditional low dynamic range (LDR) images can only represent limited brightness information. When faced with scenes with significant brightness differences, they often overexpose bright areas, resulting in a loss of detail and a failure to truly reflect what the human eye sees, impacting the visual experience. To address this issue, high dynamic range (HDR) imaging technology captures multiple images at varying exposures and fuses them into a single HDR image with a wide brightness range, improving visual quality and user experience.

[0003] Existing HDR imaging techniques typically involve capturing multiple LDR images at different exposures, aligning these images to eliminate displacement errors caused by object motion, and merging the aligned images into an HDR image.

[0004] However, due to the possible long-distance object motion during the shooting process, existing alignment algorithms are difficult to effectively handle, resulting in problems such as ghosting artifacts in the synthesized HDR images that affect image quality. Summary of the Invention

[0005] The embodiments of the present application provide an image processing method and device to achieve the effect of improving image quality.

[0006] In a first aspect, an embodiment of the present application provides an image processing method, comprising:

[0007] Based on the cross-attention mechanism, a moving object between a first image feature and a second image feature is captured, and an intermediate feature between the first image feature and the second image feature is determined, where the first image feature is an image feature in a first multi-channel image, and the second image feature is an image feature in a second multi-channel image, and the first multi-channel image and the second multi-channel image have different exposures;

[0008] determining, based on an intermediate feature between the first image feature and the second image feature, a position offset of the moving object between the first image feature and the second image feature, and a position offset feature of the position offset;

[0009] performing deformable alignment processing on the second image feature to the first image feature according to a position offset feature of the moving object between the first image feature and the second image feature to obtain an alignment feature;

[0010] Obtaining a merged image feature according to the alignment feature and the first image feature;

[0011] The target image is determined based on the merged image features.

[0012] In one possible implementation, based on a cross-attention mechanism, capturing a moving object between a first image feature and a second image feature, and determining an intermediate feature between the first image feature and the second image feature include:

[0013] determining a first window characteristic of the first image characteristic and a second window characteristic of the second image characteristic;

[0014] Based on the cross attention mechanism, the first moving object between the first window feature and the second window feature is captured to obtain the initial intermediate feature;

[0015] Obtaining a first adjustment feature and a second adjustment feature according to the first window feature and the initial intermediate feature;

[0016] Based on the cross attention mechanism, the second moving object between the first adjustment feature and the second adjustment feature is captured to obtain an intermediate feature, wherein the movement distance of the first moving object is smaller than the movement distance of the second moving object.

[0017] In a possible implementation, obtaining the first adjustment feature and the second adjustment feature according to the initial intermediate feature and the second image feature includes:

[0018] Perform window merging on the first window feature and the initial intermediate feature respectively to obtain the first window merged feature and the second window merged feature after the window merging;

[0019] Downsampling is performed on the first window merged feature and the second window merged feature to obtain a first adjusted feature and a second adjusted feature.

[0020] In one possible implementation, the method further includes:

[0021] Performing nonlinear transformation processing on the first low-dynamic image to obtain a first transformed image;

[0022] The first transformed image and the first low-dynamic image are spliced in a channel dimension to obtain a first multi-channel image.

[0023] In one possible implementation, the method further includes:

[0024] Inputting the first multi-channel image into the convolution block for convolution processing to obtain a first initial feature;

[0025] The first initial feature is input into the residual block for residual processing to obtain the first image feature.

[0026] In one possible implementation, after performing deformable alignment processing on the second image feature to the first image feature based on a position offset feature of a moving object between the first image feature and the second image feature to obtain an alignment feature, the method further includes:

[0027] The alignment feature is used as the second image feature, and the steps of capturing the moving object between the first image feature and the second image feature and determining the intermediate feature between the first image feature and the second image feature are repeatedly executed based on the cross-attention mechanism until the number of repetitions meets the preset number of requirements, thereby obtaining the alignment feature that meets the number of repetitions.

[0028] In a possible implementation, obtaining a merged image feature according to the alignment feature and the first image feature includes:

[0029] performing splicing processing on the alignment feature and the first image feature to obtain a spliced image feature;

[0030] The spliced image feature and the first image feature are subjected to feature merging processing to obtain a merged image feature.

[0031] In a possible implementation, performing feature merging processing on the spliced image feature and the first image feature to obtain the merged image feature includes:

[0032] Determining spatial dynamic weights of stitched image features;

[0033] According to the spatial dynamic weight and the spliced image features, the initial merged features are obtained;

[0034] A merged image feature is obtained according to the initial merged feature and the first image feature.

[0035] In one possible implementation, after obtaining the initial merged features based on the spatial dynamic weights and the spliced image features, the method further includes:

[0036] The initial merged features are used as the spliced image features, and the step of determining the spatial dynamic weight of the spliced image features is repeatedly performed until the number of repetitions meets the preset number, thereby obtaining the merged image features.

[0037] In a second aspect, an embodiment of the present application provides an image processing device, comprising:

[0038] a capture module configured to capture a moving object between a first image feature and a second image feature based on a cross-attention mechanism, and determine an intermediate feature between the first image feature and the second image feature, wherein the first image feature is an image feature in a first multi-channel image, and the second image feature is an image feature in a second multi-channel image, and the first multi-channel image and the second multi-channel image have different exposure levels;

[0039] a determination module, configured to determine, based on an intermediate feature between the first image feature and the second image feature, a position offset of a moving object between the first image feature and the second image feature, and a position offset feature of the position offset;

[0040] an alignment module, configured to perform a deformable alignment process on the second image feature to the first image feature based on a position offset feature of a moving object between the first image feature and the second image feature, to obtain an alignment feature;

[0041] a merging module, configured to obtain a merged image feature according to the alignment feature and the first image feature;

[0042] The determination module is used to determine the target image according to the merged image features.

[0043] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor;

[0044] Memory stores computer-executable instructions;

[0045] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0046] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementation methods of the first aspect.

[0047] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0048] An image processing method and device provided by the embodiments of the present application are based on a cross-attention mechanism. By determining the correlation between a first image feature in a first multi-channel image and a second image feature in a second multi-channel image with different exposure levels, the method focuses on and captures the features of a moving object, suppresses irrelevant information, and generates intermediate features, thereby determining the position offset and position offset features of the moving object. Based on the features, the second image features are then transformed by translation, rotation, and the like to align them with the first image features to obtain alignment features. The alignment features are then merged with the first image features, and the information of the two is combined to generate a target image. This effectively integrates the advantages of images with different exposures, highlights the details of the moving object, avoids ghosting and blurring, and improves the clarity, stability, and consistency of the image, thereby achieving the effect of improving image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0050] Figure 1 A schematic diagram of the flow of the existing image processing method provided in this application;

[0051] Figure 1a A schematic diagram of the structure of the spatial attention module of the existing image processing method provided by this application;

[0052] Figure 1b A schematic structural diagram of the alignment module of the existing image processing method provided in this application;

[0053] Figure 2 Schematic diagram of the image processing method provided in this application Figure 1 ;

[0054] Figure 3 Schematic diagram of the image processing method provided in this application Figure 2 ;

[0055] Figure 4 Schematic diagram of the image processing method provided in this application Figure 3 ;

[0056] Figure 5 A flowchart of a model training method provided in an embodiment of the present application;

[0057] Figure 5a A schematic diagram of the use of the high dynamic range imaging network model provided in an embodiment of the present application;

[0058] Figure 6 A schematic diagram of another image processing method provided in an embodiment of the present application;

[0059] Figure 7 A schematic diagram of the structure of the image processing device provided in this application;

[0060] Figure 8 This is a schematic diagram of the structure of the electronic device provided in this application.

[0061] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0062] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0063] First, let’s explain the terms involved in this application:

[0064] Exposure refers to the amount of light received by the image sensor or film during the photography and imaging process. Exposure is determined by three key parameters: shutter speed, aperture size, and sensitivity (ISO). Exposure determines the brightness level of a photo or image. Underexposure (underexposure) results in a dim photo or image, losing detail in shadows; overexposure (overexposure) results in a photo or image that is too bright, losing detail in highlights.

[0065] The cross-attention mechanism can refer to a technology used in deep learning to process multi-input sequences or feature dependencies. Its principle is to dynamically allocate attention weights by calculating the similarity between different inputs, breaking the traditional neural network's equal treatment of input information. By converting the input into query, key and value vectors, calculating the similarity between the query and key vectors and normalizing them with Softmax to obtain the attention weights, the value vectors are weighted and summed to achieve dynamic selection and focusing of input information.

[0066] A multi-channel image may refer to an image containing multiple information channels obtained by processing a low dynamic range (LDR) image.

[0067] Image features refer to specific attributes or pieces of information in an image that can describe and characterize the image content. These features can be visual elements such as color, texture, shape, edges, corners, or more complex statistical information or structural patterns extracted from image data.

[0068] A convolutional block refers to the basic building block of a convolutional neural network (CNN) in deep learning. It can be composed of one or more convolutional layers and other functional layers.

[0069] Residual Block refers to the core component of Residual Network (ResNet) in deep learning, which can solve problems such as gradient disappearance and degradation that occur as the depth of the neural network increases.

[0070] The patch embedding layer is a common component used in deep learning models to process data such as images. It divides the input data (e.g., an image) into multiple small patches (patches), similar to puzzle pieces. Each patch contains local information about the input data. These patches are then embedded into a low-dimensional feature space through linear projection, generating a corresponding feature vector for each patch.

[0071] Figure 1 A flow chart of the existing image processing method provided in this application is shown in FIG. Figure 1 As shown, the execution subject of the existing image processing method can be a server, and the server can be a mobile phone, computer, tablet and other devices. The existing image processing method may include:

[0072] Input three LDR images of the same scene with different exposures , the low dynamic range image with intermediate exposure is regarded as the reference image, and the rest of the images are regarded as non-reference images;

[0073] LDR image Perform gamma correction and convert to high dynamic range domain;

[0074] For the input LDR image Use the spatial attention module to extract features and correct the exposure of the image Use the alignment module to perform rough alignment of small movements;

[0075] For LDR images First, use a 3×3 convolution layer in each of the three branches to extract the image to the feature layer to obtain shallow features. , and then use the spatial attention module to extract features on the branch of the non-reference image to obtain the non-reference features and ;

[0076] Figure 1a The schematic diagram of the structure of the spatial attention module of the existing image processing method provided in this application is as follows: Figure 1a As shown in the figure, non-reference features and reference features are input, and the attention map is first estimated using splicing, two 3×3 convolutions and a Sigmoid function. Then, the input non-reference features and the attention map are multiplied pixel-wise to obtain the intermediate non-reference features.

[0077] For gamma-corrected images First, a 3×3 convolution is used in each of the three branches to extract the image to the feature layer to obtain shallow features. , and then use the alignment module on the branch of the non-reference image to extract the rough alignment features to obtain the non-reference features and ;

[0078] Figure 1b This is a schematic diagram of the structure of the alignment module of the existing image processing method provided by this application, such as Figure 1b As shown, non-reference features and reference features are input, and the offset is first estimated using a CNN-based offset estimator. The offset estimator contains a splicing, two 3×3 convolutions and a ReLu function, and then a 3×3 deformable convolution is used to perform geometric transformation on the input non-reference features, and the intermediate non-reference features are output;

[0079] The features that will be obtained Splicing is performed along the channel, and a 3×3 convolution is used for channel compression to obtain preliminary fusion features. ;

[0080] Furthermore, the initial fusion features Feature fusion is performed through three dense residual hole modules to obtain high-level fusion features;

[0081] Specifically, the dense residual dilated module contains four 3×3 dilated convolutions and a residual connection. The input features of each 3×3 dilated convolution include the output features of all previous dilated convolutions.

[0082] Finally, the obtained high-level fusion features are reduced through a 3×3 convolution channel to transform the features into the image domain, and then a sigmoid function is used to convert the values to between (0, 1) to output the final HDR image.

[0083] Considering the above scenario, the offset estimator in existing technologies, built on CNNs, has a localized receptive field, making it difficult to accurately perceive the motion of objects at long distances. This limitation is particularly pronounced when aligning LDR images, resulting in ghosting artifacts in the synthesized HDR image. Furthermore, poorly exposed areas are common in LDR images, and existing technologies fail to effectively suppress these areas during the fusion process, leading to fusion distortion and degradation in the performance of the synthesized HDR image.

[0084] The image processing method provided in this application effectively aligns moving objects between LDR images through a cross-attention deformable alignment method, and then uses a spatial attention fusion method to adaptively suppress the content of poorly exposed areas and select beneficial content to merge multi-exposure features, so that this patent effectively alleviates the problems of ghosting artifacts and fusion distortion, and improves the quality of HDR images.

[0085] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0086] Figure 2 Schematic diagram of the image processing method provided in this application Figure 1 ,like Figure 2 As shown, the method includes:

[0087] S201. Based on the cross-attention mechanism, a moving object between a first image feature and a second image feature is captured, and an intermediate feature between the first image feature and the second image feature is determined, where the first image feature is an image feature in a first multi-channel image, and the second image feature is an image feature in a second multi-channel image. The first multi-channel image and the second multi-channel image have different exposures.

[0088] The first multi-channel image and the second multi-channel image may refer to images with different exposure levels. For example, the first multi-channel image may refer to a picture with normal exposure, and the second multi-channel image may be multiple and may include overexposed images and underexposed images. For example, the first multi-channel image may refer to a picture with an intermediate exposure level, that is, in this embodiment of the application, the first multi-channel image may be a reference image, and the second multi-channel image may be multiple and may include overexposed images and underexposed images, that is, in this embodiment of the application, the second multi-channel image may be a non-reference image.

[0089] The first image feature may refer to an image feature of a first multi-channel image, and the second image feature may refer to an image feature of a second multi-channel image.

[0090] The moving object between the first image feature and the second image feature may refer to an object whose position in the image changes significantly in the image sequence, and whose own features such as edges, corners, and textures show differences in position and form in different images, and through feature extraction and comparison, the feature change of the object exceeds a certain threshold.

[0091] Based on the cross-attention mechanism, capturing the moving object between the first image feature and the second image feature can refer to taking the first image feature as the query and the second image feature as the key and value, or taking the second image feature as the query and the first image feature as the key and value. By calculating the correlation score between feature elements, the attention weight is dynamically allocated to highlight the features related to the moving object, thereby capturing the position changes, motion trajectory and state information of the moving object, fusing key information to enhance its feature representation, and providing high-quality input for subsequent image analysis tasks.

[0092] For example, when performing the cross attention mechanism, the first image feature , generate query Q, according to the second image feature , generate key K and value V; where:

[0093]

[0094]

[0095] in, is the corresponding weight, is the number of channel dimensions of the feature.

[0096] The intermediate feature between the first image feature and the second image feature can refer to the fusion result generated during the attention calculation process. By treating the first image feature as the query and the second image feature as the key and value, a correlation score is determined between them. Attention weights are dynamically assigned based on the correlation score, and a weighted sum is performed on the second image feature to generate the intermediate feature. This enhances the portion of the second image feature related to the moving object in the first image feature through the intermediate feature, while retaining some of the key information of the first image feature, making the characteristics of the moving object more prominent and clear.

[0097] In the embodiment of the present application, the method further includes:

[0098] Performing nonlinear transformation processing on the first low-dynamic image to obtain a first transformed image;

[0099] The first transformed image and the first low-dynamic image are spliced in a channel dimension to obtain a first multi-channel image.

[0100] The first low-motion image may refer to a low-motion image with normal exposure.

[0101] Performing a nonlinear transformation on the first low-dynamic range image to obtain the first transformed image may refer to a process of processing the first low-dynamic range image to obtain a high-dynamic range image. High-dynamic range image (HDR) may refer to a set of techniques in computer graphics and cinematography used to achieve a greater exposure dynamic range (i.e., greater differences in brightness and darkness) than conventional digital imaging techniques.

[0102] In the embodiment of the present application, the nonlinear transformation process may refer to a process of performing gamma correction on the first low-motion image.

[0103] Gamma Correction The formula can be:

[0104] ;

[0105] in, represents a parameter in the gamma transform, which can be set to 2.2 in the embodiment of the present application; Can refer to low-motion images; It can refer to the exposure time of the i-th LDR image.

[0106] Performing channel-dimensional splicing processing on the first transformed image and the first low dynamic range image to obtain a first multi-channel image may refer to splicing and combining the low dynamic range image and the high dynamic range domain image in the channel direction to obtain the first multi-channel image. For example, the RGB channels of an image can record the corresponding color component information, and the low dynamic range image and the high dynamic range domain image can be used as units containing specific information. When the low dynamic range image and the high dynamic range domain image are single-channel images, the first multi-channel image after stitching It can be a dual-channel image, storing data for low dynamic range and high dynamic range domain images respectively. When the low dynamic range and high dynamic range domain images are multi-channel images, the number of channels will increase accordingly. This can provide richer input for subsequent steps and facilitate image enhancement tasks.

[0107] In the embodiment of the present application, the method further includes:

[0108] Inputting the first multi-channel image into the convolution block for convolution processing to obtain a first initial feature;

[0109] The first initial feature is input into the residual block for residual processing to obtain the first image feature.

[0110] In the image processing process, the first multi-channel image can be first fed into a convolution block. Within the convolution block, a convolution operation is performed on the image through a convolution layer, possibly with a batch normalization layer and an activation function, to extract local features and generate first initial features. This first initial feature is then input into a residual block. The residual block directly transmits information using the identity mapping branch and simultaneously learns residual features through operations such as convolution transformations within the residual mapping branch, thereby obtaining the first image features.

[0111] S202: Determine, based on an intermediate feature between the first image feature and the second image feature, a position offset of the moving object between the first image feature and the second image feature, and a position offset feature of the position offset.

[0112] The position offset of the moving object between the first image feature and the second image feature can refer to a measurement value of the change in the spatial position of the moving object between the first image feature and the second image feature. The position offset can quantify the movement of the position of the moving object between the first image feature and the second image feature. The position offset can be determined by first calculating the difference between the first image feature and the second image feature vector, analyzing the numerical changes in each channel of the intermediate feature, parsing the position encoding information therein, determining its spatial distribution on the feature map, and then using the convolution module to capture local changes, relying on the attention mechanism to determine the weights of key areas, and finally combining global and local features with contextual information such as time series to determine the position offset.

[0113] The position offset feature of the position offset may refer to key information used to accurately describe the characteristics of the position offset, and the key information may include multi-dimensional quantitative indicators such as direction, distance, speed, and change trend.

[0114] After determining the position offset, vector analysis can be performed on the offset trajectory to clarify the directional characteristics of the position offset; the distance characteristics of the position offset can be calculated using the distance measurement model in combination with the spatial coordinate system; and the changing trend characteristics of the position offset can be discovered by performing curve fitting or trend analysis on multiple sets of continuous position offsets, thereby determining the position offset characteristics.

[0115] S203 : Perform deformable alignment processing on the second image feature to the first image feature according to the position offset feature of the moving object between the first image feature and the second image feature to obtain an alignment feature.

[0116] Deformably aligning the second image feature with the first image feature can refer to geometrically twisting and aligning the second image feature based on the positional offset characteristics of the moving object between the first and second image features, so that it more closely matches the first image feature in terms of spatial position and feature expression. This can also refer to the process of utilizing information such as direction and distance contained in the positional offset characteristics to adjust the position of the moving object in the second image feature to align with the position of the corresponding object in the first image feature through geometric transformations (such as translation, rotation, and scaling).

[0117] The alignment feature may refer to a feature result that achieves high consistency in spatial position and semantic expression after performing alignment processing on the second image feature based on the position offset feature of the moving object between the second image feature and the first image feature.

[0118] In the embodiment of the present application, after deformably aligning the second image feature with the first image feature based on the position offset feature of the moving object between the first image feature and the second image feature to obtain the alignment feature, the method further includes:

[0119] The alignment feature is used as the second image feature, and the steps of capturing the moving object between the first image feature and the second image feature and determining the intermediate feature between the first image feature and the second image feature are repeatedly executed based on the cross-attention mechanism until the number of repetitions meets the preset number of requirements, thereby obtaining the alignment feature that meets the number of repetitions.

[0120] In an embodiment of the present application, steps S201 to S203 can be performed by a cross-attention deformable alignment module, wherein, in order to ensure the effect of obtaining intermediate features, S201 to S203 can be repeatedly executed to obtain intermediate features that meet the accuracy requirements.

[0121] S204: Obtain a merged image feature according to the alignment feature and the first image feature.

[0122] Among them, merging features can refer to integrating the alignment features and the first image features into a new feature set containing richer information through a specific fusion method. Among them, the fusion method can be based on a weighted average method, assigning weights to each feature according to its importance to the overall information expression, and then performing a weighted summation of the corresponding elements, so that the merged features can retain the original details and unique information of the first image features, and can also incorporate the spatial and semantic matching relationship between the alignment features and the first image features; or a splicing strategy can be adopted to splice the two features in the channel dimension or spatial dimension to form a feature vector or feature matrix with higher dimensions and more comprehensive information, thereby providing a richer feature representation.

[0123] S205: Determine the target image according to the merged image features.

[0124] The target image may refer to an image obtained by reconstructing the merged image features through channel compression through a convolutional layer. In an embodiment of the present application, the target image may be an HDR image.

[0125] The image processing method provided in the embodiment of the present application uses a cross-attention deformable alignment module to effectively align low dynamic range images at the feature level, wherein the global information of non-reference features is learned and the position of moving objects is captured through a cross-attention-based offset estimator, and then based on the captured position, the non-reference features are geometrically transformed through deformable convolution, which effectively alleviates the problem of ghosting artifacts.

[0126] Figure 3 Schematic diagram of the image processing method provided in this application Figure 2 ,like Figure 3 As shown, this embodiment Figure 2Based on the embodiment, the steps of capturing a moving object between a first image feature and a second image feature based on a cross-attention mechanism and determining an intermediate feature between the first image feature and the second image feature are described in detail. The method includes:

[0127] S301, determining a first window feature of a first image feature and a second window feature of a second image feature;

[0128] S302: Based on the cross attention mechanism, the first moving object between the first window feature and the second window feature is captured to obtain an initial intermediate feature;

[0129] S303, performing window merging on the first window feature and the initial intermediate feature respectively to obtain a first window merged feature and a second window merged feature after the window merging;

[0130] S304: downsampling the first window merged feature and the second window merged feature to obtain a first adjusted feature and a second adjusted feature;

[0131] S305. Based on the cross-attention mechanism, capture the second moving object between the first adjustment feature and the second adjustment feature to obtain an intermediate feature, wherein the movement distance of the first moving object is smaller than the movement distance of the second moving object.

[0132] Among them, after obtaining the first image feature and the second image feature After that, the learnable,size and second image features are combined Consistent absolute position encoding added to the second image feature superior;

[0133] The first image feature and the second image feature with position coding are simultaneously divided into several windows of size P×P to obtain the first window feature of the first image feature and the second window feature of the second image feature ;

[0134] Inside the window, M cascaded cross-attention layers are used to capture the first moving object between the reference feature and the non-reference feature to obtain the initial intermediate feature ; Among them, the first moving object is a small-distance moving object. The small-distance moving object can refer to a moving object that moves a small distance in spatial position relative to the object corresponding to the second image feature within the image window. Here, "small" can be determined based on the pixel distance and the proportion of the image area. For example, in the pixel coordinate system of the image, the displacement of the moving object is measured in pixels. When the horizontal or vertical displacement of a moving object in two adjacent frames does not exceed 5 pixels, it can be considered as a moving object with a small moving distance; or the image can be divided into several areas, and the position change of the moving object in these areas is calculated. If the moving object moves from one area to an adjacent area, and the moving distance accounts for a small proportion of the entire image width or height, such as no more than 5%-10%, it can be considered as a moving object with a small moving distance.

[0135] Initial intermediate features and the first window feature Merge the windows, restore the size, and get the second window merge feature Merge features with the first window ;

[0136] Merge features of the second window Merge features with the first window , perform Patternbedding layer, perform downsampling, and obtain the second adjustment feature of low size and the first adjustment feature ;

[0137] On the entire feature map, N cascaded cross-attention layers are used to capture the second adjustment features and the first adjustment feature The second moving object between the two gets the intermediate feature ; Among them, the second moving object is a long-distance moving object. The long-distance moving object can refer to a moving object that moves a large distance in spatial position relative to the object corresponding to the second image feature within the image window. The "larger" here can be determined based on the pixel distance and the proportion of the image area. For example, in the pixel coordinate system of the image, the displacement of the moving object is measured in pixels. When the horizontal or vertical displacement of a moving object in two adjacent frames exceeds 50 pixels, it can be considered as a moving object with a large movement distance; or the image can be divided into several areas, and the position change of the moving object in these areas is calculated. If the moving object moves from one area to outside of multiple areas, and the moving distance accounts for a large proportion of the width or height of the entire image, such as more than 30%-50%, it can be determined as a moving object with a large movement distance.

[0138] After obtaining the intermediate features, you can Perform bilinear interpolation, upsample, restore the size, and then reduce the channel dimension through a layer of convolution to obtain features ; with characteristics As input, a convolution module is used to estimate the position offset of the moving object between the first image feature and the second image feature, and the position offset feature is output. ; With the second image feature and position offset features As input, a deformable convolution layer is used to geometrically transform the non-reference features to obtain aligned features .

[0139] Figure 4 Schematic diagram of the image processing method provided in this application Figure 3 ,like Figure 4 As shown, this embodiment Figure 2 Based on the embodiment, the step of obtaining the merged image feature according to the alignment feature and the first image feature is described in detail. The method includes:

[0140] S401 : performing splicing processing on the alignment feature and the first image feature to obtain a spliced image feature.

[0141] Among them, the splicing processing of the alignment feature and the first image feature is the process of integrating these two different sources of interrelated feature information. In an embodiment of the present application, the alignment feature is obtained by adjusting the second image feature through the position offset feature of the moving object between the first image feature and the second image feature. It is more consistent with the first image feature in terms of spatial position and semantic expression. The first image feature carries the initial information of the original image. During the splicing process, by connecting the channel dimensions of the alignment feature and the first image feature, the newly generated spliced image feature contains both the original details of the first image feature and the matching relationship with the second image feature reflected by the alignment feature, thereby forming a spliced image feature with richer information and higher dimension.

[0142] S402: Determine the spatial dynamic weight of the spliced image features.

[0143] The spatial dynamic weight of the spliced image features can be determined based on the spatial attention block, where the spatial dynamic weight satisfies:

[0144]

[0145] in, Represents a spatial attention block, which is a convolutional layer and a sigmoid function in series.

[0146] S403, obtaining an initial merged feature based on the spatial dynamic weight and the spliced image features;

[0147] S404: Obtain a merged image feature according to the initial merged feature and the first image feature.

[0148] Among them, the spatial dynamic weight Pixel-wise multiplication is performed with the stitched image features to suppress undesirable exposure content and select beneficial content (i.e. For positions where the pixel value is greater than the preset threshold, the corresponding pixel value is retained and emphasized). For positions that are less than or equal to a preset threshold, the corresponding pixel values in the stitched image features will be suppressed, thereby reducing the impact of the content in the positions representing poor exposure areas on subsequent processing. As a result, the stitched image features are spatially adjusted and optimized.

[0149] Finally, a layer of 1×1 convolution is used to fuse the features between channels to obtain the combined image features.

[0150] In the embodiment of the present application, after obtaining the initial merged features based on the spatial dynamic weights and the spliced image features, the method further includes:

[0151] The initial merged features are used as the spliced image features, and the step of determining the spatial dynamic weight of the spliced image features is repeatedly performed until the number of repetitions meets the preset number, thereby obtaining the merged image features.

[0152] Among them, steps S402-S404 can all be processed by the residual fusion module. In the embodiment of the present application, in order to ensure that the dynamic weights adaptively suppress the content of the poorly exposed area and select beneficial content to merge the aligned multi-exposure features, thereby effectively alleviating the effect of fusion distortion, steps S402-S404 can be repeatedly executed to improve the effect of the final merged image features.

[0153] The image processing method provided in the embodiment of the present application generates a spatial dynamic weight through a spatial attention module. The dynamic weight adaptively suppresses the content of poorly exposed areas and selects beneficial content to merge aligned multi-exposure features, effectively alleviating fusion distortion.

[0154] Figure 5 A flow chart of a model training method provided in an embodiment of the present application is shown as follows: Figure 5 As shown, the method includes:

[0155] S501: Input the pre-processed image to the model to be trained.

[0156] Among them, the preprocessed image can refer to a multi-channel image obtained through nonlinear transformation processing and channel dimension stitching processing. In an embodiment of the present application, the input multi-channel image can be divided into a normal exposure image, an over-exposure image and an under-exposure image.

[0157] The model to be trained may include a feature extraction subnetwork and a feature merging subnetwork, wherein the feature extraction subnetwork can be used to execute the subnetwork of steps S201 to S203 in the embodiment of the present application, and the feature merging subnetwork can be used to execute the subnetwork of step S204 in the embodiment of the present application.

[0158] That is, after the preprocessed image is input into the model to be trained, the predicted target image output by the model to be trained is obtained.

[0159] S502: Train the model to be trained according to the predicted target image and the label HDR image corresponding to the preprocessed image to obtain a high dynamic range imaging network model.

[0160] The predicted target image and the label HDR image corresponding to the preprocessed image can determine the loss function of the model to be trained, where the loss function satisfies:

[0161]

[0162] in, represents the predicted target image reconstructed by the model to be trained, is the label HDR image, represents a hyperparameter. Represents the Sobel operator, which detects the high-frequency area of the image through the change of the gray value of the image and extracts details. represents a tone mapping operation, where:

[0163]

[0164] in, It is a compression parameter and can be set to 5000.

[0165] After determining the loss function, the weight parameters of the model to be trained can be adjusted according to the loss function until the loss function of the model to be trained converges to obtain a high dynamic range imaging network model.

[0166] Figure 5a A schematic diagram of the use of the high dynamic range imaging network model provided in the embodiment of the present application is shown in FIG. Figure 5a As shown, the preprocessed image is input into the high dynamic range imaging network model, so that the HDR image output by the high dynamic range imaging network model can be obtained.

[0167] Figure 6A schematic diagram of another image processing method provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the executor of the image processing method is a high dynamic range imaging network model, which includes a feature extraction subnetwork and a feature merging subnetwork, wherein the feature extraction subnetwork includes a cross-attention deformable alignment module, and the feature merging subnetwork includes a residual fusion module, each of which includes two spatial attention fusion modules and a relu layer.

[0168] The image processing method may include:

[0169] 1. Determine three LDR images with different exposures ;

[0170] Optionally, the size of the three LDR images is C×H×W, where C represents the number of channels of the image, H is the height of the image, which can be expressed by the number of pixels of the image in the vertical dimension; and W is the width of the image, which can be expressed by the number of pixels of the image in the horizontal dimension.

[0171] Optionally, three LDR images at different exposures The size can be 3×256×256, which means each image has three channels: red (Red, R), green (Green, G), and blue (Blue, B), with a height of 256 pixels (pixel, px) and a width of 256px;

[0172] 2. Perform gamma correction on the LDR image and convert it to a high dynamic range domain , obtaining an image in the high dynamic range domain, wherein the size of the image in the high dynamic range domain is 3×256×256;

[0173] Among them, the parameters in gamma correction Can be set to 2.2.

[0174] 3. The obtained high dynamic range domain image and LDR images Splicing along the channel dimension to obtain a 6-channel model input , where the size is 6×256×256;

[0175] 4. Input to the high dynamic range imaging network model;

[0176] in, A 3×3 convolutional layer is first used to convert the image into features, and the size is 64×256×256;

[0177] Using a residual block, the extracted features are processed to eliminate the exposure differences caused by different exposures, and three shallow features are obtained. ,The size of the three shallow features are 64×256×256;

[0178] On the low-exposure feature branch, two cascaded cross-attention deformable alignment modules are used to take the low-exposure feature and the reference feature as input to obtain the aligned low-exposure feature. , size is 64×256×256;

[0179] On the high-exposure feature branch, two cascaded cross-attention deformable alignment modules are also used, with high-exposure features and reference features as input to obtain aligned high-exposure features. , size is 64×256×256; the corresponding cross-attention deformable alignment modules between these two branches share parameters;

[0180] Non-reference features to align and and reference features , spliced along the channel to form a merge feature , size is 128×256×256;

[0181] Optionally, a 3×3 convolution is used to perform channel compression on the initially merged features to form a more advanced fusion feature as the input of the feature merging sub-network. The feature is represented as , the size is 64×256×256;

[0182] In the feature merging subnetwork, three residual fusion modules are used to gradually merge the contents of multi-exposure images to form high dynamic range features. The output features of each residual fusion module are (i represents the i-th module) the size is 64×256×256;

[0183] Inside each residual fusion module, a spatial attention fusion module, a ReLu function, a spatial attention fusion module, and a residual connection are used to effectively fuse multi-exposure features. The output feature size of each spatial attention fusion module is 64×256×256;

[0184] Perform pixel-level addition on the output features of the last residual fusion module and the shallow reference features: ;

[0185] Optionally, the resulting features Through a layer of 3×3 convolution, an HDR image is obtained , with a size of 3×256×256.

[0186] Among them, for the cross-attention deformable alignment module:

[0187] Import any non-reference features and reference features , whose size is 64×256×256;

[0188] It is fed into the cross-attention offset estimator to estimate the position offset, i.e.:

[0189] Add learnable absolute position encoding features to non-reference features: , position encoding features Same size as non-reference features, 64×256×256;

[0190] The non-reference features and reference features with position encoding are divided into several 8×8 size windows to obtain the features of the new non-reference image. and the features of the reference image , their dimensions are (32×32)×64×(8×8);

[0191] Inside the window, M cascaded cross-attention layers are used to capture small moving objects between reference features and non-reference features to obtain the final output intermediate features. , the size is (32×32)×64×(8×8), M can be set to 2;

[0192] Non-reference intermediate features and the features of the reference image Merge windows, restore size, and obtain features and , the size is 64×256×256;

[0193] For the intermediate non-reference features and reference features The Patch embedding layer is used for downsampling. The Patch embedding layer consists of three depth-wise separable convolutional layers connected in series. The important parameters of these three depth-wise separable convolutional layers are:

[0194] First layer: input_channel=64, output_channel=128, kernel=3×3, stride=2;

[0195] Second layer: input_channel=128, output_channel=128, kernel=3×3, stride=2;

[0196] Third layer: input_channel=128, output_channel=128, kernel=3×3, stride=2;

[0197] After downsampling, low-size non-reference features are obtained and reference features , whose dimensions are 128×32×32;

[0198] On the entire feature map, N cascaded cross attention layers are used to capture long-distance moving objects between reference features and non-reference features, and the final output intermediate features are obtained. , the size is 128×32×32, and N can be set to 3;

[0199] For intermediate features Perform bilinear interpolation operation, up-sample, and restore the spatial size to 128×256×256, and then pass a layer of 3×3, convolution to reduce the channel dimension and obtain the feature , size is 64×256×256;

[0200] By characteristics As input, a CNN module is used to estimate the position offset of the moving object between the reference feature and the non-reference feature, and the position offset feature is output. , the size is 27×256×256;

[0201] Non-reference features and position offset features As input, a deformable convolution layer is used to perform geometric transformation on the non-reference features to obtain aligned non-reference features :

[0202]

[0203] in, is the number and index of convolution kernel weights, represents the weight, center position and kth offset of the kth kernel, is the weighted value of the kth kernel, Indicates The sampling offset of the kth kernel centered at and All from Here the convolution kernel size of the deformable convolution layer is 3×3. The aligned non-reference features of the output The size is 64×256×256.

[0204] Among them, for the cross attention layer:

[0205] Enter any reference token: and non-reference tokens: , their size can be (32×32)×128;

[0206] Using the reference image token: Generate query Q in cross-attention instead of reference token: Generate key K and value V and perform multi-head cross attention mechanism, where:

[0207]

[0208]

[0209] in, is the corresponding weight, is the number of channel dimensions of the feature, which can be 128. The size is (32×32)×128;

[0210] The characteristics of the above output Execute the FFN layer and output the intermediate features with a size of (32×32)×128. The FFN layer contains two MLP layers.

[0211] For the spatial attention fusion module:

[0212] Input any fusion feature , the size can be 64×256×256;

[0213] Use a spatial attention block to obtain spatial dynamic weights :

[0214] ;

[0215] in, Represents the spatial attention block, which is a 3×3 convolutional layer and a sigmoid function in series. The size is 64×256×256;

[0216] The spatial dynamic weight and fusion features Perform pixel-level multiplication to suppress content with poor exposure and select beneficial content;

[0217] Use a layer of 1×1 convolution to fuse features on the channel to obtain output features , the size is 64×256×256.

[0218] Figure 7 This is a schematic diagram of the structure of the image processing device provided by this application, such as Figure 7 As shown, the image processing device 70 provided in this embodiment includes:

[0219] a capture module 701 configured to capture a moving object between a first image feature and a second image feature based on a cross-attention mechanism, and determine an intermediate feature between the first image feature and the second image feature, wherein the first image feature is an image feature in a first multi-channel image, and the second image feature is an image feature in a second multi-channel image, and the first multi-channel image and the second multi-channel image have different exposure levels;

[0220] a determination module 702 for determining, based on an intermediate feature between the first image feature and the second image feature, a position offset of a moving object between the first image feature and the second image feature, and a position offset feature of the position offset;

[0221] An alignment module 703 is configured to perform a deformable alignment process on the second image feature to the first image feature based on a position offset feature of a moving object between the first image feature and the second image feature to obtain an alignment feature;

[0222] A merging module 704 is configured to obtain a merged image feature based on the alignment feature and the first image feature;

[0223] The determination module 705 is configured to determine a target image based on the merged image features.

[0224] In a possible implementation, the capture module 701 may also be specifically configured to:

[0225] determining a first window characteristic of the first image characteristic and a second window characteristic of the second image characteristic;

[0226] Based on the cross attention mechanism, the first moving object between the first window feature and the second window feature is captured to obtain the initial intermediate feature;

[0227] Obtaining a first adjustment feature and a second adjustment feature according to the first window feature and the initial intermediate feature;

[0228] Based on the cross attention mechanism, the second moving object between the first adjustment feature and the second adjustment feature is captured to obtain an intermediate feature, wherein the movement distance of the first moving object is smaller than the movement distance of the second moving object.

[0229] In a possible implementation, the capture module 701 may also be specifically configured to:

[0230] Perform window merging on the first window feature and the initial intermediate feature respectively to obtain the first window merged feature and the second window merged feature after the window merging;

[0231] Downsampling is performed on the first window merged feature and the second window merged feature to obtain a first adjusted feature and a second adjusted feature.

[0232] In a possible implementation, the capture module 701 may also be specifically configured to:

[0233] Performing nonlinear transformation processing on the first low-dynamic image to obtain a first transformed image;

[0234] The first transformed image and the first low-dynamic image are spliced in a channel dimension to obtain a first multi-channel image.

[0235] In a possible implementation, the capture module 701 may also be specifically configured to:

[0236] Inputting the first multi-channel image into the convolution block for convolution processing to obtain a first initial feature;

[0237] The first initial feature is input into the residual block for residual processing to obtain the first image feature.

[0238] In a possible implementation, the alignment module 703 may also be specifically configured to:

[0239] The alignment feature is used as the second image feature, and the steps of capturing the moving object between the first image feature and the second image feature and determining the intermediate feature between the first image feature and the second image feature are repeatedly performed based on the cross-attention mechanism until the number of repetitions meets the preset number of requirements, thereby obtaining the alignment feature that meets the number of repetitions.

[0240] In a possible implementation, the merging module 704 may also be specifically configured to:

[0241] performing splicing processing on the alignment feature and the first image feature to obtain a spliced image feature;

[0242] The spliced image feature and the first image feature are subjected to feature merging processing to obtain a merged image feature.

[0243] In a possible implementation, the merging module 704 may also be specifically configured to:

[0244] Determining spatial dynamic weights of stitched image features;

[0245] According to the spatial dynamic weight and the spliced image features, the initial merged features are obtained;

[0246] A merged image feature is obtained according to the initial merged feature and the first image feature.

[0247] In a possible implementation, the merging module 704 may also be specifically configured to:

[0248] The initial merged features are used as the spliced image features, and the step of determining the spatial dynamic weight of the spliced image features is repeatedly performed until the number of repetitions meets the preset number, thereby obtaining the merged image features.

[0249] The image processing device 70 provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.

[0250] Figure 8 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 8 As shown, the electronic device 80 provided in this embodiment includes: at least one processor 801 and a memory 802. Optionally, the device 80 also includes a communication component 803. The processor 801, the memory 802 and the communication component 803 are connected via a bus 804.

[0251] During the specific implementation process, at least one processor 801 executes the computer-executable instructions stored in the memory 802, so that the at least one processor 801 performs the above method.

[0252] The specific implementation process of the processor 801 can be found in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here in this embodiment.

[0253] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASICs), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly executed by a hardware processor or by a combination of hardware and software modules within the processor.

[0254] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage.

[0255] A bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be categorized as address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0256] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0257] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0258] The readable storage medium may be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0259] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0260] The division of units is merely a logical functional division; actual implementations may employ alternative divisions, such as combining or integrating multiple units or components into another system, or omitting or disabling certain features. Furthermore, any direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units, either through an interface, electrical, mechanical, or other means.

[0261] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0262] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0263] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0264] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0265] Finally, it should be noted that those skilled in the art will readily identify other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art not disclosed herein. The present invention is not limited to the precise structure described above and illustrated in the accompanying drawings, and various modifications and variations may be made without departing from the scope thereof. The scope of the present invention is limited solely by the appended claims.

Claims

1. An image processing method, characterized in that: include: Based on the cross attention mechanism, the moving object between the first image feature and the second image feature is captured, and the intermediate feature between the first image feature and the second image feature is determined. The first image feature is an image feature in a first multi-channel image, the second image feature is an image feature in a second multi-channel image, and the first multi-channel image and the second multi-channel image have different exposure levels; determining, based on an intermediate feature between the first image feature and the second image feature, a position offset of the moving object between the first image feature and the second image feature, and a position offset feature of the position offset; performing a deformable alignment process on the second image feature to the first image feature according to a position offset feature of the moving object between the first image feature and the second image feature to obtain an alignment feature; Obtaining a merged image feature according to the alignment feature and the first image feature; determining a target image according to the combined image features; The method of capturing a moving object between a first image feature and a second image feature based on a cross-attention mechanism and determining an intermediate feature between the first image feature and the second image feature includes: Determine a first window feature of a first image feature and a second window feature of a second image feature; based on a cross-attention mechanism, capture a first moving object between the first window feature and the second window feature to obtain an initial intermediate feature; perform window merging on the first window feature and the initial intermediate feature to obtain a first window merged feature and a second window merged feature after the window merging; perform downsampling on the first window merged feature and the second window merged feature to obtain a first adjustment feature and a second adjustment feature; based on the cross-attention mechanism, capture a second moving object between the first adjustment feature and the second adjustment feature to obtain an intermediate feature, wherein the movement distance of the first moving object is less than the movement distance of the second moving object.

2. The method according to claim 1, characterized in that The method further comprises: Performing nonlinear transformation processing on the first low-dynamic image to obtain a first transformed image; Channel dimension stitching processing is performed on the first transformed image and the first low-dynamic image to obtain the first multi-channel image.

3. The method according to claim 1, characterized in that The method further comprises: Inputting the first multi-channel image into the convolution block for convolution processing to obtain a first initial feature; The first initial feature is input into the residual block for residual processing to obtain a first image feature.

4. The method according to claim 1, wherein After performing deformable alignment processing on the second image feature to the first image feature based on the position offset feature of the moving object between the first image feature and the second image feature to obtain the alignment feature, the method further includes: The alignment feature is used as the second image feature, and the steps of capturing the moving object between the first image feature and the second image feature based on the cross-attention mechanism and determining the intermediate feature between the first image feature and the second image feature are repeatedly performed until the number of repetitions meets the preset number of requirements, thereby obtaining the alignment feature that meets the number of repetitions.

5. The method according to claim 1, wherein The obtaining of a merged image feature according to the alignment feature and the first image feature includes: performing a splicing process on the alignment feature and the first image feature to obtain a spliced image feature; A feature merging process is performed on the spliced image feature and the first image feature to obtain a merged image feature.

6. The method according to claim 5, characterized in that The performing feature merging processing on the spliced image feature and the first image feature to obtain a merged image feature includes: Determining spatial dynamic weights of the stitched image features; Obtaining an initial merged feature according to the spatial dynamic weight and the spliced image feature; A merged image feature is obtained according to the initial merged feature and the first image feature.

7. The method according to claim 6, characterized in that After obtaining the initial merged features according to the spatial dynamic weights and the spliced image features, the method further includes: The initial merged feature is used as the spliced image feature, and the step of determining the spatial dynamic weight of the spliced image feature is repeatedly performed until the number of repetitions meets a preset number, thereby obtaining the merged image feature.

8. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

Images