Image processing method and device, electronic equipment, chip and storage medium

Abstract

The invention provides an image processing method and device, electronic equipment, a chip and a storage medium, and relates to the field of image processing.The method comprises the steps that a reference image is determined from multiple frames of shot images based on quality scores of the multiple frames of shot images; wherein the multiple frames of shot images are obtained by shooting the same shot object by adopting different shooting view angles; performing shadow correction on the multiple frames of shot images based on shadow areas in the multiple frames of shot images to obtain multiple frames of corrected images; based on the homography matrix of the multiple frames of shot images, aligning the multiple frames of corrected images to the reference image to obtain multiple frames of aligned images; wherein the homography matrix is used for indicating a perspective transformation relation of distortion of the shot image to the first reference image; and performing image synthesis according to the multiple frames of aligned images to obtain a target image. Therefore, the problem of shadow shielding caused by illumination can be eliminated, and the imaging effect is improved. In addition, assistance of depth-of-field information and multi-exposure is not needed, and the complexity of image processing can be reduced.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device, electronic device, chip and storage medium. Background Art

[0002] Multi-frame image synthesis and enhancement technology is widely used in various fields, such as security monitoring, medical imaging, remote sensing imaging, etc. By utilizing the complementary information between multi-frame images, the resolution, clarity and color performance of the image can be effectively improved, thereby improving the overall quality of the image. Summary of the invention

[0003] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0004] To this end, the present application proposes an image processing method, device, electronic device, chip and storage medium to eliminate the occlusion and shadow problem caused by illumination and improve the imaging effect. In addition, without the need for depth of field information and multiple exposures, the complexity of image processing can be reduced.

[0005] In one aspect, an embodiment of the present application provides an image processing method, including:

[0006] Determining a first reference image from the multiple frames of captured images based on quality scores of the multiple frames of captured images; wherein the multiple frames of captured images are obtained by photographing the same subject using different shooting angles;

[0007] Based on the shadow areas in the multiple frames of captured images, performing shadow correction on the multiple frames of captured images to obtain multiple frames of corrected images;

[0008] Based on the homography matrix of the multiple frames of captured images, align the multiple frames of corrected images to the first reference image to obtain multiple frames of aligned images; wherein the homography matrix is ​​used to indicate the perspective transformation relationship of the captured images being warped to the first reference image;

[0009] Image synthesis is performed according to the multiple aligned image frames to obtain a target image.

[0010] Another aspect of the present application provides an image processing device, including:

[0011] A determination module, configured to determine a first reference image from the plurality of frames of captured images based on quality scores of the plurality of frames of captured images; wherein the plurality of frames of captured images are obtained by photographing the same subject at different shooting angles;

[0012] A correction module, configured to perform shadow correction on the multiple frames of captured images based on shadow areas in the multiple frames of captured images to obtain multiple frames of corrected images;

[0013] An alignment module, configured to align the multiple frames of corrected images to the first reference image based on a homography matrix of the multiple frames of captured images to obtain multiple frames of aligned images; wherein the homography matrix is ​​used to indicate a perspective transformation relationship of the captured images being warped to the first reference image;

[0014] A synthesis module is used to perform image synthesis based on the multiple frames of aligned images to obtain a target image.

[0015] Another aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the image processing method as described in the aforementioned aspect is implemented.

[0016] Another aspect of the present application provides a chip, which includes an interface circuit and a processing circuit coupled to each other, wherein the interface circuit is used to input or output a signal, and the processing circuit is configured to execute the image processing method as described in the aforementioned aspect.

[0017] Another aspect of the present application is a non-temporary computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the image processing method as described in the aforementioned aspect is implemented.

[0018] Another aspect of the present application provides a computer program product on which a computer program is stored. When the program is executed by a processor, the image processing method as described in the above aspect is implemented.

[0019] The image processing method, device, electronic device, chip and storage medium proposed in this application can take photos of the same subject from different angles during the shooting process, obtain multiple frames of images with different shadow areas, and perform shadow correction, alignment and synthesis on the multiple frames of images, which can eliminate the problem of occlusion and shadow caused by illumination and have better imaging effects. In addition, without the need for depth of field information and multiple exposures, the complexity of image processing can be reduced, the imaging effect can be improved, and the user experience in shooting scenes can be improved.

[0020] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0022] Figure 1A schematic diagram of a flow chart of a first image processing method provided in an embodiment of the present application;

[0023] Figure 2 A schematic diagram of a second image processing method provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of a flow chart of a third image processing method provided in an embodiment of the present application;

[0025] Figure 4 A schematic diagram of a fourth image processing method provided in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of the implementation principle of any embodiment of the present application;

[0027] Figure 6 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0028] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0029] Figure 8 It is a schematic diagram of the structure of a chip proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0030] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0031] Multi-frame image synthesis and enhancement technology mainly relies on complementary information between images. When users continuously take multiple images of the same scene, due to factors such as the shaking of the shooting equipment and the shooting angle, each image will have slight differences in details, brightness, color, etc. These differences are complementary information. By fusing the complementary information between images, a clearer and more colorful image can be obtained. However, multi-frame image synthesis and enhancement technology also faces some challenges, such as how to eliminate motion blur between different image frames, how to deal with lighting changes, and how to avoid color distortion. In order to solve these problems, researchers have proposed a series of algorithms and techniques. Among them, the method based on alignment and fusion is the most common one. This method first needs to align the input image sequence to eliminate motion blur between frames, and then fuse the complementary information between different frames through a fusion algorithm to obtain the final enhanced image.

[0032] In related technologies, the following solutions are mainly used to achieve multi-frame image synthesis and enhancement:

[0033] The first method is to obtain a composite image by synthesizing multiple images with different exposure parameters, and blur the content of the composite image according to the depth of field information of the composite image to retain the image details in the composite image and improve the dynamic range of the imaging.

[0034] However, the above method requires additional depth of field information of the synthesized image, which not only increases the complexity of image processing, but also leads to inaccurate calculation of depth of field information if the shooting conditions are limited or the image quality is poor, resulting in poor quality of image synthesis and reducing the user's shooting experience.

[0035] The second method is to use multiple images with different exposure parameters for image fusion to improve the dynamic range of imaging.

[0036] However, the above method requires the assistance of a flash and has poor flexibility.

[0037] In response to at least one of the above-mentioned problems, the present application proposes an image processing method, device, electronic device, chip and storage medium.

[0038] The image processing method, device, electronic device, chip and storage medium of the embodiments of the present application are described below with reference to the accompanying drawings.

[0039] Figure 1 A schematic flowchart of the first image processing method provided in an embodiment of the present application.

[0040] The embodiment of the present application takes the image processing method being configured in an image processing device as an example. The image processing device can be applied to any electronic device or vehicle with computing capabilities so that the electronic device or vehicle can perform image processing functions.

[0041] The electronic device may be a personal computer, a mobile terminal, a television, etc. The mobile terminal may be, for example, a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, or other hardware device with various operating systems.

[0042] In any embodiment of the present application, the image enhancement method can be executed by a chip, which can be integrated into electronic devices such as mobile terminals or AI (Artificial Intelligence) servers. Among them, the chip includes image signal processing (Image Signal Processing, referred to as ISP), central processing unit (Central Processing Unit, referred to as CPU), application-specific integrated circuit (Application-Specific Integrated Circuit, referred to as ASIC), microprocessor (Digital Signal Processor, referred to as DSP), field programmable gate array (Field-Programmable Gate Array, referred to as FPGA), system on chip (System On AChip, referred to as SOC), reduced instruction set computer (Reduced Instruction Set Computer, referred to as RISC), etc., which are not listed one by one this time.

[0043] like Figure 1 As shown, the image processing method may include the following steps S101 to S104:

[0044] Step S101, determining a first reference image from the multiple frames of captured images based on quality scores of the multiple frames of captured images; wherein the multiple frames of captured images are obtained by photographing the same subject using different shooting angles.

[0045] The photographed subjects include but are not limited to objects, people, animals (such as pets), plants, etc.

[0046] The multiple frames of captured images are obtained by using an electronic device or a shooting device to shoot the same object at different shooting angles (or shooting angles). For example, the number of frames of the marked captured images is N, and the multiple frames of captured images can be marked as I1, I2, ..., I N , where N is a positive integer greater than 1.

[0047] In the embodiment of the present application, image quality evaluation can be performed on multiple frames of captured images to obtain quality scores of the multiple frames of captured images, and a first reference image can be determined from the multiple frames of captured images according to the quality scores of the multiple frames of captured images. For example, the captured image with the highest quality score can be used as the first reference image. In this case, the first reference image can also be called the image with the best quality, for example, marked as I best .

[0048] Step S102 : performing shadow correction on the multiple frames of captured images based on shadow areas in the multiple frames of captured images to obtain multiple frames of corrected images.

[0049] In the embodiment of the present application, for any one frame of captured images I j (j=1,2,…,N), the captured image I can be segmented based on the image segmentation algorithm. j Perform instance segmentation to obtain the captured image I j The shaded area S in j , then, based on the shadow area S j For this captured image I j Perform shadow correction to obtain a corrected captured image, which is referred to as the corrected image C in this application. j .

[0050] Exemplarily, in the present application, shadows may include shadows caused by light blocking, for example, because the shooting terminal itself blocks the light, resulting in the presence of shadows of the terminal and the shooting subject (such as an arm) in the output image.

[0051] Step S103, based on the homography matrix of the multiple frames of captured images, align the multiple frames of corrected images to the first reference image to obtain multiple frames of aligned images; wherein the homography matrix is ​​used to indicate the perspective transformation relationship of the captured images warped to the first reference image.

[0052] In the embodiment of the present application, for any frame of captured image I j First, we can calculate the captured image I j Warp to the homography matrix H of the first reference image j ; Among them, the homography matrix H j Used to indicate the image capture I j Warp to the perspective transformation of the first reference image.

[0053] Afterwards, based on the captured image I j The homography matrix H j , the corrected image C j Align to the first reference image to obtain the aligned image C' j , that is, it can be based on the homography matrix H j , the corrected image C j Mapped to the image coordinate system where the first reference image is located, the aligned image C' is obtained j .

[0054] Step S104: performing image synthesis based on multiple aligned image frames to obtain a target image.

[0055] In the embodiment of the present application, image synthesis can be performed based on multiple frames of aligned images to obtain a target image.

[0056] The image processing method of the embodiment of the present application takes photos of the same subject from different angles during the shooting process, obtains multiple frames of images with different shadow areas, and performs shadow correction, alignment and synthesis on the multiple frames of images, which can eliminate the problem of occlusion and shadow caused by illumination and has a better imaging effect. In addition, without the need for depth of field information and multiple exposures, the complexity of image processing can be reduced, the imaging effect can be improved, and the user experience in the shooting scene can be improved.

[0057] The present application embodiment provides another image processing method. Figure 2 A schematic flowchart of the second image processing method provided in an embodiment of the present application.

[0058] It should be noted that the image processing method can be executed alone, or it can be executed in combination with any embodiment of the present application or a possible implementation method in the embodiment, or it can be executed in combination with any technical solution in the related technology, and the embodiments of the present application are not limited to this.

[0059] like Figure 2 As shown, the image processing method may include the following steps S201 to S206:

[0060] Step S201 : determining a first reference image from a plurality of frames of captured images based on quality scores of the plurality of frames of captured images.

[0061] The multiple frames of captured images are obtained by shooting the same object from different shooting angles.

[0062] Step S202 : performing shadow correction on the multiple frames of captured images based on the shadow areas in the multiple frames of captured images to obtain multiple frames of corrected images.

[0063] Step S203: aligning the multiple frames of corrected images to the first reference image based on the homography matrix of the multiple frames of captured images to obtain multiple frames of aligned images.

[0064] The homography matrix is ​​used to indicate the perspective transformation relationship of the captured image to the first reference image.

[0065] The explanation of steps S201 to S203 can be found in the relevant description in any embodiment of the present application and will not be repeated here.

[0066] Step S204, determining a second reference image from the multiple aligned image frames; wherein the second reference image is obtained by processing the first reference image.

[0067] For example, the first reference image is the captured image with the highest quality score, that is, the image with the best quality I best For example, the second reference image may be: Cb ' est .

[0068] Step S205 , based on the second reference image, locally register the multiple frames of aligned images to obtain multiple frames of registered images.

[0069] In an embodiment of the present application, based on the second reference image, local registration (ie, pixel-level alignment) can be performed on multiple frames of aligned images to obtain an aligned image after multiple frames of registration (referred to as a registered image in the present application).

[0070] In any embodiment of the present application, multiple frames of registered images can be obtained by registering using the following steps A to B:

[0071] Step A: Based on the optical flow algorithm in the related art (such as Lucas-Kanade algorithm, Horn-Schunck method, Farneback method, etc.), calculate the optical flow vector field between the multiple aligned frames and the second reference image; wherein any aligned frame C' j With the second reference image (such as C' best ) is used to indicate the optical flow vector field between the aligned images C' j The moving direction and moving speed of the pixel point in the reference image relative to the corresponding pixel point in the second reference image.

[0072] Step B: Based on the optical flow vector field of the multi-frame aligned images, the multi-frame aligned images are locally registered (ie, pixel-level aligned) to obtain multi-frame registered images.

[0073] That is, for any frame alignment image C' j , we can align the image C' based on j The optical flow vector field of the aligned image C' j Perform pixel-level alignment to obtain a frame of registered image.

[0074] Step S206: fuse multiple frames of registered images to obtain a target image.

[0075] In the embodiment of the present application, multiple frames of registered images can be fused based on image fusion technology to obtain a target image.

[0076] Exemplarily, multiple frames of target images can be input into a UNet (a convolutional neural network) model for image fusion to obtain a target image output by the UNet model.

[0077] The image processing method of the embodiment of the present application performs local registration on multiple frames of aligned images based on a second reference image of better quality, and then fuses the multiple frames of registered images to obtain a target image, which can further improve the imaging effect of the final synthesized target image, that is, improve the quality of the target image.

[0078] The present application embodiment provides another image processing method. Figure 3 A schematic flowchart of the third image processing method provided in an embodiment of the present application.

[0079] It should be noted that the image processing method can be executed alone, or it can be executed in combination with any embodiment of the present application or a possible implementation method in the embodiment, or it can be executed in combination with any technical solution in the related technology, and the embodiments of the present application are not limited to this.

[0080] like Figure 3 As shown, the image processing method may include the following steps S301 to S307:

[0081] Step S301 : determining a first reference image from a plurality of frames of captured images based on quality scores of the plurality of frames of captured images.

[0082] The multiple frames of captured images are obtained by shooting the same object from different shooting angles.

[0083] For the explanation of step S301, please refer to the relevant description in any embodiment of the present application, and will not be repeated here.

[0084] Step S302: for any captured image, matching the first feature point of any captured image with the second feature point of the first reference image to obtain a plurality of matching feature point pairs.

[0085] As an example, the SIFT (Scale-Invariant Feature Transform) algorithm can be used to extract feature points of each frame of the captured image. j (j=1,2,…,N), the captured image I j The feature point in the first reference image (referred to as the first feature point in this application) is best ) are matched with the feature points in the image (referred to as the second feature points in this application) to obtain multiple matching feature point pairs, wherein each matching feature point pair includes a first feature point and a second feature point that match each other.

[0086] Step S303, performing multiple rounds of iteration processes according to multiple matching feature point pairs; wherein any round of iteration process includes: randomly selecting matching feature point pairs required for this round of iteration process from multiple matching feature point pairs, and calculating the homography matrix of this round of iteration process according to the matching feature point pairs required for this round of iteration process.

[0087] The total number of execution rounds of the iterative process may be preset, such as M rounds.

[0088] In an embodiment of the present application, multiple rounds of iterations can be performed based on multiple matching feature point pairs, wherein any round of iterations includes: randomly selecting matching feature point pairs required for this round of iterations from multiple matching feature point pairs (wherein each round of iterations requires at least 4 matching feature point pairs), and calculating the homography matrix of this round of iterations based on the matching feature point pairs required for this round of iterations. Exemplarily, a RANSAC (Random Sample Consensus) algorithm can be used to calculate the homography matrix of this round of iterations based on the matching feature point pairs required for this round of iterations.

[0089] Step S304 : Selecting a homography matrix of warping any captured image to the first reference image from the homography matrices calculated in multiple rounds of iterations.

[0090] In the embodiment of the present application, the captured image I can be selected from the homography matrix calculated by multiple rounds of iterations. j Warp to the homography matrix H of the first reference image j Among them, the homography matrix H j Used to indicate the image capture I j Warp to the perspective transformation of the first reference image.

[0091] In any one of the embodiments of the present application, any frame of captured image I j Warp to the homography matrix H of the first reference image j , can be selected by following steps C to E:

[0092] Step C: For any round of iteration process, the homography matrix of any round of iteration process is used to project multiple matching feature point pairs, and the projection errors of the multiple matching feature point pairs under the homography matrix of any round of iteration process are calculated.

[0093] As an example, for any matching feature point pair (p i ,q i ), where p i It is the captured image I j The first characteristic point in iis the corresponding second feature point in the first reference image, then (p i ,q i )The projection error under the homography matrix H is:

[0094] d i =|Hp i -q i |;

[0095] Step D: Based on the projection errors of the multiple matching feature point pairs, determine the inliers in any round of iteration from the multiple matching feature point pairs.

[0096] As an example, the error threshold associated with the inliers can be queried. When the projection error of any matching feature point pair is less than or equal to the error threshold, the matching feature point pair can be used as an inlier in any of the above-mentioned iterative processes. Otherwise, that is, when the projection error of any matching feature point pair is greater than the error threshold, the matching feature point pair can be used as an outlier in any of the above-mentioned iterative processes.

[0097] Step E: Take the homography matrix of the iteration process with the largest number of inliers as the captured image I j Warp to the homography matrix H of the first reference image j .

[0098] In any one of the embodiments of the present application, any frame of captured image I j Warp to the homography matrix H of the first reference image j , can be selected by following steps C' to E':

[0099] Step C': for any round of iteration, the homography matrix of any round of iteration is used to project multiple matching feature point pairs, and the projection errors of multiple matching feature point pairs are calculated. The implementation principle is consistent with step C and will not be described here.

[0100] Step D': Determine the projection error of any round of iteration process based on the projection errors of multiple matching feature point pairs.

[0101] As an example, the accumulated value, average value, weighted sum value, etc. of the projection errors of multiple matching feature point pairs can be used as the projection error of any round of iteration process.

[0102] Step E': Take the homography matrix of the iterative process with the smallest projection error as the captured image I j Warp to the homography matrix H of the first reference image j .

[0103] Therefore, different methods can be used to select the homography matrix for warping each frame of the captured image to the first reference image, which can improve the flexibility and applicability of the method.

[0104] Step S305 , based on the shadow areas in the multiple frames of captured images, shadow correction is performed on the multiple frames of captured images to obtain multiple frames of corrected images.

[0105] Step S306: align the multiple frames of corrected images to the first reference image based on the homography matrix of the multiple frames of captured images to obtain multiple frames of aligned images.

[0106] Step S307: performing image synthesis based on multiple aligned image frames to obtain a target image.

[0107] The explanation of steps S305 to S307 can be found in the relevant description in any embodiment of the present application and will not be repeated here.

[0108] The image processing method of the embodiment of the present application obtains the homography matrix corresponding to each frame of the captured image by executing multiple rounds of iteration processes, which can improve the accuracy and rationality of the calculation of the homography matrix of each frame of the captured image.

[0109] The present application embodiment provides another image processing method. Figure 4 A schematic flowchart of the fourth image processing method provided in an embodiment of the present application.

[0110] It should be noted that the image processing method can be executed alone, or it can be executed in combination with any embodiment of the present application or a possible implementation method in the embodiment, or it can be executed in combination with any technical solution in the related technology, and the embodiments of the present application are not limited to this.

[0111] like Figure 4 As shown, the image processing method may include the following steps S401 to S406:

[0112] Step S401 : determining a first reference image from the multiple frames of captured images based on the quality scores of the multiple frames of captured images.

[0113] The multiple frames of captured images are obtained by shooting the same object from different shooting angles.

[0114] For the explanation of step S401, please refer to the relevant description in any embodiment of the present application, which will not be repeated here.

[0115] In any one of the embodiments of the present application, any frame of captured image I j The quality score of (j=1,2,…,N) can be calculated by following steps X to Z:

[0116] Step X: Extract captured image I j The multi-scale features are used to characterize the captured image I j The basic properties and statistical characteristics of images, such as contrast, edge direction distribution, etc.

[0117] Step Y: Calculate the captured image I j The degree of deviation between the multi-scale features and the statistical model; wherein the statistical model is used to indicate the distribution of multi-scale features of multiple sample images (ie, normal natural images), that is, the statistical features that characterize normal natural images.

[0118] As an example, the captured image I can be calculated j The Kullback-Leibler (KL) divergence between the probability density function of the multi-scale features of the statistical model is used as the KL divergence of the captured image I j The degree of deviation between the multi-scale characteristics of the

[0119] Step Z: Based on the degree of deviation, determine the captured image I j Quality score.

[0120] The smaller the deviation, the better the image I j The closer the multi-scale features of the image are to those of the natural image, the j The better the image quality is; on the contrary, the greater the deviation is, the j The image quality is worse.

[0121] That is, the captured image I j The quality score is negatively correlated with the above-mentioned deviation degree, that is, the smaller the deviation degree, the higher the quality score, and conversely, the greater the deviation degree, the lower the quality score.

[0122] In any one of the embodiments of the present application, any frame of captured image I j The quality score of (j=1, 2, ..., N) can also be calculated by following steps X' to Z':

[0123] Step X': Extract captured image I j The local contrast characteristics and edge clarity characteristics of

[0124] As an example, the captured image I can be calculated j The local variance of multiple image blocks in the image is used to extract local contrast features. j The edge definition features are extracted by using the contrast between edge pixels and background pixels in the image.

[0125] Step Y': fuse and normalize the local contrast features and edge clarity features to obtain the target features.

[0126] As an example, the local contrast feature and the edge clarity feature may be fused to obtain a fused feature, and the fused feature may be normalized to remove the mean and standardize the variance to obtain the target feature.

[0127] Step Z': Perform quality prediction on the target features to obtain the quality score of any captured image.

[0128] As an example, the target feature can be input into a pre-trained regression model for quality prediction, and the regression model outputs a predicted quality score to reflect the quality of the captured image I j quality level.

[0129] In any one of the embodiments of the present application, any frame of captured image I j The quality score of (j=1,2,…,N) can also be calculated by following steps X" to Z":

[0130] Step X”: Calculate the captured image I j Index values ​​under various image quality indicators.

[0131] The image quality indicators include, but are not limited to: Entropy (used to calculate the entropy of an image, a higher entropy value indicates that the image contains more information), Laplacian Variance (used to calculate the Laplace variation of an image, a larger Laplace variance value generally indicates that the image is clearer), etc.

[0132] Step Y": Get the captured image I j image sharpness.

[0133] As an example, the captured image I may be used j Laplacian variance, to evaluate the captured image I j image sharpness.

[0134] Step Z”: Calculate the captured image I according to the index values ​​and image sharpness under multiple image quality indicators. j Quality score.

[0135] As an example, the index values ​​and image sharpness under various image quality indexes may be weighted and summed to obtain the captured image I j Quality score.

[0136] In summary, different methods can be used to calculate the quality score of each frame of the captured image, which can improve the flexibility and applicability of the method.

[0137] Step S402: for any captured image, perform instance segmentation on any captured image to obtain a shadow area.

[0138] In the embodiment of the present application, for any frame of captured image I j , an image segmentation model (such as MaskR-CNN (Mask Regional Convolutional Neural Network) or other image segmentation networks) can be used to perform image segmentation on the captured image I j Perform instance segmentation to obtain the shadow area S j .

[0139] Step S403: stitch the shadow area with any captured image to obtain a stitched image.

[0140] In the embodiment of the present application, the shadow area S j With the captured image I j Perform stitching to obtain a stitched image.

[0141] Step S404: input the stitched image into the image processing model for shadow correction processing to obtain a corrected image output by the image processing model.

[0142] Among them, image processing models (such as UNet models) have learned how to restore a shadow-free image from the original image and the shadow area.

[0143] In an embodiment of the present application, the stitched image can be input into an image processing model (such as a UNet model) for shadow correction processing to obtain a corrected image output by the image processing model.

[0144] Step S405 , based on the homography matrix of the multiple frames of captured images, align the multiple frames of corrected images to the first reference image to obtain multiple frames of aligned images.

[0145] The homography matrix is ​​used to indicate the perspective transformation relationship of the captured image to the first reference image.

[0146] Step S406: performing image synthesis based on the multiple aligned image frames to obtain a target image.

[0147] The explanation of steps S405 to S406 can be found in the relevant description in any embodiment of the present application and will not be repeated here.

[0148] The image processing method of the embodiment of the present application adopts deep learning technology to perform shadow correction on the captured image, which can take into account both the processing efficiency and processing quality of the image.

[0149] In any embodiment of the present application, the image processing method provided by the present application can be applied to the following fields: image processing and synthesis, multi-frame image processing, video quality enhancement, image alignment and other fields. It can solve the problem that when shooting the same subject (such as shooting objects on the table), the shooting device (such as a smart phone) itself blocks the light, resulting in shadows of the shooting device and the arm in the output image.

[0150] As an example, during the shooting process, the same subject is photographed from different angles to obtain multiple frames of images with different shadow areas, and a neural network is used to fuse the images to generate a high-resolution, high-quality target image without shadows. The solution provided by this application does not require depth of field information, flash, and multiple exposure assistance, and has higher flexibility and better imaging effects.

[0151] The implementation principle of the solution provided in this application can be as follows Figure 5 As shown, it mainly includes the following steps:

[0152] 1. Use a camera to shoot the same object at different angles to obtain an image sequence I1, I2, …, I N ;

[0153] 2. Use Mask R-CNN or other image segmentation networks to classify the image sequence I1, I2, …, I N Extract the shadow area S1, S2, ..., S N ;

[0154] 3. I1, I2, …, I N and S1,S2,…,S N Input to the UNet model, the UNet model performs shadow correction on the image to obtain the shadow-corrected images C1, C2, …, C N ;

[0155] 4. Use the no-reference image quality evaluation method to evaluate the image sequence S1, S2, …, S N Perform quality assessment to obtain the best quality image I best ;

[0156] 5. Calculate the image sequence I1, I2, …, I N Warp to best quality image I best The homography matrix H1,H2,…,H N ;

[0157] For example, the SIFT algorithm can be used to extract the image sequence I1, I2, ..., I N The feature points of each frame of the image in the image sequence I1, I2, ..., I NThe feature points of each frame image and the best quality image I best Match the feature points in to obtain matching feature point pairs, and use the RANSAC algorithm to calculate the optimal H for each frame of image based on the matching feature point pairs;

[0158] 6. Based on H1, H2, ..., H in step 5 N , for C1,C2,…,C N To I best Align, and get C'1, C'2, ..., C' N ;

[0159] 7. Calculate C'1, C'2, ..., C' N With C' best The optical flow vector field between them is based on C'1, C'2, ..., C' N The corresponding optical flow vector fields are respectively for C'1, C'2, ..., C' N Perform pixel-level alignment to obtain multiple frame aligned images;

[0160] 8. Input multiple aligned frames into UNet for image fusion and output shadow-free, high-resolution, high-quality target images.

[0161] In summary, the solution provided by this application has at least the following advantages:

[0162] 1) When shooting a subject from multiple angles, the shadow cast by the arm / shooting device (such as a smartphone) can be removed to improve the imaging effect;

[0163] 2) No depth of field information, flash, or multiple exposure assistance is required, resulting in higher flexibility and better imaging results.

[0164] In order to implement the above embodiment, the embodiment of the present application also proposes an image processing device.

[0165] Figure 6 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application.

[0166] like Figure 6 As shown, the image processing device 600 may include: a determination module 610 , a correction module 620 , an alignment module 630 and a synthesis module 640 .

[0167] The determination module 610 is used to determine the first reference image from the multiple frames of captured images based on the quality scores of the multiple frames of captured images; wherein the multiple frames of captured images are obtained by shooting the same object using different shooting angles;

[0168] A correction module 620, configured to perform shadow correction on the multiple frames of captured images based on shadow areas in the multiple frames of captured images to obtain multiple frames of corrected images;

[0169] An alignment module 630 is used to align the multiple frames of corrected images to the first reference image based on the homography matrix of the multiple frames of captured images to obtain multiple frames of aligned images; wherein the homography matrix is ​​used to indicate the perspective transformation relationship of the captured image to the first reference image;

[0170] The synthesis module 640 is used to perform image synthesis according to multiple frames of aligned images to obtain a target image.

[0171] Furthermore, in one implementation of the embodiment of the present application, the synthesis module 640 is specifically used to: determine a second reference image from multiple frame aligned images; wherein the second reference image is obtained by processing the first reference image; based on the second reference image, locally align the multiple frame aligned images to obtain a multi-frame registered image; and fuse the multiple frame registered images to obtain a target image.

[0172] In one implementation of the embodiment of the present application, the synthesis module 640 is specifically used to: calculate the optical flow vector field between the multi-frame aligned images and the second reference image; wherein the optical flow vector field is used to indicate the moving direction and moving speed of the pixel points in the aligned image relative to the corresponding pixel points in the second reference image; based on the optical flow vector field of the multi-frame aligned images, locally align the multi-frame aligned images to obtain the multi-frame registered images.

[0173] In one implementation of the embodiment of the present application, the image processing device 600 further includes:

[0174] An acquisition module is used to match the first feature point of any captured image with the second feature point of the first reference image for any captured image to obtain multiple matching feature point pairs; perform multiple rounds of iterations based on the multiple matching feature point pairs; wherein any round of iterations includes: randomly selecting the matching feature point pairs required for this round of iterations from the multiple matching feature point pairs, and calculating the homography matrix of this round of iterations based on the matching feature point pairs required for this round of iterations; and selecting the homography matrix of any captured image warped to the first reference image from the homography matrices calculated from the multiple rounds of iterations.

[0175] In one implementation method of an embodiment of the present application, the acquisition module is specifically used to: for any round of iteration process, use the homography matrix of any round of iteration process to project multiple matching feature point pairs, and calculate the projection errors of multiple matching feature point pairs; based on the projection errors of multiple matching feature point pairs, determine the internal points in any round of iteration process from multiple matching feature point pairs; use the homography matrix of the iteration process with the largest number of internal points as the homography matrix for warping any captured image to the first reference image.

[0176] In one implementation method of an embodiment of the present application, the acquisition module is specifically used to: for any round of iterative process, use the homography matrix of any round of iterative process to project multiple matching feature point pairs, and calculate the projection errors of multiple matching feature point pairs; based on the projection errors of multiple matching feature point pairs, determine the projection error of any round of iterative process; use the homography matrix of the iterative process with the smallest projection error as the homography matrix for warping any captured image to the first reference image.

[0177] In one implementation of an embodiment of the present application, the determination module 610 is also used to: extract multi-scale features of any captured image; calculate the degree of deviation between the multi-scale features of any captured image and the statistical model; wherein the statistical model is used to indicate the distribution of multi-scale features of multiple sample images; and determine the quality score of any captured image based on the degree of deviation.

[0178] In one implementation of the embodiment of the present application, the determination module 610 is also used to: extract local contrast features and edge clarity features of any captured image; fuse and normalize the local contrast features and edge clarity features to obtain target features; and perform quality prediction on the target features to obtain a quality score of any captured image.

[0179] In one implementation of an embodiment of the present application, the determination module 610 is also used to: calculate the index value of any captured image under multiple image quality indicators; obtain the image sharpness of any captured image; and calculate the quality score of any captured image based on the index value and image sharpness under multiple image quality indicators.

[0180] In one implementation of an embodiment of the present application, the correction module 620 is specifically used to: for any captured image, perform instance segmentation on any captured image to obtain a shadow area; splice the shadow area with any captured image to obtain a spliced ​​image; input the spliced ​​image into an image processing model for shadow correction processing to obtain a corrected image output by the image processing model.

[0181] It should be noted that the above explanation of the embodiment of the image processing method is also applicable to the image processing device of this embodiment, and will not be repeated here.

[0182] In the image processing device of the embodiment of the present application, during the shooting process, the same shooting object is photographed from different angles to obtain multiple frames of shooting images with different shadow areas, and shadow correction, alignment and synthesis processing are performed on the multiple frames of shooting images, which can eliminate the problem of occlusion and shadow caused by illumination and have better imaging effects. In addition, without the need for depth of field information and multiple exposures, the complexity of image processing can be reduced, the imaging effect can be improved, and the user experience in shooting scenes can be improved.

[0183] In order to implement the above embodiments, the present application also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the image processing method as described in any of the above embodiments is implemented.

[0184] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. For example, the electronic device 700 may be a vehicle, a mobile phone, a computer, a digital broadcast terminal, a message transceiver device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0185] Reference Figure 7 , the electronic device 700 may include one or more of the following components: a processing component 702 , a memory 704 , a power component 706 , a multimedia component 708 , an audio component 710 , an input / output (I / O) interface 712 , a sensor component 714 , and a communication component 716 .

[0186] The processing component 702 generally controls the overall operation of the electronic device 700, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 702 may include one or more modules to facilitate the interaction between the processing component 702 and other components. For example, the processing component 702 may include a multimedia module to facilitate the interaction between the multimedia component 708 and the processing component 702.

[0187] The memory 704 is configured to store various types of data to support operations on the electronic device 700. Examples of such data include instructions for any application or method operating on the electronic device 700, contact data, phone book data, messages, pictures, videos, etc. The memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0188] The power component 706 provides power to the various components of the electronic device 700. The power component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 700.

[0189] The multimedia component 708 includes a screen that provides an output interface between the electronic device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 708 includes a front camera and / or a rear camera. When the electronic device 700 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0190] The audio component 710 is configured to output and / or input audio signals. For example, the audio component 710 includes a microphone (MIC), and when the electronic device 700 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 704 or sent via the communication component 716. In some embodiments, the audio component 710 also includes a speaker for outputting audio signals.

[0191] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0192] The sensor assembly 714 includes one or more sensors for providing various aspects of status assessment for the electronic device 700. For example, the sensor assembly 714 can detect the open / closed state of the electronic device 700, the relative positioning of components, such as the display and keypad of the electronic device 700, and the sensor assembly 714 can also detect the position change of the electronic device 700 or a component of the electronic device 700, the presence or absence of user contact with the electronic device 700, the orientation or acceleration / deceleration of the electronic device 700, and the temperature change of the electronic device 700. The sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 714 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 714 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0193] The communication component 716 is configured to facilitate wired or wireless communication between the electronic device 700 and other devices. The electronic device 700 can access a wireless network based on a communication standard, such as WiFi, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 716 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0194] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0195] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 704 including instructions, and the instructions can be executed by a processor 720 of an electronic device 700 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0196] In order to implement the above embodiments, the present application also proposes a chip, including: the chip includes an interface circuit and a processing circuit coupled to each other. The interface circuit is used to input or output signals, and the processing circuit is configured to execute the image processing method provided in any of the above embodiments.

[0197] Figure 8 is a schematic diagram of the structure of a chip proposed in an embodiment of the present application. Figure 8 The structure of the chip 800 is shown, but is not limited to this.

[0198] The chip 800 includes a processing circuit 801 , and the processing circuit 801 is configured to execute any of the above image processing methods.

[0199] In some embodiments, the chip 800 further includes one or more interface circuits 802. Optionally, the interface circuit 802 is connected to the memory 803, and the interface circuit 802 can be used to receive signals from the memory 803 or other devices, and the interface circuit 802 can be used to send signals to the memory 803 or other devices. For example, the interface circuit 802 can read instructions stored in the memory 803 and send the instructions to the processing circuit 801.

[0200] In some embodiments, the interface circuit 802 performs at least one of the communication steps such as sending and / or receiving in the above method, and the processing circuit 801 performs other steps.

[0201] In some embodiments, terms such as interface circuit, interface, transceiver pin, and transceiver may be used interchangeably.

[0202] In some embodiments, the chip 800 further includes one or more memories 803 for storing instructions. Optionally, all or part of the memory 803 may be outside the chip 800.

[0203] In order to implement the above embodiments, the present application also proposes a non-temporary computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the image processing method as described in any of the above method embodiments is implemented.

[0204] In order to implement the above embodiments, the present application also proposes a computer program product, on which a computer program is stored. When the computer program is executed by a processor, the image processing method as described in any of the above method embodiments is implemented.

[0205] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0206] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0207] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0208] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0209] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0210] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0211] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0212] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. An image processing method, characterized in that: include: Determining a first reference image from the multiple frames of captured images based on quality scores of the multiple frames of captured images; wherein the multiple frames of captured images are obtained by photographing the same subject using different shooting angles; Based on the shadow areas in the multiple frames of captured images, performing shadow correction on the multiple frames of captured images to obtain multiple frames of corrected images; Based on the homography matrix of the multiple frames of captured images, align the multiple frames of corrected images to the first reference image to obtain multiple frames of aligned images; wherein the homography matrix is ​​used to indicate the perspective transformation relationship of the captured images being warped to the first reference image; Image synthesis is performed according to the multiple aligned image frames to obtain a target image.

2. The method according to claim 1, characterized in that The performing image synthesis according to the multiple aligned frames to obtain a target image includes: Determine a second reference image from the multiple aligned image frames; wherein the second reference image is obtained by processing the first reference image; Based on the second reference image, locally register the multiple aligned images to obtain multiple registered images; The multiple frames of registered images are fused to obtain the target image.

3. The method according to claim 2, characterized in that The locally registering the multiple-frame aligned images based on the second reference image to obtain the multiple-frame registered images includes: Calculating an optical flow vector field between the multiple aligned images and the second reference image; wherein the optical flow vector field is used to indicate a moving direction and a moving speed of a pixel point in the aligned image relative to a corresponding pixel point in the second reference image; Based on the optical flow vector field of the multi-frame aligned images, the multi-frame aligned images are locally registered to obtain the multi-frame registered images.

4. The method according to claim 1, characterized in that: The homography matrix of the multi-frame captured images is obtained by following the steps below: For any captured image, matching the first feature point of the captured image with the second feature point of the first reference image to obtain a plurality of matching feature point pairs; Perform multiple rounds of iterations according to the multiple matching feature point pairs; wherein any round of iterations includes: randomly selecting matching feature point pairs required for the current round of iterations from the multiple matching feature point pairs, and calculating the homography matrix of the current round of iterations according to the matching feature point pairs required for the current round of iterations; From the homography matrices calculated in the multiple-round iterative process, a homography matrix of warping any captured image to the first reference image is selected.

5. The method according to claim 4, characterized in that The step of selecting, from the homography matrices calculated from the multiple rounds of iterations, a homography matrix of warping any captured image to the first reference image comprises: For any round of iteration process, the homography matrix of any round of iteration process is used to project the multiple matching feature point pairs, and the projection errors of the multiple matching feature point pairs are calculated; Determining, from the plurality of matching feature point pairs, inliers in any one round of iteration process based on projection errors of the plurality of matching feature point pairs; The homography matrix of the iterative process with the largest number of inliers is used as the homography matrix for warping any captured image to the first reference image.

6. The method according to claim 4, characterized in that The step of selecting, from the homography matrices calculated from the multiple rounds of iterations, a homography matrix of warping any captured image to the first reference image comprises: For any round of iteration process, the homography matrix of any round of iteration process is used to project the multiple matching feature point pairs, and the projection errors of the multiple matching feature point pairs are calculated; Determining the projection error of any round of iterative process based on the projection errors of the multiple matching feature point pairs; The homography matrix of the iterative process with the smallest projection error is used as the homography matrix for warping any captured image to the first reference image.

7. The method according to claim 1, characterized in that The quality scores of the multiple frames of captured images are calculated using the following steps: Extract multi-scale features of any captured image; Calculating the degree of deviation between the multi-scale features of any one of the captured images and the statistical model; wherein the statistical model is used to indicate the distribution of the multi-scale features of a plurality of sample images; Based on the degree of deviation, a quality score of any one of the captured images is determined.

8. The method according to claim 1, characterized in that The quality scores of the multiple frames of captured images are calculated using the following steps: Extract local contrast features and edge definition features of any captured image; The local contrast feature and the edge definition feature are fused and normalized to obtain a target feature; The target feature is quality predicted to obtain a quality score of any captured image.

9. The method according to claim 1, characterized in that: The quality scores of the multiple frames of captured images are calculated using the following steps: Calculate the index values ​​of any captured image under multiple image quality indicators; Obtaining the image sharpness of any one of the captured images; A quality score of any captured image is calculated according to the index values ​​under the multiple image quality indexes and the image sharpness.

10. The method according to any one of claims 1 to 9, characterized in that The step of performing shadow correction on the multiple frames of captured images based on shadow areas in the multiple frames of captured images to obtain multiple frames of corrected images includes: For any captured image, perform instance segmentation on the captured image to obtain a shadow area; Splicing the shadow area with any one of the captured images to obtain a spliced ​​image; The stitched image is input into an image processing model for shadow correction processing to obtain a corrected image output by the image processing model.

11. An image processing device, characterized in that: include: A determination module, configured to determine a first reference image from the plurality of frames of captured images based on quality scores of the plurality of frames of captured images; wherein the plurality of frames of captured images are obtained by photographing the same subject at different shooting angles; A correction module, configured to perform shadow correction on the multiple frames of captured images based on shadow areas in the multiple frames of captured images to obtain multiple frames of corrected images; An alignment module, configured to align the multiple frames of corrected images to the first reference image based on a homography matrix of the multiple frames of captured images to obtain multiple frames of aligned images; wherein the homography matrix is ​​used to indicate a perspective transformation relationship of the captured images being warped to the first reference image; A synthesis module is used to perform image synthesis based on the multiple frames of aligned images to obtain a target image.

12. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A chip, characterized in that: The chip comprises an interface circuit and a processing circuit coupled to each other, the interface circuit is used to input or output a signal, and the processing circuit is used to implement the method according to any one of claims 1 to 10.

14. A non-transitory computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 10 are implemented.

15. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 10.

Images