Image processing method, device, electronic device and storage medium

Abstract

The disclosed embodiments provide an image processing method, device, electronic device and storage medium. The method obtains a low-definition preview image before shooting and a high-definition image to be processed obtained by shooting when it is detected that the image shooting conditions are met; determines a color affine transformation matrix based on the low-definition preview image and the high-definition image to be processed; performs color processing on at least one pixel point in the high-definition image to be processed based on the color affine transformation matrix to obtain and display the high-definition preview image. The technical solution of the disclosed embodiments solves the problem that the color information of the high-definition image and the low-definition preview image in the prior art is inconsistent, resulting in a poor user shooting experience, and realizes processing the high-definition image to be processed into a high-definition preview image that matches the color information of the low-definition preview image based on the color transformation matrix, which not only ensures the clarity of the high-definition preview image, but also ensures that the color information of the high-definition preview image is consistent with the low-definition preview image.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of image processing technology, and in particular, to an image processing method, device, electronic device, and storage medium. Background Art

[0002] With the popularization of smart terminals, more and more users will use smart terminals to record their lives. For example, they can use any application that can perform shooting tasks to record their lives by taking corresponding images. Usually, when shooting, users will adjust the appropriate angle and light in the preview mode and then shoot to obtain the image required by the user.

[0003] However, the preview images are mostly real-time low-definition preview images, and the captured images are mostly non-real-time high-definition images. That is, there is a certain difference in color between the image obtained by the user a moment after pressing the shutter and the image previewed a moment before pressing the shutter, resulting in the problem of inconsistency between the preview image and the captured image. Summary of the invention

[0004] The embodiments of the present disclosure provide an image processing method, device, electronic device and storage medium to achieve color correction of non-real-time high-definition images so that the color information of the obtained high-definition image is consistent with the color information of the low-definition preview image.

[0005] In a first aspect, an embodiment of the present disclosure provides an image processing method, the method comprising:

[0006] When it is detected that the image shooting conditions are met, a low-definition preview image before shooting and a high-definition image to be processed obtained by shooting are obtained;

[0007] Determining a color affine transformation matrix based on the low-definition preview image and the high-definition image to be processed;

[0008] Based on the color affine transformation matrix, color processing is performed on at least one pixel in the high-definition image to be processed to obtain and display a high-definition preview image.

[0009] In a second aspect, an embodiment of the present disclosure further provides an image processing device, the device comprising:

[0010] An image acquisition module is used to acquire a low-definition preview image before shooting and a high-definition image to be processed obtained by shooting when it is detected that the image shooting conditions are met;

[0011] An affine transformation matrix determination module, used to determine a color affine transformation matrix based on the low-definition preview image and the high-definition image to be processed;

[0012] The preview image determination module is used to perform color processing on at least one pixel in the high-definition image to be processed based on the color affine transformation matrix to obtain and display a high-definition preview image.

[0013] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the device comprising:

[0014] one or more processors;

[0015] a storage device for storing one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in any one of the embodiments of the present disclosure.

[0017] In a fourth aspect, the embodiments of the present disclosure further provide a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the image processing method as described in any one of the embodiments of the present disclosure is implemented.

[0018] The technical solution of the disclosed embodiment, when it is detected that the image shooting conditions are met, obtains a low-definition preview image before shooting and a high-definition image to be processed obtained by shooting, and determines a corresponding color affine transformation matrix based on the obtained low-definition preview image and the high-definition image to be processed, and then can perform color processing on at least one pixel point in the high-definition image to be processed based on the color affine transformation matrix to obtain and display the high-definition preview image, thereby solving the problem of inconsistency in color information between the high-definition image and the low-definition preview image in the prior art, resulting in a poor user shooting experience, and determines a color transformation matrix based on the low-definition preview image and the high-definition image to be processed, and performs color transformation processing on the high-definition image to be processed, so as to process the high-definition image to be processed into a high-definition preview image matching the color information of the low-definition preview image, thereby ensuring both the clarity and the color information of the high-definition preview image, and further improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0020] Figure 1 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0021] Figure 2 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0022] Figure 3 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0023] Figure 4 A schematic diagram of a segmented image provided by an embodiment of the present disclosure;

[0024] Figure 5 A schematic diagram of transformation of the segmented area provided in an embodiment of the present disclosure;

[0025] Figure 6 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0026] Figure 7 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0027] Figure 8 A structural block diagram of an image processing device provided by an embodiment of the present disclosure;

[0028] Fig. 9 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0030] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0031] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0032] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0033] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0034] Before introducing the present technical solution, an exemplary description of the application scenario can be given first. The disclosed technical solution can be applied in any scenario that requires image processing, for example, it can be applied in the process of image shooting, and the image corresponding to the photographed user can be processed, such as the user's selfie scenario. It should be noted that the existing image processing software can be software for live broadcast or video shooting, or it can be the camera software that comes with the terminal device, and the shooting method can be to shoot the image through the front camera method, or it can be to shoot through the rear camera method.

[0035] Usually, in order to ensure that users can preview images in real time when taking images, images with lower definition are often obtained first, and low-definition preview images are displayed to users. When the preset conditions are met, high-definition images are obtained by taking pictures. However, when taking pictures, existing camera devices (smart terminals, PC terminals and / or proprietary cameras) can first determine the shooting angle and shooting light, etc. After the determination is completed, the shutter button or shooting control can be pressed to start image shooting. At this time, the image obtained is different from the image during the preview. This is because: the image obtained before the shutter button is pressed is a low-definition preview image with poor image quality (low resolution), and the image obtained after the shutter button is pressed is a non-real-time high-definition image (the image cannot be obtained in real time for preview). At this time, the high-definition image and the low-definition preview image have large differences in color. The above problems greatly interfere with the user's photography creation process and seriously affect the user experience.

[0036] Based on the above problems, in this embodiment, when shooting an image, in order to make the color information of the captured high-definition image and the low-definition preview image consistent, the color of the captured high-definition image can be corrected based on the technical solution of the embodiment of the present disclosure, thereby achieving the technical effect of making the color effects of the high-definition image and the low-definition image consistent.

[0037] Figure 1This is a flow chart of an image processing method provided by an embodiment of the present disclosure. This embodiment can be applied to the situation where color correction is performed on the obtained high-definition image to obtain and display a high-definition preview image that is consistent with the color information of the low-definition preview image. The method can be executed by an image processing device, which can be implemented in the form of software and / or hardware. The hardware can be an electronic device, such as a mobile terminal, a PC, a camera or a server for shooting images, etc.

[0038] It should be noted that the device for executing the image processing method provided by the embodiment of the present disclosure can be integrated into an application software that supports image processing, and the software can be installed in an electronic device. Optionally, the electronic device can be a mobile terminal or a PC. The application software can be a type of software for image / video processing, and its specific application software will not be described one by one here, as long as image / video processing can be achieved. It can also be a specially developed application program to implement the image processing software, or it can be integrated in the corresponding page, and the user can achieve image processing through the page integrated in the PC.

[0039] S110: When it is detected that the image shooting conditions are met, a low-definition preview image before shooting and a high-definition image to be processed obtained by shooting are obtained.

[0040] Among them, the low-definition preview image can be a low-definition image displayed to the user for real-time preview. It is understandable that when the user uses the camera device to shoot, in order to allow the user to understand the approximate effect of the current shooting, the low-definition image is often captured at a set acquisition frequency and displayed to the user. The high-definition image to be processed can be an image captured by the user through the camera device. For example, the high-definition image to be processed can be a high-definition image captured after the user triggers the shooting control. It is understandable that the high-definition image to be processed is a static image, that is, an image captured by the user through the shooting software on the mobile terminal, and can also be a video captured by the video shooting software. Any video frame in the video can be used as the high-definition image to be processed.

[0041] Specifically, in the process of image processing, the image in the real scene can be photographed by the camera on the mobile terminal, the image obtained after the photographing is completed is used as the high-definition image to be processed, and a low-definition preview image displayed to the user before the photographing is obtained. If the video is processed, a frame of image can be selected from the video after the photographing as the high-definition image to be processed, and a low-definition image at the same position in the preview video is selected as the low-definition preview image.

[0042] Exemplarily, the original pre-shooting image data A and post-shooting image data B acquired by the hardware are obtained through the interface provided by the system. Optionally, the result acquired using the camera image interface is A, and a frame of image acquired using the camera video interface is the post-shooting image B. In other application systems, the system can provide two categories for acquiring relevant data, and other platforms can also process similarly. At this time, A is a low-definition but normal-color, real-time image, and B is a high-definition, green-colored, non-real-time image.

[0043] Based on the above scheme, the image shooting conditions include at least one of the following: detecting a triggering image shooting control; detecting that the stay time on the current display interface reaches a preset stay time threshold; detecting that the collected voice information includes image shooting instructions; detecting that the facial information of the target subject is consistent with the preset facial information.

[0044] Among them, the image shooting control can be an interface control displayed on the display interface. The user can trigger the interface control to issue a corresponding instruction, and the application program executes the corresponding function according to the instruction. The display interface can be understood as an operation interface displayed by a terminal device used by the user. The stay time threshold can be a preset time length threshold. The voice information can be a voice command information issued by the user. The shooting instruction can be understood as a preset key word used to instruct the camera to shoot. The target subject can be a user, or any preset shooting subject with facial information. The facial information can be understood as the user's facial feature information.

[0045] In this technical solution, the image shooting conditions may include multiple conditions, that is, there are multiple judgment schemes. The following describes in detail the acquisition method corresponding to each scheme:

[0046] The first method is: when the user needs to take a photo, the user can trigger the image shooting control on the current display interface to issue a corresponding shooting instruction, and then the application will shoot the high-definition image to be processed according to the shooting instruction, and obtain the corresponding low-definition preview image.

[0047] The second method is: the user can pre-set the stay time threshold, for example, the user sets the stay time threshold to 5S. When the application detects that the picture on the current display interface meets the preset 5S stay time, the application completes the shooting by controlling the camera, and uses the shot image as the high-definition image to be processed, and determines the corresponding low-definition preview image according to the preset method.

[0048] The third method is: It is understandable that when the user needs to call the camera in the device through the application and take real-time photos through the camera, when the user cannot directly operate the terminal device to complete the shooting, for example, the user needs to take a full-body photo, or the user needs to pose in a fixed posture and both hands cannot touch the terminal device, the user can send a corresponding voice message, which contains a preset shooting instruction, and then when the application detects the corresponding shooting instruction, it obtains the corresponding high-definition image to be processed and the low-definition preview image.

[0049] The fourth method is: the user's facial information can be pre-recorded. When the application calls the camera to shoot, it can detect whether there is a target subject that matches the user's facial information in the current picture. When the target subject is detected in the picture, the application controls the camera to complete the shooting, and uses the captured image as the high-definition image to be processed, and obtains the corresponding low-definition preview image according to a preset method.

[0050] S120: Determine a color affine transformation matrix based on the low-definition preview image and the high-definition image to be processed.

[0051] The color affine transformation matrix can be used to correct the color of the high-definition image to be processed to obtain a high-definition preview image that matches the color information of the low-definition preview image.

[0052] Specifically, a color affine transformation matrix is ​​obtained by obtaining the low-definition preview image and the high-definition image to be processed. For example, the color features in the low-definition preview image and the high-definition image to be processed are sampled according to a preset sampling method to obtain corresponding color features, and then the color affine transformation matrix is ​​determined according to the color features of the low-definition preview image and the high-definition image to be processed, and then the color of the high-definition image to be processed is corrected according to the color affine transformation matrix to obtain a high-definition preview image that matches the color information of the low-definition preview image.

[0053] It should be noted that the color simulation change matrix determined according to different high-definition images to be processed and low-definition preview images is different, that is, under different environmental conditions, the color difference between the high-definition images to be processed and the low-definition preview images is different. Therefore, different color affine transformation matrices need to be determined for the high-definition images to be processed and the low-definition preview images obtained under different conditions.

[0054] It can be understood that when shooting any image or video, the color affine transformation matrix corresponding to the moment when the shooting control is triggered can be determined in real time or non-real time.

[0055] On the basis of the above technical solution, the color affine transformation matrix is ​​determined based on the low-definition preview image and the high-definition image to be processed, including: determining the low-definition pixel value distribution corresponding to the low-definition preview image, and determining the high-definition pixel value distribution corresponding to the high-definition image to be processed; based on the low-definition pixel value distribution and the high-definition pixel value distribution, determining the color affine transformation matrix.

[0056] The low-definition pixel value distribution may be the distribution information of the pixel values ​​in the low-definition preview image, and correspondingly, the high-definition pixel value distribution may be understood as the distribution information of the pixel values ​​in the high-definition image.

[0057] Specifically, after obtaining the low-definition preview image and the high-definition image, the pixel value distribution in the low-definition preview image and the high-definition image can be statistically analyzed to obtain the corresponding pixel value distribution data, and the corresponding color affine transformation matrix can be obtained according to the corresponding pixel value distribution data. For example, the statistical pixel point data can be formed into a point cloud in its color space, and the point cloud of the high-definition pixel value can be scaled and transformed so that the transformed point cloud is overlapped with the point cloud of the low-definition pixel value as much as possible, thereby obtaining the corresponding color affine transformation matrix.

[0058] It should be noted that the low-definition preview image and the high-definition preview image can be represented in a variety of forms, that is, the color of the image can be described in different color spaces, such as RGB space, Lab space, HSV space, HSL space, YUV space, CMKY space and XYZ space, etc. Correspondingly, the low-definition pixel value distribution and the high-definition pixel value distribution are also pixel distribution information on the corresponding color control, and for any of the above-mentioned color spaces, the pixel value distribution can be analyzed by the technical method provided by the present invention to obtain the corresponding color affine transformation matrix, for example, in the RGB space, by obtaining the RGB pixel value distribution of each pixel point, in the Lab space, the brightness value corresponding to each pixel and the brightness values ​​of the two color channels are obtained, and then the pixel value distribution of the image in the Lab space is obtained.

[0059] S130. Based on the color affine transformation matrix, perform color processing on at least one pixel in the high-definition image to be processed to obtain and display a high-definition preview image.

[0060] Among them, the number of at least one pixel point can be one or more. That is, only a few pixel points in the high-definition image to be processed can be processed, or color processing can be performed on all pixel points in the high-definition image. If only some of the pixel points are processed, the feature points in the image can be determined and color processing can be performed on the feature points. Color processing can be understood as correcting the color values ​​of corresponding pixel points in the high-definition image according to the color affine transformation matrix. The high-definition preview image can be an image obtained by processing the high-definition image to be processed based on the color affine transformation matrix.

[0061] Specifically, color correction is performed on at least one pixel in the high-definition image to be processed according to the color affine transformation matrix, thereby obtaining a corresponding high-definition preview image and displaying it to the user. It is understandable that an image is composed of multiple pixels, and different pixels may have different colors, and different color pixels require different color processing.

[0062] The technical solution of the disclosed embodiment, when it is detected that the image shooting conditions are met, obtains a low-definition preview image before shooting and a high-definition image to be processed obtained by shooting, and determines a corresponding color affine transformation matrix based on the obtained low-definition preview image and the high-definition image to be processed, and then can perform color processing on at least one pixel point in the high-definition image to be processed based on the color affine transformation matrix to obtain and display the high-definition preview image, thereby solving the problem of inconsistency in color information between the high-definition image and the low-definition preview image in the prior art, resulting in a poor user shooting experience, and determines a color transformation matrix based on the low-definition preview image and the high-definition image to be processed, and performs color transformation processing on the high-definition image to be processed, so as to process the high-definition image to be processed into a high-definition preview image matching the color information of the low-definition preview image, thereby ensuring both the clarity and the color information of the high-definition preview image, and further improving the user experience.

[0063] Figure 2 A flowchart of an image processing method provided in an embodiment of the present disclosure further refines "determining a color affine transformation matrix based on the low-definition preview image and the high-definition image to be processed" on the basis of the aforementioned embodiment. It should be noted that there are multiple representation forms of low-definition preview images and high-definition images to be processed. In order to more clearly introduce the specific technical solution of the present application, the RGB color space is used as an example to illustrate how the present application determines the color affine transformation matrix. The specific implementation method can be found in the technical solution of this embodiment. Among them, the technical terms that are the same as or corresponding to the above embodiments are not repeated here.

[0064] S210: When it is detected that the image shooting conditions are met, a low-definition preview image before shooting and a high-definition image to be processed obtained by shooting are obtained.

[0065] S220, determining a low-definition RGB value distribution of at least one low-definition pixel in the low-definition preview image, and a high-definition RGB value distribution of at least one high-definition pixel in the high-definition image to be processed.

[0066] The low-definition pixel point may be a pixel point in the low-definition preview image, and correspondingly, the high-definition pixel point may be understood as a pixel point in the high-definition image to be processed. The low-definition RGB value distribution may be RGB distribution information of at least one pixel point in the low-definition preview image, and correspondingly, the high-definition RGB value distribution may be an RGB value corresponding to at least one pixel point in the high-definition image to be processed.

[0067] It should be noted that the number of low-definition pixels and high-definition pixels can be one or more. The specific number of pixels can be obtained based on a pre-set sampling method and sampling step. The sampling method can be set according to needs to obtain the corresponding number of pixels.

[0068] It can be understood that the RGB value distribution of a pixel is determined by determining the brightness level of the three channels of red, green and blue corresponding to the pixel. It should be noted that the low-definition RGB value distribution can be an RGB value distribution map generated by counting the RGB values ​​of the pixels of the low-definition preview image, and correspondingly, the high-definition RGB value distribution can be an RGB value distribution map generated by counting the RGB values ​​of the pixels in the high-definition image, and the RGB value distribution includes the distribution information of the three color channels of red, green and blue.

[0069] On the basis of the above scheme, at least one low-definition pixel point and at least one high-definition pixel point are determined, including: pixel sampling of the corresponding low-definition preview image and the high-definition image to be processed is performed with a preset sampling step size to obtain the at least one low-definition pixel point and at least one high-definition pixel point.

[0070] The sampling step size includes the number of pixel sampling intervals and / or feature point sampling.

[0071] Specifically, the low-definition preview image and the high-definition image to be processed are sampled by pixels through a preset sampling step length, thereby obtaining at least one low-definition pixel and at least one high-definition pixel. It is understandable that different sampling methods may correspond to different settings of the sampling step length, such as full sampling, that is, sampling of each pixel in the image, sampling of a specific number of pixels with a specific probability, or combining the importance of the pixels and collecting a specific number of pixels based on the importance as a standard.

[0072] Exemplarily, a number of pixel points can be sampled in the low-definition preview image and the high-definition image to be processed, and the values ​​of their color channels can be extracted to obtain two sets of point clouds defined in a specific color space. In addition, sampling can have various forms and strategies, including but not limited to: full sampling, that is, taking all pixel points; sampling a specific number of points with a specific probability; or sampling a specific number of points with importance as probability, etc.

[0073] In the present technical solution, the color space can be selected in a variety of ways, and can be at least one of: RGB space, Lab space, HSV space, HSL space, YUV space, CMKY space and XYZ space.

[0074] S230. Determine a scaling matrix and a center transformation matrix according to the low-definition RGB value distribution and the high-definition RGB value distribution.

[0075] The expansion matrix may be a matrix used to transform the length of the feature vector, and the center change matrix may be used to align the center of the feature vector.

[0076] Specifically, based on the RGB distribution of the low-definition preview image and the RGB distribution of the high-definition image, the corresponding scaling matrix and center transformation matrix can be determined.

[0077] On the basis of the above scheme, according to the low-definition RGB value distribution and the high-definition RGB value distribution, a scaling matrix and a center transformation matrix are determined, including: processing the low-definition RGB value distribution based on the principal component analysis method to determine the low-definition pixel mean of each pixel in the color space; and, processing the high-definition RGB value distribution based on the principal component analysis method to determine the high-definition pixel mean of each pixel in the color space; processing the low-definition preview image or the low-definition preview sub-image based on the principal component analysis method to determine the low-definition matrix; and, processing the high-definition image to be processed or the high-definition sub-image to be processed based on the principal component analysis method to determine the high-definition matrix; orthogonally processing the low-definition matrix and the high-definition matrix to determine the low-definition standard deviation matrix of the low-definition preview image or the low-definition preview sub-image, and the high-definition standard deviation matrix of the high-definition image to be processed or the high-definition sub-image to be processed; and determining the scaling matrix and the center transformation matrix based on the high-definition standard deviation matrix, the low-definition standard deviation matrix, the high-definition matrix and the low-definition matrix.

[0078] Among them, principal component analysis (PCA) can be a method of converting a set of variables that may be correlated into a set of linearly unrelated variables through orthogonal transformation. The converted set of variables is called principal components, and then the information of the original variables can be represented as much as possible based on the principal components. Color space can be understood as a mathematical model for representing color, and specifically, color can be represented by the coordinates of one-dimensional, two-dimensional, three-dimensional or even four-dimensional controls. The low-definition pixel mean can be the average value of the color of each pixel in the low-definition preview image in the color space, and correspondingly, the high-definition pixel mean can be the average value of the color of each pixel in the high-definition image to be processed in the color space. The low-definition matrix can be understood as a matrix composed of multiple eigenvectors obtained after principal component analysis of the low-definition preview image, and correspondingly, the high-definition matrix can be understood as a matrix composed of multiple eigenvectors obtained after principal component analysis of the high-definition image to be processed. The low-definition standard deviation matrix can be a standard deviation matrix obtained after orthogonal transformation of the low-definition matrix, and correspondingly, the high-definition standard deviation matrix can be a standard deviation matrix obtained after orthogonal transformation of the high-definition matrix.

[0079] Exemplarily, after sampling the low-definition preview image and the high-definition image to be processed according to a preset sampling step size, a corresponding number of pixel points are obtained, and the values ​​of the color channels of the sampled number of pixel points are extracted, so that two groups of point clouds defined in the color space can be obtained. Principal component analysis is performed on the two different groups of point clouds to obtain the corresponding means mA and mB and matrices UA and UB consisting of eigenvectors, as well as matrices DA and DB consisting of standard deviations in corresponding directions. It should be noted that A represents the low-definition preview image, and image B represents the high-definition image to be processed. The means mA and mB are the average values ​​of the pixel colors corresponding to image A and image B in the color space, respectively, and they are both vectors. Correspondingly, UA and UB are multiple eigenvectors obtained after principal component analysis of the pixel colors in image A and image B in the color space. and The obtained eigenvectors are arranged in a vector form to form a matrix. Then, for image A, the eigenvectors of image A can be orthogonal to each other, and the color data of each pixel is in the first vector. The standard deviation after projection in the direction is the largest, and this standard deviation is recorded as Next is the second vector The corresponding standard deviation is recorded as Then the third vector The corresponding standard deviation is recorded as And process the feature vectors of and in turn. Correspondingly, the same method can be used for the feature vector of image B. Processing to obtain the corresponding standard deviation DA and DB are the standard deviations of image A and image B. as well as ..., is a matrix composed of diagonal lines, and for the case of using a three-dimensional color space (such as RGB, Lab space, etc.), the above DA and DB as well as UA and UB can be written as:

[0080]

[0081] It should be noted that weighting can be performed in the process of solving the principal component analysis, and there are various ways of weighting. You can choose the appropriate weighting method according to the specific situation. For example, each pixel is weighted equally, or weighted according to the color difference of each pixel before and after taking the photo, the smaller the difference, the greater the weight, or weighted according to the position of each pixel, the closer to the edge of the image, the greater the weight, or using optical flow information to match the image with higher confidence, the greater the weight, or a combination of the above weighting schemes, etc.

[0082] Furthermore, after obtaining the above matrix, by The transformation coefficients of the homogeneous part are calculated. The linear transformation consists of two parts. The first part is the whitening transform (WT): And the second part is Coloring Transform (CT): The two transformations are combined into the WCT transformation, that is: Among them, U -1 represents the inverse operation of matrix U, U T Represents the transpose operation of the matrix U.

[0083] It should be noted that for the whitening transformation, by scaling in the feature direction and dividing by the standard deviation in the corresponding direction, the color distribution of each pixel of the B image becomes an isotropic unit standard deviation distribution, that is, the standard deviation of the data after projection in any direction is 1; and for the coloring transformation, by scaling in the feature direction and multiplying by the standard deviation in the corresponding direction, the isotropic unit standard deviation distribution becomes the color distribution of each pixel of the A image. The WCT transformation obtained by combining the two transforms the color distribution of each pixel of the B image into the color distribution of each pixel of the A image by using the isotropic unit standard deviation distribution as an intermediate medium, thereby achieving color uniformity. Moreover, if the color distribution of each pixel of the A and B images is regarded as ellipsoids A and B respectively, the whitening transformation compresses the ellipsoid B into a unit sphere, and the coloring transformation stretches the unit sphere into the ellipsoid A. The WCT transformation obtained by combining the two transforms the ellipsoid B into the ellipsoid A with the unit sphere as an intermediate medium, thereby achieving color uniformity, because the ellipsoid here represents the color distribution.

[0084] S240. Determine the color affine transformation matrix based on the scaling matrix and the center transformation matrix.

[0085] Specifically, after obtaining the scaling matrix and the center transformation matrix, the final simulation color transformation matrix can be obtained according to the obtained matrix, and the transformation coefficients of the secondary and non-secondary parts can be substituted into the following formula to obtain the complete transformation: And x∈R 3 It represents the value of the RGB color channel of each pixel of the high-definition image to be processed, and the result of the transformation indicates that the color distribution of the image obtained after processing is consistent with that of the high-definition image to be processed.

[0086] It should be noted that the significance of mA and mB is that not only can the direction and standard deviation of the distribution of the eigenvector be kept consistent, but also the average value can be kept consistent, thereby ensuring color consistency. For example, if the color distribution of each pixel of image A and image B is regarded as ellipsoid A and ellipsoid B respectively, then the significance of mA and mB is to ensure that the orientation and center point of the two ellipsoids are consistent.

[0087] S250. Based on the color affine transformation matrix, perform color processing on at least one pixel in the high-definition image to be processed to obtain and display a high-definition preview image.

[0088] On the basis of the above scheme, the color processing is performed on each pixel in the high-definition image to be processed based on the color affine transformation matrix to obtain and display a high-definition preview image, including: multiplying the RGB value of each pixel by the scaling matrix, and then summing it with the center transformation matrix to obtain the high-definition preview image.

[0089] Specifically, when the high-definition image to be processed is processed according to the color affine transformation matrix, the RGB value of each pixel point can be multiplied by the scaling matrix for scaling processing, and then the scaling result is summed with the center transformation matrix, thereby ensuring that the center point position is the same, obtaining the final high-definition preview image, and displaying it to the user.

[0090] The technical solution of the disclosed embodiment, when it is detected that the image shooting conditions are met, obtains a low-definition preview image before shooting and a high-definition image to be processed obtained by shooting, and determines a corresponding color affine transformation matrix based on the obtained low-definition preview image and the high-definition image to be processed, and then can perform color processing on at least one pixel point in the high-definition image to be processed based on the color affine transformation matrix to obtain and display the high-definition preview image, thereby solving the problem of inconsistency in color information between the high-definition image and the low-definition preview image in the prior art, resulting in a poor user shooting experience, and determines a color transformation matrix based on the low-definition preview image and the high-definition image to be processed, and performs color transformation processing on the high-definition image to be processed, so as to process the high-definition image to be processed into a high-definition preview image matching the color information of the low-definition preview image, thereby ensuring both the clarity and the color information of the high-definition preview image, and further improving the user experience.

[0091] Figure 3 This is a flow chart of an image processing method provided by an embodiment of the present disclosure. On the basis of the above-mentioned embodiment, the method of "determining the color affine transformation matrix based on the low-definition preview image and the high-definition image to be processed" is further refined. The specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same or corresponding to the above-mentioned embodiment are not repeated here.

[0092] S310: When it is detected that the image shooting conditions are met, a low-definition preview image before shooting and a high-definition image to be processed obtained by shooting are obtained.

[0093] S320, segmenting the low-definition preview image and the high-definition image to be processed respectively based on a preset segmentation rule to obtain at least one low-definition preview sub-image and at least one high-definition sub-image to be processed.

[0094] On the basis of the above technical solution, in order to further improve the determination of matching the color information of the high-definition preview image with the color information of the low-definition preview image, the high-definition image and the low-definition preview image can be segmented respectively, and the color transformation simulation matrix corresponding to the corresponding segmented area can be determined to improve the accuracy of processing the corresponding sub-image.

[0095] That is to say, when determining the color affine transformation matrix, the information of the entire image is usually processed, that is, the obtained color affine transformation matrix is ​​only applicable to the center point of the image. There may be corresponding errors when using the color affine transformation matrix to process the image. The farther the pixel point is from the center point, the worse the processing effect. In order to solve such problems, the low-definition preview image and the high-definition image to be processed can be segmented, and the color affine transformation matrix corresponding to each segmented area can be obtained to improve the processing effect of the image.

[0096] The preset segmentation rule may be a rule for dividing an image into multiple sub-images in a certain manner. For example, the preset segmentation rule divides the image to be processed into a nine-square grid.

[0097] It should be noted that based on the preset segmentation rules, the low-definition preview image can be segmented into multiple low-definition preview sub-images, and the multiple low-definition preview sub-images can be integrated to obtain the low-definition preview image. Correspondingly, based on the preset segmentation rules, the high-definition image to be processed can be segmented into multiple low-definition to-be-processed high-definition sub-images, and the multiple high-definition to-be-processed high-definition sub-images can be integrated to obtain the high-definition image to be processed.

[0098] Specifically, the obtained low-definition preview image and the to-be-processed high-definition image are segmented according to the segmentation rules preset by the user to obtain at least one low-definition sub-preview image and at least one to-be-processed high-definition sub-image. It is understandable that the segmentation rules can be set according to the needs. For example, if the perfect restoration of the picture color is pursued, the image can be segmented as much as possible, or the image can be segmented into 9 sub-images based on the actual situation.

[0099] Exemplarily, for example, the preset segmentation rule may be to segment the low-definition preview image and the high-definition image to be processed into 16 grids. Then, the low-definition preview image and the high-definition image to be processed may be equally divided according to their length and width information or their pixel information to obtain 4x4 sub-images.

[0100] S330. Determine a color affine transformation matrix corresponding to each segmented area based on the at least one low-definition preview sub-image and the at least one high-definition sub-image to be processed.

[0101] Specifically, after the acquired low-definition preview image and the high-definition image to be processed are segmented, the color affine transformation matrix corresponding to each segmented area is determined according to at least one low-definition preview sub-image and at least one high-definition sub-image to be processed. In other words, the low-definition preview sub-image and the high-definition sub-image to be processed corresponding to the corresponding grid number can be processed to determine the color affine transformation matrix of the corresponding segmented area.

[0102] It should be noted that when determining the color affine transformation matrix corresponding to each segmented area, it is necessary to process the low-definition preview sub-image and the high-definition sub-image to be processed of the corresponding area. For example, they can be divided into 4x4 grids. The sub-images can be marked in the same order, and the sub-images with the same marking symbols can be processed to obtain the color affine transformation matrix corresponding to each grid area. The specific determination method can be referred to the detailed description of Example 2.

[0103] For example, Figure 4 As shown, if the currently acquired low-definition preview image is image A and the high-definition image to be processed is image B, then image A and image B are subjected to the same segmentation processing according to the preset segmentation rule, and then K sub-images are obtained respectively, and finally the high-definition sub-images to be processed B_1, B_2, ..., B_K and the low-definition preview sub-images A_1, A_2, ..., A_K are obtained, and the content positions of the sub-images with the same numerical subscripts are one-to-one corresponding. The preset segmentation processing rule can also be to divide m and n parts in the horizontal and vertical directions respectively, and cut into K = m*n grids, where m and n are both positive integers.

[0104] It should be noted that the methods for processing sub-images with the same mark are the same, and the method for determining the color affine transformation matrix corresponding to a sub-image is used as an example for explanation.

[0105] On the basis of the above scheme, the color affine transformation matrix corresponding to each segmented area is determined based on the at least one low-definition preview sub-image and the at least one high-definition sub-image to be processed, including: for each segmented area, pixel sampling is performed on the low-definition preview sub-image and the high-definition sub-image to be processed of the current segmented area to obtain at least one low-definition pixel and at least one high-definition pixel; the low-definition RGB value distribution of the at least one low-definition pixel and the RGB value distribution of the at least one high-definition pixel are determined; the expansion matrix and the center transformation matrix of each segmented area are determined according to the low-definition RGB value distribution of each segmented area and the corresponding high-definition RGB value distribution; the color affine transformation matrix is ​​determined based on the expansion matrix of each segmented area and the center transformation matrix.

[0106] For example, Figure 5 As shown, for each two corresponding sub-images A_i and B_i, the affine transformation matrix T_i in a color space can be obtained by the method mentioned in the above embodiment to obtain the corresponding relationship between the colors of the two. That is, for each pixel of the high-definition sub-image B_i to be processed, the image B_i' obtained after T_i is applied to each pixel, the color of the image B_i' obtained at this time is consistent with the color of the original image A_i.

[0107] It should be noted that, for each region, the color affine transformation matrix of each segmented region can be determined using S210 to S250 of the embodiments of the present disclosure.

[0108] S340, determining the segmented area corresponding to each pixel point, and adjusting the RGB value of the corresponding pixel point according to the color affine transformation matrix corresponding to the segmented area to obtain the high-definition preview image.

[0109] Specifically, according to the simulated color transformation matrix of each word image to be processed, the RGB values ​​of the pixels in the high-definition sub-images to be processed corresponding to all the segmented areas are adjusted in turn, and then a high-definition preview image is obtained. For example, B_1 is processed by the simulated color transformation matrix obtained based on A_1 and B_1 to obtain B_1', and B_2 is processed by the simulated color transformation matrix obtained based on A_2 and B_2 to obtain B_2', and so on until the high-definition sub-images to be processed in all the segmented areas are processed, and then the corresponding high-definition preview image is obtained.

[0110] For example, see Figure 6 , the image before shooting is recorded as image A, and the image obtained after shooting is recorded as image B. Image A and image B are divided into K sub-images according to a preset blocking algorithm. It should be noted that in order to ensure the correspondence between the sub-images, the blocking algorithms used by image A and image B are the same. Then, an affine transformation solution algorithm is used to process each corresponding sub-image to obtain the affine transformation corresponding to the area. For example, A_1 and B_1 are processed to obtain the affine transformation T_1.

[0111] The benefit of determining the color affine transformation matrix corresponding to each sub-image is that the accuracy of image processing can be further improved, so that the color information of the obtained high-definition preview image is consistent with that of the low-definition preview image.

[0112] The technical solution of the disclosed embodiment is to obtain at least one low-definition preview sub-image and at least one high-definition sub-image to be processed by respectively segmenting the low-definition preview image and the high-definition image to be processed based on preset segmentation rules, and to determine the color affine transformation matrix corresponding to each segmented area based on the at least one low-definition preview sub-image and the at least one high-definition sub-image to be processed, determine the segmented area corresponding to each pixel point, and adjust the RGB value of the corresponding pixel point according to the color affine transformation matrix corresponding to the segmented area to obtain the high-definition preview image, thereby achieving the technical effect of improving the image color processing accuracy.

[0113] Figure 7 This is a flow chart of an image processing method provided by an embodiment of the present disclosure. On the basis of the above-mentioned embodiment, the method of "determining the color affine transformation matrix based on the low-definition preview image and the high-definition image to be processed" is further refined. The specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same or corresponding to the above-mentioned embodiment are not repeated here.

[0114] S410: When it is detected that the image shooting conditions are met, a low-definition preview image before shooting and a high-definition image to be processed obtained by shooting are obtained.

[0115] S420 , using the expansion and contraction matrix and the center transformation matrix of each segmented area as the color affine transformation matrix of the center pixel point of the corresponding segmented area.

[0116] The segmentation area center may be the geometric center of the current segmentation area.

[0117] Specifically, the scaling matrix and the center transformation matrix of the current segmented area are obtained according to a preset method, and are used as the color affine transformation matrix corresponding to the center pixel of the current segmented area. It is understandable that the corresponding scaling matrix and center transformation matrix of different segmented areas may be different, and the positions of different segmented areas are also different, so it is necessary to determine the color affine transformation matrix of the corresponding center pixel of each area according to the scaling matrix and center transformation matrix of each area.

[0118] In order to further improve the color processing accuracy of the HD image to be processed, the color affine transformation matrix of each pixel in the HD image to be processed can be determined, and the HD image to be processed can be transformed in color based on the color affine transformation matrix of each pixel to obtain a HD preview image.

[0119] S430, determining the color affine transformation matrix corresponding to each pixel point by interpolating the color affine transformation matrix of each central pixel point.

[0120] The interpolation process can be understood as using the information of known adjacent pixels to generate the information of unknown pixels, and then determining the color affine transformation matrix corresponding to the unknown pixels.

[0121] Specifically, the information of the surrounding unknown pixels is interpolated according to the information of the central pixel of the current segmented area, and then the color affine transformation matrix corresponding to the unknown pixel is determined according to the information of the unknown pixel obtained after interpolation. It should be noted that the interpolation object may include the transformation coefficient M of the homogeneous part, the mean value m of the low-definition preview image and the high-definition image to be processed A and m B , the non-homogeneous part m A -M*m B Any information that needs to be interpolated, and the interpolation method can be pre-set according to needs, including nearest neighbor interpolation, linear interpolation, bilinear interpolation, bicubic interpolation and other methods.

[0122] Exemplarily, taking the color affine transformation matrix of two adjacent regions as an example, after the color affine transformation matrix of the two adjacent regions is determined, since the color affine transformation matrix corresponds to the color affine transformation matrix of the pixel at the center point of the current region, the color affine transformation matrix of the intermediate pixel can be obtained based on the color affine transformation matrix and the number of pixels between the center points of the adjacent regions using a preset interpolation method, and then the corresponding pixel points can be processed based on the obtained color affine transformation matrix.

[0123] S440, adjusting the RGB value of the corresponding pixel point according to the color affine transformation matrix corresponding to the pixel point to obtain the high-definition preview image.

[0124] Specifically, after adjusting the RGB value of the current pixel point according to the color affine transformation matrix corresponding to the current pixel point, the corresponding high-definition preview image is obtained. It can be understood that after processing each pixel point in turn according to the color affine transformation matrix corresponding to the multiple pixel points obtained after the interpolation processing, a high-definition preview image with color correction is obtained.

[0125] For example, see Figure 6 , each corresponding sub-image is processed by an affine transformation solution algorithm to obtain the affine transformation matrix T_1, T_2, T_3...T_K corresponding to each sub-image. Based on the affine transformation matrix difference processing, the color transformation simulation matrix of each pixel is obtained. Based on the color transformation simulation matrix of each pixel, the corresponding pixel in the high-definition image to be processed is processed to obtain a high-definition preview image.

[0126] The technical solution provided by the embodiment of the present disclosure is to first divide the collected image into blocks, determine the color affine transformation matrix corresponding to each segmented area, and interpolate the color affine transformation matrix to obtain the color affine transformation matrix corresponding to each pixel point, and further, based on each color affine transformation matrix, perform color transformation on the corresponding pixel points to obtain a high-definition preview image, which is consistent with the color information of the low-definition preview image, so as to achieve the effect that the displayed high-definition preview image is consistent with the real-time preview image.

[0127] Figure 8 This is a structural block diagram of an image processing device provided in an embodiment of the present disclosure, which can execute the image processing method provided in any embodiment of the present disclosure and has the corresponding functional modules and beneficial effects of the execution method.

[0128] like Figure 8 As shown, the device specifically includes: an image acquisition module 810, an affine transformation matrix determination module 820 and a preview image determination module 830.

[0129] The image acquisition module 810 is used to acquire a low-definition preview image before shooting and a high-definition image to be processed obtained by shooting when it is detected that the image shooting conditions are met; the affine transformation matrix determination module 820 is used to determine the color affine transformation matrix based on the low-definition preview image and the high-definition image to be processed; the preview image determination module 830 is used to perform color processing on at least one pixel point in the high-definition image to be processed based on the color affine transformation matrix to obtain and display a high-definition preview image.

[0130] On the basis of the above technical solution, the affine transformation matrix determination module is specifically used to: determine the low-definition pixel value distribution corresponding to the low-definition preview image, and determine the high-definition pixel value distribution corresponding to the high-definition image to be processed; based on the low-definition pixel value distribution and the high-definition pixel value distribution, determine the color affine transformation matrix.

[0131] On the basis of the above technical solution, the affine transformation matrix determination module further includes:

[0132] Determine a low-definition RGB value distribution of at least one low-definition pixel in the low-definition preview image, and a high-definition RGB value distribution of at least one high-definition pixel in the high-definition image to be processed;

[0133] Determine a scaling matrix and a center transformation matrix according to the low-definition RGB value distribution and the high-definition RGB value distribution;

[0134] The color affine transformation matrix is ​​determined based on the scaling matrix and the center transformation matrix.

[0135] On the basis of the above technical solution, the affine transformation matrix determination module further includes:

[0136] The image segmentation unit is used to segment the low-definition preview image and the high-definition image to be processed respectively based on a preset segmentation rule to obtain at least one low-definition preview sub-image and at least one high-definition sub-image to be processed; based on the at least one low-definition preview sub-image and the at least one high-definition sub-image to be processed, determine the color affine transformation matrix corresponding to each segmented area.

[0137] On the basis of the above technical solution, the image segmentation unit is specifically used to: for each segmented area, perform pixel sampling on the low-definition preview sub-image and the high-definition sub-image to be processed of the current segmented area to obtain at least one low-definition pixel and at least one high-definition pixel; determine the low-definition RGB value distribution of the at least one low-definition pixel and the RGB value distribution of the at least one high-definition pixel; determine the expansion matrix and the center transformation matrix of each segmented area according to the low-definition RGB value distribution and the corresponding high-definition RGB value distribution of each segmented area; determine the color affine transformation matrix based on the expansion matrix of each segmented area and the center transformation matrix.

[0138] On the basis of the above technical solution, the affine transformation matrix determination module further includes:

[0139] A pixel sampling unit is used to perform pixel sampling on the corresponding low-definition preview image and the high-definition image to be processed with a preset sampling step to obtain the at least one low-definition pixel and at least one high-definition pixel; wherein the sampling step includes the number of pixel sampling intervals and / or feature point sampling.

[0140] On the basis of the above technical solution, the affine transformation matrix determination module further includes:

[0141] A pixel mean value determination unit: used to process the low-definition RGB value distribution based on the principal component analysis method to determine the low-definition pixel mean of each pixel point in the color space; and to process the high-definition RGB value distribution based on the principal component analysis method to determine the high-definition pixel mean of each pixel point in the color space;

[0142] A matrix determination unit, configured to process the low-definition preview image or the low-definition preview sub-image based on the principal component analysis method to determine the low-definition matrix; and to process the high-definition image to be processed or the high-definition sub-image to be processed based on the principal component analysis method to determine the high-definition matrix;

[0143] a standard deviation matrix determining unit, configured to perform orthogonal processing on the low-definition matrix and the high-definition matrix to determine a low-definition standard deviation matrix of the low-definition preview image or the low-definition preview sub-image, and a high-definition standard deviation matrix of the high-definition image to be processed or the high-definition sub-image to be processed;

[0144] The scaling matrix and center transformation matrix determining unit is used to determine the scaling matrix and the center transformation matrix based on the high definition standard deviation matrix, the low definition standard deviation matrix, the high definition matrix and the low definition matrix.

[0145] On the basis of the above technical solution, the image segmentation unit is further used to use the expansion matrix and the center transformation matrix of each segmented area as the color affine transformation matrix of the central pixel point of the corresponding segmented area;

[0146] The color affine transformation matrix corresponding to each pixel point is determined by interpolating the color affine transformation matrix of each central pixel point.

[0147] On the basis of the above technical solution, the preview image determination module is used to: multiply the RGB value of each pixel by the scaling matrix, and then sum it with the central transformation matrix to obtain the high-definition preview image.

[0148] Based on the above technical solution, the preview image determination module is also used to determine the segmented area corresponding to each pixel point, and adjust the RGB value of the corresponding pixel point according to the color affine transformation matrix corresponding to the segmented area to obtain the high-definition preview image.

[0149] On the basis of the above technical solution, the preview image determination module is further used to adjust the RGB value of the corresponding pixel point according to the color affine transformation matrix corresponding to the pixel point to obtain the high-definition preview image.

[0150] The technical solution of the disclosed embodiment, when it is detected that the image shooting conditions are met, obtains a low-definition preview image before shooting and a high-definition image to be processed obtained by shooting, and determines a corresponding color affine transformation matrix based on the obtained low-definition preview image and the high-definition image to be processed, and then can perform color processing on at least one pixel point in the high-definition image to be processed based on the color affine transformation matrix to obtain and display the high-definition preview image, thereby solving the problem of inconsistency in color information between the high-definition image and the low-definition preview image in the prior art, resulting in a poor user shooting experience, and determines a color transformation matrix based on the low-definition preview image and the high-definition image to be processed, and performs color transformation processing on the high-definition image to be processed, so as to process the high-definition image to be processed into a high-definition preview image matching the color information of the low-definition preview image, thereby ensuring both the clarity and the color information of the high-definition preview image, and further improving the user experience.

[0151] The image processing device provided in the embodiments of the present disclosure can execute the image processing method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0152] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0153] Fig. 9 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Fig. 9, which shows an electronic device (eg, Fig. 9 The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0154] like Fig. 9 As shown, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An edit / output (I / O) interface 905 is also connected to the bus 904.

[0155] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although Fig. 9 The electronic device 900 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0156] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0157] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0158] The electronic device provided by the embodiment of the present disclosure and the image processing method provided by the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0159] The embodiments of the present disclosure provide a computer storage medium on which a computer program is stored. When the program is executed by a processor, the image processing method provided by the above embodiments is implemented.

[0160] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0161] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0162] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0163] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0164] When it is detected that the image shooting conditions are met, a low-definition preview image before shooting and a high-definition image to be processed obtained by shooting are obtained;

[0165] Determining a color affine transformation matrix based on the low-definition preview image and the high-definition image to be processed;

[0166] Based on the color affine transformation matrix, color processing is performed on at least one pixel in the high-definition image to be processed to obtain and display a high-definition preview image.

[0167] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0168] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0169] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0170] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0171] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0172] According to one or more embodiments of the present disclosure, [Example 1] provides an image processing method, the method comprising:

[0173] When it is detected that the image shooting conditions are met, a low-definition preview image before shooting and a high-definition image to be processed obtained by shooting are obtained;

[0174] Determining a color affine transformation matrix based on the low-definition preview image and the high-definition image to be processed;

[0175] Based on the color affine transformation matrix, color processing is performed on at least one pixel in the high-definition image to be processed to obtain and display a high-definition preview image.

[0176] According to one or more embodiments of the present disclosure, [Example 2] provides an image processing method, further comprising:

[0177] Determining a low-definition pixel value distribution corresponding to the low-definition preview image, and determining a high-definition pixel value distribution corresponding to the high-definition image to be processed;

[0178] Based on the low-definition pixel value distribution and the high-definition pixel value distribution, the color affine transformation matrix is ​​determined. According to one or more embodiments of the present disclosure, [Example 3] provides an image processing method, further comprising:

[0179] Determine a low-definition RGB value distribution of at least one low-definition pixel in the low-definition preview image, and a high-definition RGB value distribution of at least one high-definition pixel in the high-definition image to be processed;

[0180] Determine a scaling matrix and a center transformation matrix according to the low-definition RGB value distribution and the high-definition RGB value distribution;

[0181] The color affine transformation matrix is ​​determined based on the scaling matrix and the center transformation matrix.

[0182] According to one or more embodiments of the present disclosure, [Example 4] provides an image processing method, further comprising:

[0183] Based on a preset segmentation rule, the low-definition preview image and the high-definition image to be processed are segmented to obtain at least one low-definition preview sub-image and at least one high-definition sub-image to be processed;

[0184] Based on the at least one low-definition preview sub-image and the at least one high-definition sub-image to be processed, a color affine transformation matrix corresponding to each segmented area is determined.

[0185] According to one or more embodiments of the present disclosure, [Example 5] provides an image processing method, further comprising:

[0186] For each segmented area, pixel sampling is performed on the low-definition preview sub-image and the high-definition sub-image to be processed of the current segmented area to obtain at least one low-definition pixel and at least one high-definition pixel;

[0187] Determine a low-definition RGB value distribution of the at least one low-definition pixel and a RGB value distribution of the at least one high-definition pixel;

[0188] Determine the scaling matrix and the center transformation matrix of each segmented area according to the low-definition RGB value distribution and the corresponding high-definition RGB value distribution of each segmented area;

[0189] The color affine transformation matrix is ​​determined based on the scaling matrix of each segmented area and the central transformation matrix.

[0190] According to one or more embodiments of the present disclosure, [Example 6] provides an image processing method, further comprising:

[0191] Performing pixel sampling on the corresponding low-definition preview image and the high-definition image to be processed with a preset sampling step length to obtain the at least one low-definition pixel and the at least one high-definition pixel;

[0192] The sampling step size includes the number of pixel sampling intervals and / or feature point sampling.

[0193] According to one or more embodiments of the present disclosure, [Example 7] provides an image processing method, further comprising:

[0194] Processing the low-definition RGB value distribution based on the principal component analysis method to determine the low-definition pixel mean of each pixel point in the color space; and processing the high-definition RGB value distribution based on the principal component analysis method to determine the high-definition pixel mean of each pixel point in the color space;

[0195] Processing a low-definition preview image or a low-definition preview sub-image based on the principal component analysis method to determine a low-definition matrix; and processing a high-definition image to be processed or a high-definition sub-image to be processed based on the principal component analysis method to determine a high-definition matrix;

[0196] orthogonally processing the low-definition matrix and the high-definition matrix to determine a low-definition standard deviation matrix of the low-definition preview image or the low-definition preview sub-image, and a high-definition standard deviation matrix of the high-definition image to be processed or the high-definition sub-image to be processed;

[0197] The scaling matrix and the center transformation matrix are determined based on the high definition standard deviation matrix, the low definition standard deviation matrix, the high definition matrix and the low definition matrix.

[0198] According to one or more embodiments of the present disclosure, [Example 8] provides an image processing method, further comprising:

[0199] The expansion matrix and the center transformation matrix of each segmented area are used as the color affine transformation matrix of the center pixel point of the corresponding segmented area;

[0200] The color affine transformation matrix corresponding to each pixel point is determined by interpolating the color affine transformation matrix of each central pixel point.

[0201] According to one or more embodiments of the present disclosure, [Example 9] provides an image processing method, further comprising:

[0202] After the RGB value of each pixel is multiplied by the scaling matrix, the sum is processed with the central transformation matrix to obtain the high-definition preview image.

[0203] According to one or more embodiments of the present disclosure, [Example 10] provides an image processing method, further comprising:

[0204] The segmented area corresponding to each pixel is determined, and the RGB value of the corresponding pixel is adjusted according to the color affine transformation matrix corresponding to the segmented area to obtain the high-definition preview image.

[0205] According to one or more embodiments of the present disclosure, [Example 11] provides an image processing method, further comprising:

[0206] According to the color affine transformation matrix corresponding to the pixel point, the RGB value of the corresponding pixel point is adjusted to obtain the high-definition preview image.

[0207] According to one or more embodiments of the present disclosure, [Example 12] provides an image processing device, including:

[0208] An image acquisition module is used to acquire a low-definition preview image before shooting and a high-definition image to be processed obtained by shooting when it is detected that the image shooting conditions are met;

[0209] An affine transformation matrix determination module, used to determine a color affine transformation matrix based on the low-definition preview image and the high-definition image to be processed;

[0210] The preview image determination module is used to perform color processing on at least one pixel in the high-definition image to be processed based on the color affine transformation matrix to obtain and display a high-definition preview image.

[0211] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0212] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0213] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. An image processing method, characterized in that: include: When it is detected that the image shooting conditions are met, a low-definition preview image before shooting and a high-definition image to be processed obtained by shooting are obtained; Determining a color affine transformation matrix based on the low-definition preview image and the high-definition image to be processed; Based on the color affine transformation matrix, color processing is performed on at least one pixel in the high-definition image to be processed to obtain and display a high-definition preview image; The color affine transformation matrix is ​​used to correct the color of the high-definition image to be processed to obtain the high-definition preview image that matches the color information of the low-definition preview image.

2. The method according to claim 1, characterized in that The step of determining a color affine transformation matrix based on the low-definition preview image and the high-definition image to be processed includes: Determining a low-definition pixel value distribution corresponding to the low-definition preview image, and determining a high-definition pixel value distribution corresponding to the high-definition image to be processed; The color affine transformation matrix is ​​determined based on the low-definition pixel value distribution and the high-definition pixel value distribution.

3. The method according to claim 2, characterized in that The determining of the low-definition pixel value distribution corresponding to the low-definition preview image and the determining of the affine transformation of the high-definition pixel value distribution corresponding to the high-definition image to be processed include: Determine a low-definition RGB value distribution of at least one low-definition pixel in the low-definition preview image, and a high-definition RGB value distribution of at least one high-definition pixel in the high-definition image to be processed; Accordingly, based on the low-definition pixel value distribution and the high-definition pixel value distribution, determining the color affine transformation matrix includes: Determine a scaling matrix and a center transformation matrix according to the low-definition RGB value distribution and the high-definition RGB value distribution; The color affine transformation matrix is ​​determined based on the scaling matrix and the center transformation matrix.

4. The method according to claim 1, characterized in that The color affine transformation matrix determined based on the low-definition preview image and the high-definition image to be processed includes: Based on a preset segmentation rule, the low-definition preview image and the high-definition image to be processed are segmented to obtain at least one low-definition preview sub-image and at least one high-definition sub-image to be processed; Based on the at least one low-definition preview sub-image and the at least one high-definition sub-image to be processed, a color affine transformation matrix corresponding to each segmented area is determined.

5. The method according to claim 4, characterized in that The step of determining a color affine transformation matrix corresponding to each segmented area based on the at least one low-definition preview sub-image and the at least one high-definition sub-image to be processed comprises: For the segmented area, pixel sampling is performed on the low-definition preview sub-image and the high-definition sub-image to be processed of the current segmented area to obtain at least one low-definition pixel and at least one high-definition pixel; Determine a low-definition RGB value distribution of the at least one low-definition pixel and a RGB value distribution of the at least one high-definition pixel; Determine the scaling matrix and the center transformation matrix of the corresponding segmented area according to the low-definition RGB value distribution and the corresponding high-definition RGB value distribution of the segmented area; The color affine transformation matrix is ​​determined based on the scaling matrix and the center transformation matrix of the segmented area.

6. The method according to claim 3 or 5, characterized in that: Determining at least one low-definition pixel and at least one high-definition pixel includes: Performing pixel sampling on the corresponding low-definition preview image and the high-definition image to be processed with a preset sampling step length to obtain the at least one low-definition pixel and the at least one high-definition pixel; The sampling step size includes the number of pixel sampling intervals and / or feature point sampling.

7. The method according to claim 3 or 5, characterized in that: Determining a scaling matrix and a center transformation matrix according to the low-definition RGB value distribution and the high-definition RGB value distribution includes: Processing the low-definition RGB value distribution based on the principal component analysis method to determine the low-definition pixel mean of each pixel point in the color space; and processing the high-definition RGB value distribution based on the principal component analysis method to determine the high-definition pixel mean of each pixel point in the color space; Processing a low-definition preview image or a low-definition preview sub-image based on the principal component analysis method to determine a low-definition matrix; and processing a high-definition image to be processed or a high-definition sub-image to be processed based on the principal component analysis method to determine a high-definition matrix; orthogonally processing the low-definition matrix and the high-definition matrix to determine a low-definition standard deviation matrix of the low-definition preview image or the low-definition preview sub-image, and a high-definition standard deviation matrix of the high-definition image to be processed or the high-definition sub-image to be processed; The scaling matrix and the center transformation matrix are determined based on the high definition standard deviation matrix, the low definition standard deviation matrix, the high definition matrix and the low definition matrix.

8. The method according to claim 5, characterized in that Also includes: The expansion matrix and the center transformation matrix of each segmented area are used as the color affine transformation matrix of the center pixel point of the corresponding segmented area; The color affine transformation matrix corresponding to each pixel point is determined by interpolating the color affine transformation matrix of each central pixel point.

9. The method according to claim 7, characterized in that: The color processing is performed on each pixel in the high-definition image to be processed based on the color affine transformation matrix to obtain and display a high-definition preview image, including: After the RGB value of each pixel is multiplied by the scaling matrix, the sum is processed with the central transformation matrix to obtain the high-definition preview image and display it.

10. The method according to claim 5, characterized in that The color processing is performed on each pixel in the high-definition image to be processed based on the color affine transformation matrix to obtain and display a high-definition preview image, including: The segmented area corresponding to each pixel is determined, and the RGB value of the corresponding pixel is adjusted according to the color affine transformation matrix corresponding to the segmented area to obtain and display the high-definition preview image.

11. The method according to claim 8, characterized in that The color processing is performed on each pixel in the high-definition image to be processed based on the color affine transformation matrix to obtain and display a high-definition preview image, including: According to the color affine transformation matrix corresponding to the pixel point, the RGB value of the corresponding pixel point is adjusted to obtain the high-definition preview image and display it.

12. The method according to claim 1, characterized in that The resolution of the high-definition preview image is greater than that of the low-definition preview image and is consistent with the resolution of the high-definition image to be processed; the color information of the high-definition preview image is consistent with the color information of the low-definition preview image.

13. An image processing device, characterized in that: include: An image acquisition module is used to acquire a low-definition preview image before shooting and a high-definition image to be processed obtained by shooting when it is detected that the image shooting conditions are met; An affine transformation matrix determination module, used to determine a color affine transformation matrix based on the low-definition preview image and the high-definition image to be processed; A preview image determination module, used to perform color processing on at least one pixel in the high-definition image to be processed based on the color affine transformation matrix, obtain a high-definition preview image and display it; The color affine transformation matrix is ​​used to correct the color of the high-definition image to be processed to obtain the high-definition preview image that matches the color information of the low-definition preview image.

14. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in any one of claims 1 to 12.

15. A storage medium comprising computer executable instructions, wherein the computer executable instructions are used to perform the image processing method according to any one of claims 1 to 12 when executed by a computer processor.

Images