Color difference detection method and device, electronic device, and storage medium

Abstract

One or more embodiments of the present specification provide a color difference detection method, device, electronic device and storage medium. The color difference detection method includes performing image alignment processing on a captured image and a reference image based on image features, and obtaining a color difference detection result based on a difference between a first color value of a first target pixel area on the captured image and a second color value of a second target pixel area on the reference image. By performing image alignment on the captured image and the reference image, the difference between the captured image and the reference image caused by factors such as shooting angle and lens distortion can be eliminated or alleviated, the color difference performance in real scenes can be effectively characterized, and the shooting color effect can be measured more comprehensively and accurately. In addition, in the process of color difference calculation, the image area is screened based on color smoothness, and an image area with higher smoothness is selected for color calculation, which can effectively reduce the color error caused by edge pixels and improve the color detection effect and accuracy.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of image processing technology, and in particular, to a color difference detection method and device, an electronic device, and a storage medium. Background Art

[0002] Virtual photography is an emerging photography technology based on display screens (e.g., LED (Light Emitting Diode) displays), and is mostly used in special effects shooting for TV series and movies. An important goal of virtual photography is to restore the color of the shooting results to be consistent with the original assets, which requires an effective evaluation of the color difference between the two.

[0003] In the related art, several pure color color cards are generally used to evaluate the shooting color difference of the display screen. This method is difficult to fully reflect the shooting color difference of the display screen, and is far from the color distribution of the real scene, making it difficult to represent the real color effect in the actual scene. Summary of the invention

[0004] In order to improve the color evaluation effect and accuracy of LED virtual photography, one or more embodiments of this specification provide a color difference detection method, device, electronic device and storage medium.

[0005] According to a first aspect of one or more embodiments of this specification, a color difference detection method is proposed, including:

[0006] Acquire a captured image and a reference image, wherein the captured image is an image obtained by capturing a screen displaying the reference image;

[0007] Based on the image features of the captured image and the reference image, performing image alignment processing on the captured image and the reference image;

[0008] Determining a first target pixel region on the captured image and a second target pixel region on the reference image that matches the first target pixel region, wherein color smoothness of the first target pixel region meets a preset condition;

[0009] A color difference detection result is obtained based on a difference between a first color value of the first target pixel area and a second color value of the second target pixel area.

[0010] In one or more implementations of this specification, the performing image alignment processing on the captured image and the reference image based on image features of the captured image and the reference image includes:

[0011] Extracting features from the captured image and the reference image to obtain a plurality of feature points;

[0012] Performing feature matching based on the feature points of the captured image and the feature points of the reference image to obtain multiple groups of feature matching pairs;

[0013] determining a homography matrix between the captured image and the reference image based on the feature matching pairs;

[0014] The captured image is spatially transformed by using the homography matrix to align the captured image to the reference image.

[0015] In one or more embodiments of the present specification, obtaining a color difference detection result based on a difference between a first color value of the first target pixel area and a second color value of the second target pixel area includes:

[0016] For an i-th pixel in the first target pixel area, determining a pixel color difference based on a first color value of the i-th pixel and a second color value of a pixel in the second target pixel area that matches the i-th pixel;

[0017] The color difference detection result is obtained by counting the pixel color differences of all pixels.

[0018] In one or more implementations of this specification, determining a first target pixel region on the captured image and a second target pixel region on the reference image that matches the first target pixel region includes:

[0019] Calculating the gradient value of each pixel on the captured image, and determining the color smoothness corresponding to each image area on the captured image based on the gradient value of the pixel in each image area on the captured image;

[0020] In response to the color smoothness of the image area meeting the preset condition, determining the image area as the first target pixel area;

[0021] The reference image is matched according to the first target pixel region to obtain the second target pixel region.

[0022] In one or more embodiments of the present specification, the color difference detection result includes a color difference distribution histogram, and the color difference detection result is obtained by counting the pixel color differences of all pixels, including:

[0023] The corresponding relationship between the brightness value and the pixel color difference of all pixels is counted to obtain the color difference distribution histogram.

[0024] In one or more implementations of this specification, the performing image alignment processing on the captured image and the reference image based on image features of the captured image and the reference image includes:

[0025] Performing format conversion on the captured image, converting the captured image from a log format to an RGB format;

[0026] Before obtaining the color difference detection result based on the difference between the first color value of the reference image pixel and the second color value of the captured image pixel, the method further includes:

[0027] The reference image and the captured image are converted into a Lab color space by performing a color space conversion.

[0028] In one or more implementations of this specification, the acquiring of the captured image and the reference image includes:

[0029] Acquire the reference image from an asset library;

[0030] The reference image is displayed on the screen, and the screen is photographed to obtain the photographed image.

[0031] According to a second aspect of one or more embodiments of this specification, a color difference detection device is provided, comprising:

[0032] An image acquisition module, configured to acquire a captured image and a reference image, wherein the captured image is an image obtained by capturing a screen displaying the reference image;

[0033] an image alignment module, configured to perform image alignment processing on the captured image and the reference image based on image features of the captured image and the reference image;

[0034] an area determination module, configured to determine a first target pixel area on the captured image, and a second target pixel area on the reference image that matches the first target pixel area, wherein the color smoothness of the first target pixel area meets a preset condition;

[0035] The color difference detection module is configured to obtain a color difference detection result based on a difference between a first color value of the first target pixel area and a second color value of the second target pixel area.

[0036] In one or more embodiments of this specification, the image alignment module is configured to:

[0037] Extracting features from the captured image and the reference image to obtain a plurality of feature points;

[0038] Performing feature matching based on the feature points of the captured image and the feature points of the reference image to obtain multiple groups of feature matching pairs;

[0039] determining a homography matrix between the captured image and the reference image based on the feature matching pairs;

[0040] The captured image is spatially transformed by using the homography matrix to align the captured image to the reference image.

[0041] In one or more embodiments of this specification, the color difference detection module is configured as follows:

[0042] For an i-th pixel in the first target pixel area, determining a pixel color difference based on a first color value of the i-th pixel and a second color value of a pixel in the second target pixel area that matches the i-th pixel;

[0043] The color difference detection result is obtained by counting the pixel color differences of all pixels.

[0044] In one or more implementations of this specification, the area determination module is configured to:

[0045] Calculating the gradient value of each pixel on the captured image, and determining the color smoothness corresponding to each image area on the captured image based on the gradient value of the pixel in each image area on the captured image;

[0046] In response to the color smoothness of the image area meeting the preset condition, determining the image area as the first target pixel area;

[0047] The reference image is matched according to the first target pixel region to obtain the second target pixel region.

[0048] In one or more embodiments of this specification, the color difference detection module is configured as follows:

[0049] The corresponding relationship between the brightness value and the pixel color difference of all pixels is counted to obtain the color difference distribution histogram.

[0050] In one or more embodiments of this specification, the image alignment module is configured to:

[0051] The captured image is format converted from a log format to an RGB format.

[0052] In one or more embodiments of this specification, the color difference detection module is configured as follows:

[0053] The reference image and the captured image are converted into a color space by converting the reference image and the captured image from an RGB color space to a Lab color space.

[0054] In one or more embodiments of this specification, the image acquisition module is configured as follows:

[0055] Acquire the reference image from an asset library;

[0056] The reference image is displayed on the screen, and the screen is photographed to obtain the photographed image.

[0057] According to a third aspect of one or more embodiments of this specification, an electronic device is provided, including:

[0058] Processor; and

[0059] A memory storing computer instructions, wherein the computer instructions are used to enable a processor to execute the method according to any embodiment of the first aspect.

[0060] According to a fourth aspect of one or more embodiments of this specification, a storage medium is provided, which stores computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in any embodiment of the first aspect.

[0061] The color difference detection method of one or more embodiments of the present specification includes acquiring a captured image and a reference image, and performing image alignment processing on the captured image and the reference image based on image features, determining a first target pixel area on the captured image and a second target pixel area on the reference image, and obtaining a color difference detection result based on the difference between a first color value of the first target pixel area and a second color value of the second target pixel area. By performing image alignment on the captured image and the reference image, the difference between the captured image and the reference image caused by factors such as shooting angle and lens distortion can be eliminated or alleviated, the color difference performance in real scenes can be effectively characterized, and the shooting color effect can be measured more comprehensively and accurately. In addition, in the process of color difference calculation, the image area is screened based on color smoothness, and an image area with higher smoothness is selected for color calculation, which can effectively reduce the color error caused by edge pixels and improve the color detection effect and accuracy. In addition, the detection process is simpler, with high detection efficiency and low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a schematic diagram of an application scenario of LED virtual photography in an exemplary embodiment of this specification.

[0063] Figure 2 It is a schematic diagram of a color card used for color difference detection in the related art.

[0064] Figure 3 FIG. 1 is a flowchart of a color difference detection method in an exemplary embodiment of the present specification.

[0065] Figure 4FIG. 1 is a flowchart of a color difference detection method in an exemplary embodiment of the present specification.

[0066] Figure 5 FIG. 1 is a flowchart of a color difference detection method in an exemplary embodiment of the present specification.

[0067] Figure 6 is a schematic diagram of a color difference detection method in an exemplary embodiment of this specification.

[0068] Figure 7 FIG. 1 is a flowchart of a color difference detection method in an exemplary embodiment of the present specification.

[0069] Figure 8 FIG. 1 is a flowchart of a color difference detection method in an exemplary embodiment of the present specification.

[0070] Fig. 9 is a schematic diagram of a color difference detection method in an exemplary embodiment of this specification.

[0071] Fig.10 1 is a block diagram of a color difference detection device in an exemplary embodiment of the present specification.

[0072] Fig.11 is a structural block diagram of an electronic device in an exemplary embodiment of this specification. DETAILED DESCRIPTION

[0073] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0074] It should be noted that: in other implementations, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other implementations, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other implementations; and multiple steps described in this specification may be combined into a single step for description in other implementations.

[0075] In addition, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0076] LED (Light Emitting Diode) is a semiconductor light-emitting device, which is widely used in screen display, lighting and other fields. LED virtual shooting is an emerging photography technology based on LED display screens, which is mostly used in special effects shooting of TV series and movies.

[0077] In traditional video shooting, the crew needs to set up the actual scene and use a lot of props and lights to create the desired atmosphere and effects. This shooting method is costly, time-consuming, and often limited by external factors such as the location and weather. The emergence of LED virtual shooting technology has brought revolutionary changes to the shooting of movies and TV series.

[0078] The basic principle of LED virtual shooting is to use a high-definition LED screen to simulate the actual scene. By projecting a virtual image on the screen, the camera can capture realistic virtual scenes, realizing the fusion shooting of virtual scenes and real scenes. There is no need to arrange the actual scene, which greatly reduces the cost of shooting. In addition, through LED virtual shooting technology, special effects such as rotation, scaling, and movement can be easily achieved, realizing a more realistic special effect vision.

[0079] For example Figure 1 A schematic diagram of an application scenario of LED virtual photography is shown. Figure 1 As shown, the virtual shooting system includes three physical screens (021, 022, 023), the camera 01 can establish communication connections with the three physical screens (021, 022, 023) respectively, the camera 01 can establish a communication connection with the computing device 03, the camera 01 can control each physical screen to display an identification graphic array and send multiple screen images obtained by image acquisition of each physical screen to the computing device 03, and the computing device 03 can generate and display a screen model according to the multiple screen images sent by the camera 01.

[0080] The physical screen used in the virtual shooting system may be an LED display screen, a liquid crystal display screen, or other types, and may be a curved screen or a flat screen. It should be understood that those skilled in the art can customize the type, quantity, size, resolution, etc. of the physical screen in the virtual shooting system according to actual needs, and the embodiments of the present disclosure are not limited to this. It should be understood that the embodiments of the present disclosure are not limited to the communication connection method between devices.

[0081] The disadvantage of LED virtual shooting is that there is often some color difference in LED screens. Therefore, an important goal of LED virtual shooting is to restore the color consistency between the shooting results and the original assets, which requires an effective evaluation of the color difference between the two.

[0082] In the related art, several pure color color cards are generally used to evaluate the color difference of LED photography, such as Figure 2 A color card style is shown. It is worth noting that Figure 2 The color card in the image is grayscaled. In reality, the color card includes blocks of multiple colors. When detecting the color difference of LED photography, you can use the LED screen display Figure 2 The color card shown is then used to shoot the LED screen to obtain the shooting result. By collecting a small number of sampling points on the shooting result and comparing the color values ​​of the sampling points with the color values ​​on the standard color card, the evaluation result of the LED shooting color difference can be calculated.

[0083] However, in this color difference evaluation method, the color difference is calculated only through a small number of pure color sampling points, which lacks representativeness and is difficult to fully reflect the color difference of LED photography. In addition, the color card composed of pure color blocks is far from the color distribution of the real scene, making it difficult to represent the real color effect in the actual scene. Therefore, the accuracy of the LED photography color difference evaluation results in the related art is poor and cannot objectively and accurately reflect the color difference situation.

[0084] Based on the defects existing in the above-mentioned related technologies, the embodiments of this specification provide a color detection method, device, electronic device and storage medium, which aim to eliminate or alleviate the influence of angle, lens distortion and the like in the actual shooting process by aligning the captured image and the reference image, so that the reference image is no longer limited to the color card image, and color difference detection can be achieved using the real scene image, so that the detection effect is more in line with the real color effect in the actual scene, thereby improving the color evaluation effect and accuracy for LED virtual shooting.

[0085] like Figure 3 As shown, in some embodiments, the color difference detection method exemplified in this specification includes:

[0086] S210: Acquire a captured image and a reference image.

[0087] The detection method of the implementation mode of this specification can be mainly used to evaluate the color difference in the LED virtual shooting scene. For the LED virtual shooting scene, the LED screen will display the corresponding virtual scene as needed, so that the camera can capture the image displayed on the LED screen to obtain the captured image.

[0088] The reference image refers to a pre-acquired image of standard color. The reference image can be obtained from an asset library. The asset library can be, for example, a UE (Unireal Engine) asset library. The UE asset library provides various models, textures and other resources. In some implementations of this specification, the reference image can be obtained from the UE asset library. After obtaining the reference image, the reference image can be output and displayed on the LED screen, and then the LED screen is photographed by a camera, and the captured image is obtained by capturing the image with the camera.

[0089] In other words, the contents displayed by the captured image and the reference image are at least partially the same. Due to the influence of shooting angle, distortion, color difference, etc., the image contents displayed by the captured image and the reference image may have color difference, angle deflection, etc.

[0090] It is worth noting that in e.g. Figure 2 In the conventional scheme shown, the reference image is Figure 2 As shown in the standard color card, when the camera shoots the LED screen, it is generally required to face the LED screen to ensure that the captured image has as little angle offset, rotation and distortion as possible compared with the reference image, thereby reducing the error in the color difference evaluation process.

[0091] In the implementation of this specification, the reference image is not limited to being a standard color card image, but can be a material image of any real scene, and the camera does not need to be facing the LED screen to shoot, and the image can be collected completely according to the real shooting scene. For example, in one example, the reference image in the implementation of this specification can be a landscape image obtained from the asset library, and the camera shoots the LED screen from a bird's-eye view to obtain an oblique shooting image.

[0092] S220: Perform image alignment processing on the captured image and the reference image based on image features of the captured image and the reference image.

[0093] It is worth noting that when using material images of real scenes to evaluate color difference, due to the influence of factors such as angle and distortion in the actual shooting process, there may be problems such as angle offset, rotation, and image content distortion between the actual captured image and the reference image.

[0094] Therefore, in the implementation of this specification, before color difference detection is performed on the captured image and the reference image, image alignment processing needs to be performed on the captured image and the reference image. The purpose of image alignment is to spatially align the image content of the captured image with the reference image through spatial transformation methods such as rotation, translation, and scaling, so that the two can perform color difference detection in the same dimension.

[0095] In some implementations, feature extraction may be performed on the captured image and the reference image respectively, feature matching may be performed based on the extracted feature points, and then spatial alignment of the captured image and the reference image may be achieved based on the matching results of the feature points. This process is described in the implementation methods below in this specification.

[0096] It is understandable that when shooting using material images of real scenes, by performing image alignment processing on the captured image and the reference image, the difference between the captured image and the reference image caused by factors such as shooting angle and lens distortion can be eliminated or alleviated, providing an accurate image data basis for subsequent color difference calculations.

[0097] In some embodiments, considering that the image format of the captured image collected by the camera is often different from that of the reference image, after the captured image and the reference image are acquired, the image formats of the two images can be made the same by format conversion. For example, in an exemplary embodiment, the reference image is in sRGB (standard RGB, universal color) format, and the captured image collected by the camera is in log format, so the captured image can be converted from log format to sRGB format by format conversion. This specification describes the format conversion process below.

[0098] S230: Determine a first target pixel region on the captured image and a second target pixel region on the reference image that matches the first target pixel region.

[0099] It is worth noting that in the embodiments of this specification, when using the material image of the real scene as the reference image, considering the influence of factors such as the different blur levels (such as resolution) between the captured image and the reference image, scaling or deformation during the image alignment process, etc., there may be large color errors at the edge texture position of the displayed content of the captured image. Figure 6 Taking the landscape in the example, after the image alignment, the pixels at the edge positions such as the boundary between the mountain and the sky, the boundary between the ocean and the sky, etc. on the captured image may have a large degree of color distortion, thus causing a large color error in color evaluation.

[0100] Based on this, in the implementation manner of this specification, pixel-by-pixel color calculation is not performed on the entire image based on the captured image and the reference image, but the image area on the captured image is screened. The purpose of image area screening is to filter out edge pixel areas that may have large errors, and only retain image areas with higher color smoothness, that is, more uniform color transitions, for color difference calculation, so as to ensure the effect and accuracy of color evaluation.

[0101] For example, in some implementations, the color smoothness of each image area on the captured image can be first determined based on the pixel gradient on the captured image. The color smoothness can reflect the uniformity of color transition in each image area. The higher the color smoothness, the more uniform the color change, and vice versa.

[0102] Then, a corresponding preset condition may be set in advance based on the color smoothness, and the preset condition may be, for example, that the color smoothness of the image area is greater than a preset threshold. That is, when the color smoothness corresponding to a certain image area is greater than the preset threshold, it means that the color smoothness of the image area meets the preset condition.

[0103] Afterwards, through the above screening process for each image area on the captured image, the image area that meets the preset conditions is determined as the target area on the captured image, that is, the first target pixel area described in this specification. Finally, combined with the above, it can be seen that each pixel on the captured image can be matched to a corresponding pixel on the reference image. Therefore, after determining the first target pixel area on the captured image, the second target pixel area corresponding to it on the reference image can be matched through pixel matching.

[0104] S240 , obtaining a color difference detection result based on a difference between a first color value of the first target pixel region and a second color value of the second target pixel region.

[0105] It can be understood that in the LED virtual shooting scene, the image range captured by the camera generally does not exceed the display range of the LED screen. Therefore, the image content of the captured image is generally less than or equal to the image content of the reference image. Therefore, each pixel on the captured image can be matched with the corresponding pixel on the reference image.

[0106] In addition, in the related art, there are many colorimetric modes for representing colors, such as the common RGB color model, CMY color model, Lab color model, CIE1931-XYZ color model, CIE1931-Yxy color model, etc. These color models are all three-dimensional color spaces, that is, they have color expression components in three dimensional directions.

[0107] To capture any pixel on the image iFor example, in the RGB color space, its color value is (R i , G i , B i ), the pixel can be matched to the corresponding pixel Pixel in the reference image i-ref Pixel i-ref The color value is (R i-ref , G i-ref , B i-ref ). It can be understood that the color value on the reference image (R i-ref , G i-ref , B i-ref ) is the standard color, and the color value on the captured image (R i , G i , B i ) is the captured color, so the difference between the two is the LED shooting color difference.

[0108] Therefore, in the implementation manner of this specification, the color value of the pixel on the captured image is defined as the first color value, and the color value of the pixel on the reference image is defined as the second color value. The color difference detection for LED shooting is to detect the color difference between the first color value of the pixel on the captured image and the second color value of the pixel on the reference image.

[0109] In combination with the aforementioned S230, it can be known that in the implementation mode of this specification, the color difference between the first color value of the first target pixel area on the captured image and the second color value of the second target pixel area on the reference image can be calculated pixel by pixel, and then the color difference results are statistically analyzed to obtain the final color difference detection result.

[0110] In some embodiments, before calculating the color difference between the captured image and the reference image, the captured image and the reference image may be converted from sRGB format to Lab image format, which is described below in this specification.

[0111] It is worth noting that in the implementation manner of this specification, by aligning the captured image and the reference image, the difference between the captured image and the reference image caused by factors such as shooting angle and lens distortion can be eliminated or alleviated, so the reference image is not limited to the use of Figure 2 Instead of using the standard color chart shown, you can use the material images of real scenes. Figure 2 The traditional solution shown in the figure uses real scene material images for color difference evaluation, which has the following advantages:

[0112] 1) By utilizing the diversity of color distribution in real scenes, we can effectively characterize the color difference in real scenes, avoid the huge difference between simple pure color sampling points and actual scenes, and measure the color effect of shooting more comprehensively and accurately;

[0113] 2) There is no need to impose any restrictions on the camera's shooting angle and hardware. Adaptive and rapid color difference calibration can be achieved at any angle, avoiding the time-consuming and high-cost manual debugging.

[0114] From the above, it can be seen that in the implementation of this specification, color difference detection of real scenes can be realized. By aligning the captured image and the reference image, the difference between the captured image and the reference image caused by factors such as shooting angle and lens distortion can be eliminated or alleviated. The color difference performance in real scenes can be effectively characterized, and the shooting color effect can be measured more comprehensively and accurately. The detection process is simpler, with high detection efficiency and low cost. In addition, in the process of color difference calculation, the image area is screened based on color smoothness, and the image area with higher smoothness is selected for color calculation, which can effectively reduce the color error caused by edge pixels and improve the color detection effect and accuracy.

[0115] like Figure 4 As shown, in some embodiments, in the color difference detection method exemplified in this specification, the process of acquiring the captured image and the reference image includes:

[0116] S310 , obtaining a reference image from an asset library.

[0117] S320: Display the reference image on the screen, and photograph the screen to obtain a photographed image.

[0118] In the implementation manner of this specification, the asset library refers to a resource library that provides images of various standard colors. In the related art, there are many such asset libraries, such as the UE asset library, etc. In the implementation manner of this specification, images of standard colors can be obtained from these asset libraries, that is, the reference image I mentioned in this specification. ref .

[0119] After obtaining the reference image I ref After that, the reference image I can be output and displayed on the LED screen ref Then use the camera to shoot the LED screen to get the captured image I cap .

[0120] It can be understood from the above that since the LED screen displays the reference image I ref , so the captured image I cap The image content is similar to the reference image I ref The image content is the same as that of the image I, and the image I is generally taken cap The image range is smaller, so the image I cap The image content is generally the reference image I ref In addition, in the implementation mode of this specification, there is no need to limit the shooting angle, hardware, etc., so the shooting image Icap The image content is generally similar to the reference image I ref The image content has angle offset, rotation, distortion, etc.

[0121] In some implementations, the image I captured by the camera cap Generally in log format, the reference image I ref Generally, it is in (s)RGB format. Therefore, in order to keep the captured image I cap With reference image I ref The format of the captured image I cap Specifically, the captured image I cap The log signal domain is converted to the linear domain, and then the color domain is converted from the camera's own color domain to the BT709 color domain, and finally the image signal is converted to the sRGB signal to achieve the captured image I cap Conversion from log format to sRGB format. For the process of image format conversion, those skilled in the art can undoubtedly understand and fully implement it by referring to the relevant technology, and this specification will not elaborate on it.

[0122] After completing the image format conversion, the captured image I cap With reference image I ref Perform image alignment processing, and combine Figure 5 Provide explanation.

[0123] like Figure 5 As shown, in some embodiments, in the color difference detection method exemplified in this specification, the image alignment process includes:

[0124] S410: Extract features from the captured image and the reference image to obtain multiple feature points.

[0125] In the implementation mode of this specification, first, the captured images I can be respectively cap and reference image I ref Perform feature extraction. The purpose of feature extraction is to extract multiple feature points from the image. The feature points are used to characterize the texture characteristics of the image.

[0126] In the implementation manner of this specification, there is no limitation on the algorithm for extracting image feature points. For example, algorithms such as SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Feature), ORB (Oriented FAST and Rotated BRIEF), Harris corner detection, etc. can be used. Those skilled in the art can undoubtedly understand and fully implement them by referring to the relevant technology, and this specification will not elaborate on them.

[0127] In the case of taking images I cap and reference image I ref After feature extraction, the captured image I can be obtained cap The multiple feature points extracted from the image and the reference image I ref For the sake of convenience, the following definition is used for reference image I ref The feature point on is the first feature point, and the captured image I cap The feature point on is the second feature point.

[0128] S420 , performing feature matching based on feature points of the captured image and feature points of the reference image to obtain multiple groups of feature matching pairs.

[0129] Through the above method process, the reference image I is extracted ref A plurality of first feature points on the image I cap Next, feature matching can be performed based on these feature points. The purpose of feature matching is to associate the first feature point and the second feature point that describe the same point on the image. The first feature point and the second feature point that have such an association relationship are a set of feature matching pairs.

[0130] For example Figure 6 In this example, the left side of the figure shows the captured image I cap , the right side is the reference image I ref Through feature extraction, multiple feature points on the two images can be extracted respectively, and then through feature matching, the first feature point and the second feature point describing the same point on the two images can be associated as a set of feature matching pairs. Each connecting line in the figure represents a set of feature matching pairs.

[0131] In some implementations, after matching multiple sets of feature matching pairs, data filtering can be performed on the multiple feature matching pairs to filter out feature matching pairs with obvious anomalies. In addition, feature points that are not matched to feature matching pairs can also be directly filtered out.

[0132] S430 : Determine a homography matrix between the captured image and the reference image based on the feature matching pair.

[0133] Combined with the above, it can be seen that the two feature points included in a set of feature matching pairs represent the same point on the image, so the captured image I can be determined based on the feature matching pairs. cap With reference image I ref The spatial mapping relationship between the captured images I cap With reference image I ref The homography matrix between them.

[0134] The homography matrix is ​​also called the homography matrix or the H matrix. It refers to the mapping relationship between the two plane projections of an object. That is, the projection coordinates of an object in one plane can be mapped to another plane through the homography matrix.

[0135] S440 , spatially transform the captured image using a homography matrix to align the captured image to the reference image.

[0136] Combined with the above, we can know that the captured image I can be transformed into cap The points on the image are mapped to the reference image I ref In the space, therefore, when obtaining the captured image I cap With reference image I ref After the homography matrix between the two is obtained, the homography matrix can be used to map the captured image I cap Perform spatial transformation, which may include, for example, scaling, translation, rotation, etc., and the captured image I can be transformed into cap Mapped to the reference image I ref In the space of cap With reference image I ref Align the images.

[0137] For example Figure 6 In this example, the captured image I on the left can be transformed into cap Compared with the reference image I on the right ref Perform image alignment.

[0138] From the above, it can be seen that in the implementation mode of this specification, when shooting using material images of real scenes, the differences between the captured images and the reference images caused by factors such as shooting angle and lens distortion can be eliminated or alleviated by performing image alignment processing on the captured images and the reference images, thereby providing an accurate image data basis for subsequent color difference calculations.

[0139] It is worth mentioning that in the RGB color model, the R (red) value may be negative. The reason for the negative value is that the RGB brightness is cumulative. If there is darker light, the RGB component must be added to the color to be matched, resulting in a negative value. It can be seen that the RGB color space is actually a very uneven space.

[0140] CIE (Commission International Eclairage) developed the Lab color space in 1931. The Lab color model makes up for the shortcomings of the RGB color model. It is a device-independent color model. One element in Lab is brightness L, and a and b are two color channels.

[0141] In the embodiments of this specification, considering the non-uniformity of the RGB color space, a slight change in one color channel will cause a significant change in color, so it is difficult to directly compare color differences. cap With reference image I ref After image alignment, the captured image I cap With reference image I ref Perform color space conversion, convert both from the original RGB color space to the Lab color space, and then implement color difference calculation based on the Lab color space.

[0142] As for the color space conversion process, those skilled in the art can undoubtedly understand and fully implement it by referring to the relevant technologies, and this specification will not elaborate on it.

[0143] In the captured image I cap With reference image I ref After color space conversion, the color difference between the two can be calculated. Figure 7 and Figure 8 The implementation method is described.

[0144] like Figure 7 As shown, in some embodiments, the color difference detection method exemplified in this specification, the process of determining the first target pixel area and the second target pixel area includes:

[0145] S710: Calculate the gradient value of each pixel on the captured image, and determine the color smoothness corresponding to the image area based on the gradient value of the pixel in each image area on the captured image.

[0146] S720: In response to the color smoothness of the image area meeting a preset condition, determine the image area as a first target pixel area.

[0147] S730: Match the captured image according to the first target pixel area to obtain a second target pixel area.

[0148] In the implementation manner of this specification, the image gradient can be understood as the uniformity of the change of image pixels. The higher the gradient value, the more likely it is an edge texture, and the lower the gradient value, the more likely it is a flat texture. Therefore, in some implementation manners, the image I can be calculated. cap There are many algorithms for calculating the gradient value of each pixel, such as the pixel gradient formula, etc., which will not be described in detail in this manual. Figure 6 In the example, the gradient value of the pixel located at the edge contour where the mountain and the sea meet is generally very high, while the gradient value of the pixel located in the flat area in the middle of the sky or the ocean is generally low.

[0149] In some embodiments, the captured image can be divided into multiple image areas. For each image area, the color smoothness corresponding to the image area can be calculated based on the gradient value of the pixels included in the image area. The color smoothness can reflect the uniformity of the color change in the image area, so the higher the color smoothness, the more uniform the color change, and vice versa.

[0150] For example, in an exemplary implementation, taking an image region as an example, the average value of the gradient values ​​of all pixels in the image region can be calculated, and then the color smoothness corresponding to the average value can be calculated based on the correlation function between the color smoothness and the gradient average value. The correlation function is a negative correlation function, that is, the smaller the gradient average value, the higher the color smoothness of the image region. The specific expression of the correlation function can be selected according to specific needs, such as a reciprocal expression, etc., and the present disclosure does not limit this.

[0151] After calculating the color smoothness of each image region, the color smoothness can be compared with a preset threshold value. The preset threshold value represents a critical value. Taking an image region as an example, if the color smoothness of the image region is greater than the preset threshold value, it means that the color change in the image region is more uniform, and the image region is a flat area. On the contrary, if the color smoothness of the image region is less than or equal to the preset threshold value, it means that the color change in the image region is more uneven, and the image region is likely to be an edge region. For the specific value of the preset threshold value, those skilled in the art can select it according to the specific scene requirements, and the present disclosure does not limit this.

[0152] In the implementation manner of the present specification, when the color smoothness is greater than a preset threshold, it can be determined that the color smoothness meets the preset condition, so that the image area on the captured image that meets the preset condition is determined as the first target pixel area.

[0153] It is to be understood that the method of screening the first target pixel area in this specification is not limited to the above example, and other suitable methods may be used. For example, in an exemplary implementation, after calculating the gradient value of each pixel on the captured image, the pixel gradient value may be compared with a preset gradient threshold value, which represents a critical value for pixel screening. If the captured image I cap If the gradient value of a pixel on the image is greater than or equal to the preset gradient threshold, it means that the pixel is likely to be located at the edge contour position. If it is involved in the color difference calculation, it will introduce a large calculation error and needs to be screened out. cap If the gradient value of a certain pixel on the image is less than the preset gradient threshold, it means that the pixel is likely to be located in a flat area and can participate in the color difference calculation. Therefore, through this pixel screening process, the image area formed by all pixels on the captured image whose gradient values ​​are less than the preset gradient threshold can be determined as the first target pixel area. Those skilled in the art can understand and fully implement this, and this disclosure will not elaborate on it.

[0154] Combining the above, it can be seen that the captured image I cap Each pixel on the reference image I ref The corresponding pixel is matched to the image I. cap After the first target pixel area on the image is obtained, the reference image I can be matched by pixel matching. ref A second target pixel area corresponding thereto.

[0155] like Figure 8 As shown, in some embodiments, the color difference detection method of the present disclosure example, the process of determining the color difference detection result includes:

[0156] S810 . For an i-th pixel in a first target pixel region, determine a pixel color difference based on a first color value of the i-th pixel and a second color value of a pixel matching the i-th pixel in a second target pixel region.

[0157] In the implementation manner of this specification, any pixel Pixel in the first target pixel area i For example, define it when shooting image I cap The color value on is the first color value. In the Lab color space, the first color value can be expressed as (L i , a i , b i ). Similarly, the pixel i The corresponding pixel Pixel can be matched in the second target pixel area i-ref , define pixel Pixel i-ref In the reference image I refThe color value above is the second color value. In the Lab color space, the second color value can be expressed as (L i-ref , a i-ref , b i-ref ).

[0158] Combined with the above, we can know that the reference image I ref The second color value (L i-ref , a i-ref , b i-ref ) represents the standard color, and the captured image I cap The first color value (L i , a i , b i ) represents the LED shooting color, and the difference between the two is the pixel color difference ΔE of the pixel. In some implementations of this specification, the CIEDE2000 color difference formula proposed by the International Commission on Illumination CIE can be used to calculate the pixel color difference ΔE. Those skilled in the art can understand and fully implement it by referring to relevant technologies, and this specification will not elaborate on it.

[0159] S820: Count the pixel color differences of all pixels to obtain a color difference detection result.

[0160] In the implementation manner of the present specification, after the pixel color differences of all pixels in the first target pixel area are calculated through the above process, the pixel color differences can be counted to obtain the final color difference detection result.

[0161] In some embodiments of the present specification, a histogram of pixel brightness value (L) and pixel color difference (ΔE) can be used to represent the color difference detection result. That is, the pixel brightness value and pixel color difference of each pixel are counted and expressed as (L, ΔE). For pixels with the same pixel brightness value, the color difference average value of these pixels with the same pixel brightness value can be calculated, and the color difference average value is used as the pixel color difference (ΔE) corresponding to the pixel brightness value. Then, the pixel brightness value and pixel color difference of each pixel counted are represented in a histogram. For example, in one example, the histogram of the color difference detection result can be as follows: Fig. 9 As shown, in Fig. 9 In the figure, the horizontal axis represents the pixel brightness value (L), and the vertical axis represents the pixel color difference (ΔE). Fig. 9 From the color difference detection results shown, it can be seen that the color difference is higher in the low brightness range, while the color difference is relatively lower in the high brightness range, which intuitively reflects the color difference distribution and degree.

[0162] Of course, those skilled in the art can understand that after obtaining the pixel color difference of each pixel, the color difference detection result can also be represented by other methods, such as calculating the variance or mean of the pixel color difference of all pixels, and using the variance or mean to represent the overall shooting color difference, and this manual does not limit this.

[0163] As can be seen from the above, in the implementation of this specification, the color difference detection of the real scene can be realized, and the difference between the captured image and the reference image caused by factors such as shooting angle and lens distortion can be eliminated or alleviated, and the color difference performance under the real scene can be effectively characterized, and the shooting color effect can be measured more comprehensively and accurately, and the detection process is simpler, and the detection efficiency is high and the cost is low. And in the color difference calculation process, the image area is screened based on the color smoothness, and the image area with higher smoothness is selected for color calculation, which can effectively reduce the color error caused by the edge pixels and improve the color detection effect and accuracy. In addition, before calculating the color difference, the image is converted from the RGB color space to the Lab color space to improve the color difference calculation accuracy. In addition, before calculating the color difference, by pixel screening, the edge contour pixels with large calculation errors can be filtered out, thereby further improving the color difference calculation accuracy.

[0164] like Fig.10 As shown, in some embodiments, this specification provides a color difference detection device, including:

[0165] The image acquisition module 10 is configured to acquire a captured image and a reference image, wherein the captured image is an image obtained by capturing a screen displaying the reference image;

[0166] An image alignment module 20, configured to perform image alignment processing on the captured image and the reference image based on image features of the captured image and the reference image;

[0167] The region determination module 30 is configured to determine a first target pixel region on the captured image and a second target pixel region on the reference image that matches the first target pixel region, wherein the color smoothness of the first target pixel region meets a preset condition;

[0168] The color difference detection module 40 is configured to obtain a color difference detection result based on a difference between a first color value of the first target pixel area and a second color value of the second target pixel area.

[0169] From the above, it can be seen that in the implementation of this specification, color difference detection of real scenes can be realized. By aligning the captured image and the reference image, the difference between the captured image and the reference image caused by factors such as shooting angle and lens distortion can be eliminated or alleviated. The color difference performance in real scenes can be effectively characterized, and the shooting color effect can be measured more comprehensively and accurately. The detection process is simpler, with high detection efficiency and low cost. In addition, in the process of color difference calculation, the image area is screened based on color smoothness, and the image area with higher smoothness is selected for color calculation, which can effectively reduce the color error caused by edge pixels and improve the color detection effect and accuracy.

[0170] In one or more embodiments of the present specification, the image alignment module 20 is configured as follows:

[0171] Extracting features from the captured image and the reference image to obtain a plurality of feature points;

[0172] Performing feature matching based on the feature points of the captured image and the feature points of the reference image to obtain multiple groups of feature matching pairs;

[0173] determining a homography matrix between the captured image and the reference image based on the feature matching pairs;

[0174] The captured image is spatially transformed by using the homography matrix to align the captured image to the reference image.

[0175] In one or more embodiments of this specification, the color difference detection module 40 is configured as follows:

[0176] For an i-th pixel in the first target pixel area, determining a pixel color difference based on a first color value of the i-th pixel and a second color value of a pixel in the second target pixel area that matches the i-th pixel;

[0177] The color difference detection result is obtained by counting the pixel color differences of all pixels.

[0178] In one or more implementations of this specification, the area determination module 30 is configured to:

[0179] Calculating the gradient value of each pixel on the captured image, and determining the color smoothness corresponding to each image area on the captured image based on the gradient value of the pixel in each image area on the captured image;

[0180] In response to the color smoothness of the image area meeting the preset condition, determining the image area as the first target pixel area;

[0181] The reference image is matched according to the first target pixel region to obtain the second target pixel region.

[0182] In one or more embodiments of this specification, the color difference detection module 40 is configured as follows:

[0183] The corresponding relationship between the brightness value and the pixel color difference of all pixels is counted to obtain the color difference distribution histogram.

[0184] In one or more embodiments of the present specification, the image alignment module 20 is configured as follows:

[0185] The captured image is format converted from a log format to an RGB format.

[0186] In one or more embodiments of this specification, the color difference detection module 40 is configured as follows:

[0187] The reference image and the captured image are converted into a color space by converting the reference image and the captured image from an RGB color space to a Lab color space.

[0188] In one or more embodiments of this specification, the image acquisition module 10 is configured as follows:

[0189] Acquire the reference image from an asset library;

[0190] The reference image is displayed on the screen, and the screen is photographed to obtain the photographed image.

[0191] As can be seen from the above, in the implementation of this specification, the color difference detection of the real scene can be realized, and the difference between the captured image and the reference image caused by factors such as shooting angle and lens distortion can be eliminated or alleviated, and the color difference performance under the real scene can be effectively characterized, and the shooting color effect can be measured more comprehensively and accurately, and the detection process is simpler, and the detection efficiency is high and the cost is low. And in the color difference calculation process, the image area is screened based on the color smoothness, and the image area with higher smoothness is selected for color calculation, which can effectively reduce the color error caused by the edge pixels and improve the color detection effect and accuracy. In addition, before calculating the color difference, the image is converted from the RGB color space to the Lab color space to improve the color difference calculation accuracy. In addition, before calculating the color difference, by pixel screening, the edge contour pixels with large calculation errors can be filtered out, thereby further improving the color difference calculation accuracy.

[0192] In one or more implementations of this specification, an electronic device is proposed, including:

[0193] Processor; and

[0194] A memory stores computer instructions, wherein the computer instructions are used to enable a processor to execute the method described in any of the above embodiments.

[0195] In one or more implementations of the present specification, a storage medium is provided, which stores computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in any of the above implementations.

[0196] Fig.11 is a schematic structural diagram of a device provided by an exemplary embodiment. Fig.11 At the hardware level, the device includes a processor 702, an internal bus 704, a network interface 706, a memory 708, and a non-volatile memory 710, and may also include hardware required for other scenarios. One or more embodiments of this specification may be implemented based on software, such as the processor 702 reading the corresponding computer program from the non-volatile memory 710 into the memory 708 and then running it. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0197] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0198] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0199] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0200] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0201] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0202] The above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0203] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "one", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0204] It should be understood that although the terms first, second, third, etc. may be used to describe various information in one or more embodiments of this specification, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0205] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of this specification shall be included in the scope of protection of one or more embodiments of this specification.

Claims

1. A color difference detection method for evaluating color difference in a virtual LED shooting scene, the method comprising: Acquire a captured image and a reference image, wherein the captured image is an image obtained by capturing a screen displaying the reference image, and an image range of the captured image is smaller than or equal to that of the reference image; and the reference image is a material image of a real scene; Based on the image features of the captured image and the reference image, performing image alignment processing on the captured image and the reference image; Determine a first target pixel area on the captured image, and a second target pixel area on the reference image that matches the first target pixel area, wherein the first target pixel area is a local area within the screen of the captured image, and the color smoothness of the first target pixel area meets a preset condition, wherein the preset condition includes: the color smoothness is greater than a preset threshold; A color difference detection result is obtained based on a difference between a first color value of the first target pixel area and a second color value of the second target pixel area, wherein the difference is LED shooting color difference.

2. The method according to claim 1, wherein the performing image alignment processing on the captured image and the reference image based on image features of the captured image and the reference image comprises: Extracting features from the captured image and the reference image to obtain a plurality of feature points; Performing feature matching based on the feature points of the captured image and the feature points of the reference image to obtain multiple groups of feature matching pairs; determining a homography matrix between the captured image and the reference image based on the feature matching pairs; The captured image is spatially transformed by using the homography matrix to align the captured image to the reference image.

3. The method according to claim 1, wherein obtaining a color difference detection result based on a difference between a first color value of the first target pixel area and a second color value of the second target pixel area comprises: For an i-th pixel in the first target pixel area, determining a pixel color difference based on a first color value of the i-th pixel and a second color value of a pixel in the second target pixel area that matches the i-th pixel; The color difference detection result is obtained by counting the pixel color differences of all pixels.

4. The method according to claim 1, wherein determining a first target pixel region on the captured image and a second target pixel region on the reference image that matches the first target pixel region comprises: Calculating the gradient value of each pixel on the captured image, and determining the color smoothness corresponding to each image area on the captured image based on the gradient value of the pixel in each image area on the captured image; In response to the color smoothness of the image area meeting the preset condition, determining the image area as the first target pixel area; The reference image is matched according to the first target pixel region to obtain the second target pixel region.

5. The method according to claim 3, wherein the color difference detection result comprises a color difference distribution histogram, and the step of obtaining the color difference detection result by counting the pixel color differences of all pixels comprises: The corresponding relationship between the brightness value and the pixel color difference of all pixels is counted to obtain the color difference distribution histogram.

6. The method according to claim 1, wherein the performing image alignment processing on the captured image and the reference image based on image features of the captured image and the reference image comprises: Performing format conversion on the captured image, converting the captured image from a log format to an RGB format; Before obtaining the color difference detection result based on the difference between the first color value of the reference image pixel and the second color value of the captured image pixel, the method further includes: The reference image and the captured image are converted into a Lab color space by performing a color space conversion.

7. The method according to claim 1, wherein acquiring the captured image and the reference image comprises: Acquire the reference image from an asset library; The reference image is displayed on the screen, and the screen is photographed to obtain the photographed image.

8. A color difference detection device for evaluating the color difference in a virtual LED shooting scene, the device comprising: an image acquisition module, configured to acquire a captured image and a reference image, wherein the captured image is an image obtained by capturing a screen displaying the reference image, and an image range of the captured image is smaller than or equal to that of the reference image; and the reference image is a material image of a real scene; an image alignment module, configured to perform image alignment processing on the captured image and the reference image based on image features of the captured image and the reference image; an area determination module, configured to determine a first target pixel area on the captured image, and a second target pixel area on the reference image that matches the first target pixel area, wherein the first target pixel area is a local area within the screen on the captured image, and the color smoothness of the first target pixel area meets a preset condition, wherein the preset condition includes: the color smoothness is greater than a preset threshold; The color difference detection module is configured to obtain a color difference detection result based on a difference between a first color value of the first target pixel area and a second color value of the second target pixel area, wherein the difference is LED shooting color difference.

9. An electronic device, comprising: processor; and A memory storing computer instructions, wherein the computer instructions are used to enable a processor to execute the method according to any one of claims 1 to 7.

10. A storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images