Color adjustment method, device, electronic device, storage medium, and program product

Abstract

The present disclosure relates to the field of virtual photography, and proposes a color adjustment method, device, electronic device, storage medium, and program product. The method includes: acquiring a camera screen, the camera screen includes a real object and a virtual object, the virtual object is an object included in a first image displayed on a real screen, the first image is generated by a rendering engine, and the virtual object corresponds to a real object; respectively acquiring the colors of the virtual object and the real object in the camera screen; determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired colors; causing the rendering engine to regenerate the first image using the color adjustment method and color adjustment parameters, and reacquiring the camera screen when the new first image is displayed on the real screen. The color adjustment method of the embodiment of the present disclosure supports automatic acquisition of color, automatic determination of color adjustment method and color adjustment parameters, reduces the labor cost of color adjustment, and improves the efficiency and accuracy of color adjustment.

Description

Technical Field

[0001] The present disclosure relates to the field of virtual photography, and in particular to a color adjustment method, device, electronic device, storage medium, and program product. Background Art

[0002] Virtual filming is a technique used in film and television production, including movies, TV series, and commercials. A common virtual filming approach involves combining a virtual scene generated by a computer rendering engine with a real scene for filming. For example, a screen is placed within a real scene, and an image containing virtual objects is generated by a rendering engine and displayed on the screen. A real object corresponding to the virtual object is then placed in front of the screen. This allows the virtual and real objects to appear simultaneously in the camera's frame.

[0003] Ideally, the colors of virtual objects and real objects in the camera image should be consistent. However, in real-world camera images, virtual objects may differ in color from real objects, resulting in a sense of disconnection between the virtual and real objects in the camera image. To improve the shooting effect, the colors of virtual objects and real objects in the camera image need to be as close as possible. Existing techniques typically involve manually acquiring the color of objects in the camera image, determining the color adjustment method and parameters, etc. This increases the labor cost of color adjustment and, due to the low accuracy of manual operation, reduces the efficiency and accuracy of color adjustment. Summary of the Invention

[0004] In view of this, the present disclosure provides a color adjustment method, apparatus, electronic device, storage medium, and program product. The color adjustment method of the disclosed embodiments supports automatic acquisition of color, automatic determination of color adjustment methods and color adjustment parameters, reducing the labor cost of color adjustment and improving the efficiency and accuracy of color adjustment.

[0005] According to one aspect of the present disclosure, a color adjustment method is provided, the method comprising: acquiring a camera screen, the camera screen comprising a real object and a virtual object, the virtual object being an object included in a first image displayed on a real screen, the first image being generated by a rendering engine, and the virtual object corresponding to the real object; respectively acquiring the colors of the virtual object and the real object in the camera screen; determining a color adjustment method and color adjustment parameters of the rendering engine based on the acquired colors, wherein the color adjustment method is related to the degree of color difference between the virtual object and the real object in the camera screen; causing the rendering engine to regenerate the first image using the color adjustment method and the color adjustment parameters, and reacquiring the camera screen when the new first image is displayed on the real screen.

[0006] In one possible implementation, after reacquiring the camera image, the method further includes: respectively acquiring the colors of the virtual object and the real object in the camera image; and when the difference in color between the virtual object and the real object in the camera image is greater than a first preset threshold, re-executing the steps of determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired colors and subsequent steps.

[0007] In a possible implementation, before acquiring the camera screen, the method further includes: performing color calibration on the camera based on the first image on the real screen and the first image in the camera screen, so that the color of the first image on the real screen is consistent with the color of the first image in the camera screen.

[0008] In one possible implementation, the color space of the rendering engine is a linear color space. Before determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired color, the method further includes: performing linear processing on the acquired colors of the virtual object and the real object, and converting the linearly processed colors into colors in the color space of the rendering engine.

[0009] In one possible implementation, the color adjustment parameters include the gain of each color channel, and determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired color includes: determining a first color vector based on the acquired color of the real object; determining a second color vector based on the acquired color of the virtual object; for each color channel, determining the gain of the color channel based on the ratio of the values ​​of the first color vector and the second color vector in the color channel; in the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the same color channel in the previously generated first image and the gain of the color channel.

[0010] In one possible implementation, the color adjustment parameters include a gain and an offset. Determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired color includes: determining a third color vector and a fourth color vector based on the acquired color of the real object; determining a fifth color vector and a sixth color vector based on the acquired color of the virtual object; calculating, for each color channel, a first difference between the values ​​of the third color vector and the fourth color vector in the color channel, and a second difference between the values ​​of the fifth color vector and the sixth color vector in the color channel, and determining the gain of the color channel based on a ratio of the first difference to the second difference; calculating, for each color channel, a product of the value of the fifth color vector in the color channel and the gain of the color channel, and determining the offset of the color channel based on the difference between the value of the third color vector in the color channel and the product; and in the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the previously generated first image in the same color channel and the gain of the color channel, and then adding the product to the offset of the color channel.

[0011] In a possible implementation, determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired color includes: determining the hue distribution of the real object based on the acquired color of the real object; determining the hue distribution of the virtual object based on the acquired color of the virtual object; when the difference between the hue distribution of the real object and the hue distribution of the virtual object is greater than a second preset threshold, determining that the color adjustment method is global color adjustment; when the difference between the hue distribution of the real object and the hue distribution of the virtual object is less than or equal to the second preset threshold, determining that the color adjustment method is local color adjustment.

[0012] In a possible implementation, the method is executed by a plug-in of a rendering engine.

[0013] According to another aspect of the present disclosure, a color adjustment device is provided, which includes: a first acquisition module, used to acquire a camera screen, wherein the camera screen includes a real object and a virtual object, the virtual object is an object included in a first image displayed on a real screen, the first image is generated by a rendering engine, and the virtual object corresponds to the real object; a second acquisition module, used to respectively acquire the colors of the virtual object and the real object in the camera screen; a first determination module, used to determine a color adjustment method and color adjustment parameters of the rendering engine based on the acquired colors, wherein the color adjustment method is related to the degree of color difference between the virtual object and the real object in the camera screen; a first generation module, used to enable the rendering engine to regenerate the first image using the color adjustment method and the color adjustment parameters, and when the real screen displays a new first image, enable the first acquisition module to reacquire the camera screen.

[0014] In one possible implementation, the device further includes: a third acquisition module, configured to respectively acquire the colors of the virtual object and the real object in the camera image after the first acquisition module reacquires the camera image; the device is further configured to, when the difference in color between the virtual object and the real object in the camera image is greater than a first preset threshold, re-execute the steps of determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired colors and subsequent steps.

[0015] In one possible implementation, the device further includes: a calibration module, configured to perform color calibration on the camera based on the first image on the real screen and the first image in the camera screen before the first acquisition module acquires the camera screen, so that the color of the first image on the real screen is consistent with the color of the first image in the camera screen.

[0016] In one possible implementation, the color space of the rendering engine is a linear color space, and the device further includes: a conversion module, configured to perform linear processing on the acquired colors of the virtual object and the real object before the first determination module determines the color adjustment method and color adjustment parameters of the rendering engine based on the acquired colors, and convert the linearly processed colors into colors in the color space of the rendering engine.

[0017] In one possible implementation, the color adjustment parameters include the gain of each color channel, and determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired color includes: determining a first color vector based on the acquired color of the real object; determining a second color vector based on the acquired color of the virtual object; for each color channel, determining the gain of the color channel based on the ratio of the values ​​of the first color vector and the second color vector in the color channel; in the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the same color channel in the previously generated first image and the gain of the color channel.

[0018] In one possible implementation, the color adjustment parameters include a gain and an offset. Determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired color includes: determining a third color vector and a fourth color vector based on the acquired color of the real object; determining a fifth color vector and a sixth color vector based on the acquired color of the virtual object; calculating, for each color channel, a first difference between the values ​​of the third color vector and the fourth color vector in the color channel, and a second difference between the values ​​of the fifth color vector and the sixth color vector in the color channel, and determining the gain of the color channel based on a ratio of the first difference to the second difference; calculating, for each color channel, a product of the value of the fifth color vector in the color channel and the gain of the color channel, and determining the offset of the color channel based on the difference between the value of the third color vector in the color channel and the product; and in the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the previously generated first image in the same color channel and the gain of the color channel, and then adding the product to the offset of the color channel.

[0019] In a possible implementation, determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired color includes: determining the hue distribution of the real object based on the acquired color of the real object; determining the hue distribution of the virtual object based on the acquired color of the virtual object; when the difference between the hue distribution of the real object and the hue distribution of the virtual object is greater than a second preset threshold, determining that the color adjustment method is global color adjustment; when the difference between the hue distribution of the real object and the hue distribution of the virtual object is less than or equal to the second preset threshold, determining that the color adjustment method is local color adjustment.

[0020] In a possible implementation, the device is a plug-in of a rendering engine.

[0021] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0022] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0023] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0024] According to the color adjustment method of the embodiment of the present disclosure, a camera image is acquired, the camera image including a real object and a virtual object, the virtual object being an object included in a first image displayed on a real screen, the first image being generated by a rendering engine, and the virtual object corresponding to the real object, so that the virtual object in the camera image can be used as an object to be color-adjusted, and the real object in the camera image can be used as a color adjustment standard; the colors of the virtual object and the real object in the camera image are respectively acquired, and a color adjustment method and color adjustment parameters of the rendering engine are determined based on the acquired colors. Because the color adjustment method and color adjustment parameters are determined based on the colors of the virtual object and the real object in the camera image, the color adjustment method and color adjustment parameters can make the color of the virtual object in the camera image closer to the color of the real object; the color adjustment method is related to the degree of color difference between the virtual object and the real object in the camera image, and therefore the determined color adjustment method is more compatible with actual application scenarios; the rendering engine regenerates the first image using the color adjustment method and color adjustment parameters, and the first image will automatically complete color adjustment without pixel-by-pixel color adjustment. When the new first image is displayed on the real screen, the colors of the virtual object and the real object in the reacquired camera image will be closer. In summary, the color adjustment method of the embodiment of the present disclosure supports automatic acquisition of color, automatic determination of color adjustment method and color adjustment parameters, reduces the labor cost of color adjustment, and improves the efficiency and accuracy of color adjustment.

[0025] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0027] Figure 1 An exemplary application scenario of the color adjustment method according to an embodiment of the present disclosure is shown.

[0028] Figure 2 A schematic diagram illustrating the process of a color adjustment method according to an embodiment of the present disclosure.

[0029] Figure 3 A schematic diagram illustrating the process of a color adjustment method according to an embodiment of the present disclosure.

[0030] Figure 4 A schematic diagram illustrating the process of a color adjustment method according to an embodiment of the present disclosure.

[0031] Figure 5 A schematic diagram illustrating the process of a color adjustment method according to an embodiment of the present disclosure.

[0032] Figure 6 An example of a first portion of pixels and a second portion of pixels occupied by a virtual object and a real object in a camera screen according to an embodiment of the present disclosure is shown.

[0033] Figure 7 A schematic diagram illustrating the structure of a color adjustment device according to an embodiment of the present disclosure.

[0034] Figure 8 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0035] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0036] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0037] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0038] The following is an introduction to the terms that may appear in this article.

[0039] Virtual filming: refers to the process of creating and capturing real-time images and videos using digital technology, game engines (such as Unreal Engine or Unity), and technologies such as virtual reality or augmented reality.

[0040] Color space: A mathematical model for describing and representing color. Using a color space and testing on physical devices, fixed analog and digital representations of color can be obtained.

[0041] Color calibration: In virtual shooting, color calibration means that when the camera captures the screen, the on-screen image displayed on the camera screen is consistent with the actual color of the image displayed on the screen.

[0042] Post-Processing Volume (PPV): A tool for controlling visual effects after rendering. It can apply various visual effects such as color correction, blur, depth of field, and bloom to specific areas of the game scene, and is often used for global color adjustments.

[0043] Color Correction (CCR) is an important post-processing effect used to adjust the color and brightness in a specific area to make it consistent with the rest of the scene or stand out in a specific visual style. It is often used for local color adjustment.

[0044] Plugin: A carrier that can customize functions in the rendering engine and can be applied to other projects.

[0045] Figure 1 An exemplary application scenario of the color adjustment method according to an embodiment of the present disclosure is shown.

[0046] like Figure 1 As shown, in an exemplary application scenario, a first device, a camera, a screen, and a real object may be included. A camera generally refers to a device with the ability to capture images, such as a video camera. The camera image can be displayed on the first device. The first device may be provided with a rendering engine (not shown), such as Unreal Engine (UE). The first device may use the rendering engine to render an image including a virtual object, and output it to the screen for display. Virtual objects may correspond to real objects. For example, when the real object is a beach, the virtual object may also be a beach. The embodiments of the present disclosure do not limit the specific types of virtual objects and real objects.

[0047] The color spaces of the rendering engine, screen, and camera image can be of the same type, for example, all RGB three-channel color spaces, but the numerical ranges of the same color channel in these three color spaces may be different. For example, in the rendering engine's color space, the numerical range of red may be 0-100, in the screen's color space, the numerical range of red may be 0-70, and in the camera's color space, the numerical range of red may be 0-50. The disclosed embodiments do not impose any restrictions on the type of each color space or the numerical range of each color channel.

[0048] To ensure rendering accuracy, the rendering engine's color space can be set to a linear color space. In a linear color space, color components are linearly related to the physical intensity of light, and color mixing and transformations follow the rules of linear algebra. This makes color processing and calculations more intuitive and convenient. However, screens and cameras display images to humans. To accommodate human visual sensitivity, the color space of real screens and camera screens can be set to a nonlinear color space.

[0049] The staff can adjust the camera's viewing angle and the position of the real object as needed, so that the virtual object and the real object appear simultaneously in the camera image. The first device can then execute the color adjustment method of the disclosed embodiment to determine the color adjustment method and color adjustment parameters of the rendering engine. When the rendering engine renders the image using the new color adjustment method and color adjustment parameters and displays it on the screen, the color of the virtual object in the camera image becomes closer to the color of the real object, ultimately adjusting the color of the content on the camera image.

[0050] Those skilled in the art should understand that, in actual applications, the rendering function and the color adjustment function may also be implemented by different electronic devices. For example, a second device (not shown) may be used to implement the rendering function, and the first device may be used to implement the color adjustment function. The embodiments of the present disclosure do not limit the specific functions of the first device.

[0051] Figure 2 A schematic diagram illustrating the process of a color adjustment method according to an embodiment of the present disclosure.

[0052] like Figure 2 As shown, in one possible implementation, the method includes:

[0053] Step S21, obtaining a camera image, where the camera image includes real objects and virtual objects, where the virtual objects are objects included in a first image displayed on the real screen, where the first image is generated by a rendering engine, and the virtual objects correspond to real objects;

[0054] Step S22, respectively obtaining the colors of the virtual object and the real object in the camera image;

[0055] Step S23, determining a color adjustment method and color adjustment parameters of a rendering engine according to the acquired color, wherein the color adjustment method is related to the degree of color difference between the virtual object and the real object in the camera image;

[0056] Step S24 , enabling the rendering engine to regenerate the first image using the color adjustment method and the color adjustment parameters, and reacquiring the camera image when the new first image is displayed on the real screen.

[0057] For example, when a first image generated by a rendering engine has been displayed on a real screen and the first image includes a virtual object, step S21 can be executed to obtain a camera screen. The camera screen may include real objects and virtual objects. Virtual objects correspond to real objects. For example, virtual objects and real objects may be objects of the same type whose colors need to be aligned. Virtual objects and real objects with a corresponding relationship that need to be aligned in color can be determined by object recognition and other related technologies, such as identifying virtual objects and real objects of the same type and adjacent positions. Virtual objects and real objects that need to be aligned in color can also be manually selected. Due to the viewing angle of the camera, the camera screen may include all or part of the real object, as well as all or part of the virtual object. The embodiment of the present disclosure does not limit whether the real objects and virtual objects included in the camera screen are complete.

[0058] Next, step S22 can be executed to obtain the colors of the virtual object and the real object in the camera image, respectively. Taking obtaining the color of the virtual object as an example, assuming that the color space of the camera image is a three-channel RGB color space, and the virtual object occupies multiple pixels in the camera image, a certain number of pixels can be selected from them, and the values ​​of the selected pixels in the three color channels can be obtained. Based on the values ​​of each pixel in the three color channels, a color vector can be obtained. The method for obtaining the color of the real object can be the same as the method for obtaining the color of the virtual object, and will not be repeated here. The details of obtaining the colors of the virtual object and the real object in the camera image are given later.

[0059] The embodiments of the present disclosure do not limit the method of selecting pixels. Manual selection can be performed, or certain selection conditions can be set. A deep learning algorithm can be used to identify pixels that meet the conditions and select pixels based on the recognition results. As long as the selected pixels are the pixels occupied by the object whose color is to be obtained in the camera image, it can be selected.

[0060] Those skilled in the art should understand that the acquired color can also be represented in forms other than color vectors, as long as the acquired color can represent the actual color of the object whose color is to be acquired in the camera image. The embodiments of the present disclosure do not limit the specific form of the acquired color.

[0061] After obtaining the colors of the virtual object and the real object in the camera image, step S23 may be executed to determine the color adjustment method and color adjustment parameters of the rendering engine based on the obtained colors. The color adjustment method and color adjustment parameters are used to adjust the color of the image output by the rendering engine, thereby changing the color of the image displayed on the real screen, ultimately making the color of the virtual object in the camera image more similar to the color of the real object.

[0062] Exemplarily, the rendering engine may support multiple color adjustment methods, such as the post-processing volume PPV and regional color correction CCR described above. The determined color adjustment method may be related to the degree of color difference between the virtual object and the real object in the camera image. The determined color adjustment method may be one of the multiple color adjustment methods supported by the rendering engine. The rendering engine may use at least one color adjustment parameter, such as gain (Gain) and offset (Offset), etc., in each color adjustment method. The embodiment of the present disclosure does not limit the specific type of the color adjustment method determined in step S23 and the specific type of color adjustment parameter. Details of determining the color adjustment method and color adjustment parameters of the rendering engine are given later.

[0063] Furthermore, a prerequisite for determining the rendering engine's color adjustment method and color adjustment parameters can be that the color difference between the virtual object and the real object in the camera image is significant, for example, greater than a first preset threshold. The first preset threshold can be the boundary value at which the color difference between the virtual object and the real object meets the requirements of the application scenario. By adding this prerequisite, data processing costs can be further reduced. The value of the first preset threshold can be set according to the requirements of the application scenario, and the embodiments of the present disclosure do not limit the specific value of the first preset threshold.

[0064] After determining the color adjustment method and color adjustment parameters of the rendering engine, step S24 can be executed to cause the rendering engine to regenerate the first image using the determined color adjustment method and color adjustment parameters. The regenerated first image can have the same content as the original first image, and the difference can be the color of the image.

[0065] The rendering engine can output the regenerated first image to the real screen for display. While the real screen is displaying the new first image, the camera image can be recaptured. In this case, the color of the virtual object in the recaptured camera image can be closer to the color of the real object.

[0066] Steps S23 and S24 can be implemented using a pre-written plug-in for the rendering engine. By starting the plug-in before executing step S23, steps S23 and S24 can be automatically executed. Steps S21 and S22 can also be implemented using the plug-in (starting the plug-in before executing step S21) or directly by the processor, which is not limited in the present embodiment.

[0067] According to the color adjustment method of the embodiment of the present disclosure, a camera image is acquired, the camera image including a real object and a virtual object, the virtual object being an object included in a first image displayed on a real screen, the first image being generated by a rendering engine, and the virtual object corresponding to the real object, so that the virtual object in the camera image can be used as an object to be color-adjusted, and the real object in the camera image can be used as a color adjustment standard; the colors of the virtual object and the real object in the camera image are respectively acquired, and a color adjustment method and color adjustment parameters of the rendering engine are determined based on the acquired colors. Because the color adjustment method and color adjustment parameters are determined based on the colors of the virtual object and the real object in the camera image, the color adjustment method and color adjustment parameters can make the color of the virtual object in the camera image closer to the color of the real object; the color adjustment method is related to the degree of color difference between the virtual object and the real object in the camera image, and therefore the determined color adjustment method is more compatible with actual application scenarios; the rendering engine regenerates the first image using the color adjustment method and color adjustment parameters, and the first image will automatically complete color adjustment without pixel-by-pixel color adjustment. When the new first image is displayed on the real screen, the colors of the virtual object and the real object in the reacquired camera image will be closer. In summary, the color adjustment method of the embodiment of the present disclosure supports automatic acquisition of color, automatic determination of color adjustment method and color adjustment parameters, reduces the labor cost of color adjustment, and improves the efficiency and accuracy of color adjustment.

[0068] Figure 3 A schematic diagram illustrating the process of a color adjustment method according to an embodiment of the present disclosure.

[0069] like Figure 3 As shown, in a possible implementation, after reacquiring the camera image, the method further includes:

[0070] Step S25, respectively obtaining the colors of the virtual object and the real object in the camera image;

[0071] When the color difference between the virtual object and the real object in the camera image is greater than the first preset threshold, the color adjustment method and color adjustment parameters of the rendering engine are determined according to the acquired color (step S23) and subsequent steps are re-executed.

[0072] For example, if the color of the first image displayed on the real screen in the actual scene differs significantly from the color of the first image displayed on the real screen in the camera image, or if the determined color adjustment parameters are relatively simple, then the color of the virtual object in the re-acquired camera image may still differ significantly from the color of the real object. In this case, the colors of the virtual object and the real object in the re-acquired camera image can be obtained, and the difference in color between the virtual object and the real object in the camera image can be determined. If the difference is greater than a first preset threshold, it is determined that the color adjustment method and color adjustment parameters need to be iterated further, that is, the steps of determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired colors and subsequent steps are re-executed until the difference in color between the virtual object and the real object in the camera image is less than or equal to the first preset threshold.

[0073] Those skilled in the art should understand that the color difference between the virtual object and the real object in the camera picture can be determined based on the existing technology, and the embodiments of the present disclosure do not limit the specific method of determining the difference. In one example, the color vector of the virtual object (such as the first color vector, the third color vector, and the fourth color vector described below) can be determined by the color of the virtual object in the camera picture, and the color vector of the real object (such as the second color vector, the fifth color vector, and the sixth color vector described below) can be determined by the color of the real object in the camera picture. The color difference between the virtual object and the real object in the camera picture can be determined by the ratio of the first color vector and the second color vector in the same color channel, or the ratio of the third color vector and the fifth color vector in the same color channel, or the ratio of the fourth color vector and the sixth color vector in the same color channel. The determination method of the first, second, third, fourth, fifth, and sixth color vectors and the associated relationship are given later.

[0074] In this way, the accuracy of color adjustment can be guaranteed.

[0075] Furthermore, the colors in the acquired camera image may be nonlinear. Before determining the color vectors of virtual objects and real objects based on the colors in the acquired camera image, the colors in the acquired camera image can be linearly processed to obtain linear colors, and then the linear colors are used to determine the color vectors of virtual objects and real objects to improve the accuracy of the determined color vectors.

[0076] Figure 4 A schematic diagram illustrating the process of a color adjustment method according to an embodiment of the present disclosure.

[0077] like Figure 4 As shown, in a possible implementation, before acquiring the camera image, the method further includes:

[0078] Step S26 , color calibrating the camera according to the first image on the real screen and the first image in the camera picture, so that the color of the first image on the real screen is consistent with the color of the first image in the camera picture.

[0079] For example, there may be two reasons for the color difference between the virtual object displayed on the camera screen and the real object: first, the color of the first image displayed on the real screen in the actual scene is inconsistent with the color of the first image displayed on the real screen in the camera screen, resulting in inconsistent colors between the virtual object on the real screen and the virtual object in the camera screen; second, the color of the real object in the camera screen is inconsistent with the color of the virtual object displayed on the real screen.

[0080] The smaller the color difference between the first image displayed on the real screen in the actual scene and the first image displayed on the real screen in the camera image, the closer the color of the virtual object in the re-acquired camera image will be to the color of the real object. Based on this, before executing step S21 to acquire the camera image, camera calibration can be performed. This can be done by first capturing the real screen with a pre-calibrated camera. Based on the first image on the real screen and the first image in the camera image, the camera is color-calibrated to ensure that the color of the first image on the real screen is consistent with the color of the first image in the camera image. Step S21 can then be executed using the color-calibrated camera.

[0081] Since the color space of the camera and the real screen is nonlinear, when color calibrating the camera, you can first obtain the color of the first image in the camera screen and the color of the first image displayed on the screen, perform linear processing on the obtained colors, and then calibrate the colors after linear processing. For example, assuming that the color of the first image obtained in the camera screen is linearly processed to obtain color A, and the color of the first image displayed on the screen is linearly processed to obtain color B, it can be that color A is mapped to color B. Color calibration can be implemented based on existing technologies. For example, a correspondence between the difference between the first image on the real screen and the first image in the camera screen and the camera parameters can be established in advance, and the camera parameters can be adjusted according to the correspondence to achieve calibration. Among them, the adjustable camera parameters may include white balance parameters, contrast, aperture, shutter speed, color saturation, etc. The specific implementation method of color calibration will not be repeated here.

[0082] On the premise that the camera is color calibrated in advance, the colors of the virtual objects in the re-acquired camera image are closer to the real objects, which can reduce the number of iterations.

[0083] Figure 5 A schematic diagram illustrating the process of a color adjustment method according to an embodiment of the present disclosure.

[0084] like Figure 5As shown, in one possible implementation, the color space of the rendering engine is a linear color space.

[0085] Before determining the color adjustment mode and color adjustment parameters of the rendering engine according to the acquired color, the method further includes:

[0086] Step S27 , performing linear processing on the acquired colors of the virtual object and the real object, and converting the colors after the linear processing into colors in the color space of the rendering engine.

[0087] For example, the first image on a real screen is output to the real screen by a rendering engine. During this process, the colors of the first image in the rendering engine's color space are converted to the colors of the real screen's color space. Therefore, the method for converting the colors in the real screen's color space back to the colors in the rendering engine's color space is also known. Assuming that the camera has been color-calibrated in advance so that the colors of the first image on the real screen are consistent with the colors of the first image in the camera image, the colors of the first image in the camera image can be converted to the rendering engine's color space using the same conversion method. The colors of real objects in the camera image can also be converted to the rendering engine's color space using the same method.

[0088] That is, color_ue_LED_workspace = f(color_ue_LED), color_real_workspace = f(color_real), where color_ue_LED can represent the color of a virtual object after linear processing, and color_real can represent the color of a real object after linear processing. color_ue_LED_workspace can represent the color of a virtual object after conversion to the color space of a rendering engine. color_real_workspace can represent the color of a real object after conversion to the color space of a rendering engine. Linear processing can be based on related technologies to convert colors from nonlinear to linear. Function f() can represent the function used to convert from the color space of a camera screen to the color space of a rendering engine. When the color space of a camera screen and the color space of a rendering engine are of the same type, the function is used to map the numerical range of the color space of the camera screen to the numerical range of the color space of the rendering engine. Function f() can be implemented based on existing technologies and will not be described in detail here.

[0089] Because the camera has been color calibrated in advance, the acquired colors of the virtual object can be converted to the colors in the rendering engine's color space. Assuming that all steps of the color adjustment method are error-free, the color adjustment method and color adjustment parameters determined based on the colors in the converted rendering engine's color space are applied to the rendering engine, resulting in the virtual object in the camera image matching the colors of the real object. In other words, the color adjustment method and color adjustment parameters only need to be determined once to complete the virtual object's color adjustment, thus omitting step S25, further reducing the number of iterations and improving color adjustment efficiency.

[0090] The above steps S25-S27 may also be implemented by a pre-written plug-in of the rendering engine.

[0091] The following describes an exemplary method for determining a color adjustment mode according to an embodiment of the present disclosure.

[0092] In one possible implementation, determining a color adjustment method and color adjustment parameters of a rendering engine according to the acquired color includes:

[0093] Determine the hue distribution of the real object according to the acquired color of the real object;

[0094] Determine the hue distribution of the virtual object according to the obtained color of the virtual object;

[0095] When the difference between the hue distribution of the real object and the hue distribution of the virtual object is greater than a second preset threshold, determining that the color adjustment mode is global color adjustment;

[0096] When the difference between the hue distribution of the real object and the hue distribution of the virtual object is less than or equal to a second preset threshold, the color adjustment mode is determined to be local color adjustment.

[0097] For example, the acquired color of a real object may include a color vector corresponding to the pixels occupied by the real object in the camera image. Based on the acquired color of the real object, the hue distribution of the real object can be determined, that is, the extent to which the color of the real object matches different hues. Similarly, the acquired color of a virtual object may include a color vector corresponding to the pixels occupied by the virtual object in the camera image. Based on the acquired color of the virtual object, the hue distribution of the virtual object can be determined, that is, the extent to which the color of the virtual object matches different hues. Determining the hue distribution can be achieved based on existing technologies, for example, and will not be further described here.

[0098] The difference between the hue distribution of a real object and the hue distribution of a virtual object can be calculated based on existing techniques. In one example, some hues that are hit by a virtual object's color may not be hit by a real object's color, and similarly, some hues that are hit by a real object's color may not be hit by a virtual object's color. The number of these missed hues can be counted, and the ratio of this number to the total number of hues that are hit by the virtual and real objects' colors can be used as the difference between the hue distribution of the real object and the virtual object. The embodiments of this disclosure do not limit the specific method for calculating the difference in hue distribution.

[0099] It is understandable that the greater the difference between the hue distribution of the real object and the hue distribution of the virtual object, the more obvious the color difference between the real object and the virtual object in the camera image will be. Therefore, a second preset threshold can be set according to the application scenario requirements, for example, set to 50%. When the difference between the hue distribution of the real object and the hue distribution of the virtual object is greater than the second preset threshold, it can be confirmed that the color difference between the real object and the virtual object in the camera image is large, and the effect of using global color adjustment is better. The color adjustment method can be determined as global color adjustment, such as PPV. Conversely, when the difference between the hue distribution of the real object and the hue distribution of the virtual object is less than or equal to the second preset threshold, it can be confirmed that the color difference between the real object and the virtual object in the camera image is small, and the effect of using local color adjustment is better. The color adjustment method can be determined as local color adjustment, such as CCR. The embodiments of the present disclosure do not limit the specific value of the second preset threshold.

[0100] Furthermore, after the color adjustment method is determined to be local color adjustment, the area requiring color adjustment can be determined based on the acquired colors of the real object and the virtual object. This step can be implemented based on existing technologies, and the disclosed embodiments do not limit the method for determining the specific area for local color adjustment. For example, the pixel occupied by the real object that causes the hue distribution of the real object and the hue distribution of the virtual object to differ can be found, and the area requiring color adjustment can be determined based on the position of the pixel, so that the determined area can include the pixel.

[0101] In this way, the determined color adjustment method is more adapted to the actual application scenario.

[0102] In an exemplary application scenario, the camera image may also contain multiple real objects and multiple virtual objects, such as virtual object 1 and virtual object 2, or real object 1 and real object 2. Assuming that virtual object 1 corresponds to real object 1, and virtual object 2 corresponds to real object 2, after using the color adjustment method of the embodiment of the present disclosure to make the colors of virtual object 1 and real object 1 consistent in the camera image, if there is a difference in the hue distribution of real object 2 and the hue distribution of virtual object 2, the color adjustment method can be directly determined to be local color adjustment, and the area for local color adjustment can be selected from the area occupied by virtual object 2. In this way, it is possible to avoid the subsequent color adjustment step changing the color of the area that has already been color-adjusted.

[0103] An exemplary method for determining color adjustment parameters according to an embodiment of the present disclosure is described below.

[0104] In one possible implementation, the color adjustment parameters include the gain of each color channel,

[0105] The color adjustment method and color adjustment parameters of the rendering engine are determined based on the obtained color, including:

[0106] Determine a first color vector according to the acquired color of the real object;

[0107] Determining a second color vector according to the acquired color of the virtual object;

[0108] For each color channel, determining a gain of the color channel according to a ratio of the values ​​of the first color vector and the second color vector in the color channel;

[0109] In the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the same color channel in the previously generated first image and the gain of the color channel.

[0110] For example, the rendering engine renders a texture map in its own color space using the asset's textures, lighting, etc., and then uses color adjustment parameters to adjust the color of the texture map. Common color adjustment parameters may include gain. Gain can be used to amplify or reduce the value of a color channel through multiplication to control the brightness and contrast of the image, thereby changing the image color. A larger gain will make the image brighter, while a smaller gain will make the image darker. Since color is usually represented by a color vector composed of the values ​​of three color channels, the color adjustment parameters of the embodiment of the present disclosure may include the gain of each color channel.

[0111] Because the acquired color includes multiple color vectors, a first color vector that best represents the color of the real object can be determined based on the acquired color of the real object. For example, the value of each color channel of the first color vector can be the average of the values ​​of each color vector of the real object in the same color channel. Similarly, a second color vector that best represents the color of the virtual object can be determined based on the acquired color of the virtual object. For example, the value of each color channel of the second color vector can be the average of the values ​​of each color vector of the virtual object in the same color channel.

[0112] Those skilled in the art should understand that, in actual applications, there are more ways to determine the first color vector and the second color vector, such as using the mode of the values ​​of each color channel of the real object as the value of the first color vector in the same color channel, using the mode of the values ​​of each color channel of the virtual object as the value of the second color vector in the same color channel, etc. The embodiments of the present disclosure do not limit the specific method for determining the first color vector and the second color vector.

[0113] Next, for each color channel, the gain of that color channel can be determined based on the ratio of the values ​​of the first color vector and the second color vector in that color channel. That is, Gain = color_1 / color_2. Here, color_1 represents the value of the first color vector in a particular color channel, and color_2 represents the value of the second color vector in the same color channel.

[0114] Those skilled in the art will appreciate that, when the camera is not color calibrated, color_2 may be the result of linear processing of the color in the color space of the camera image. When the camera is color calibrated, color_2 may be the color in the rendering engine's color space converted from the color in the camera image's color space via linear processing.

[0115] In this way, the method of determining the color adjustment parameters is simpler and the efficiency of determining the color adjustment parameters is higher.

[0116] When the color adjustment parameters include gain, the value of any color channel of each pixel whose color changes in the regenerated first image is equal to the product of the value of the same pixel in the same color channel in the previously generated first image and the gain of the color channel. That is, color_out = color_in × Gain, where color_in represents the value of a color in a certain color channel in the color space of the rendering engine, and color_out represents the value of the color channel after the color adjustment. Therefore, after the rendering engine applies the color adjustment parameters, each color rendered in the area to be color adjusted is automatically color-adjusted, causing the color of the regenerated first image to change, and the color of the reacquired camera image also changes accordingly, and the color of the virtual object in the reacquired camera image is closer to the color of the real object.

[0117] Those skilled in the art should understand that each color channel may also use the same gain, for example, the average value of the gains of each color channel may be used as the common gain for each color channel, etc. The embodiments of the present disclosure do not limit the specific calculation method of the gain.

[0118] Another exemplary method for determining color adjustment parameters according to an embodiment of the present disclosure is described below.

[0119] In one possible implementation, the color adjustment parameters include gain and offset.

[0120] Based on the acquired color, determine the color adjustment method and color adjustment parameters of the rendering engine, including:

[0121] Determine a third color vector and a fourth color vector according to the acquired color of the real object;

[0122] Determining a fifth color vector and a sixth color vector according to the acquired color of the virtual object;

[0123] For each color channel, calculating a first difference between the values ​​of the third color vector and the fourth color vector in the color channel, and a second difference between the values ​​of the fifth color vector and the sixth color vector in the color channel, and determining a gain for the color channel based on a ratio of the first difference to the second difference;

[0124] For each color channel, calculate the product of the value of the fifth color vector in the color channel and the gain of the color channel, and determine the offset of the color channel according to the difference between the value of the third color vector in the color channel and the product;

[0125] In the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the same color channel in the previously generated first image and the gain of the color channel and the sum of the offset of the color channel.

[0126] For example, common color adjustment parameters of rendering engines also include offset. Offset is used to control the overall brightness shift of the image and can be achieved by increasing or decreasing the value of the color channel through addition operations. Based on this, the color adjustment parameters of the embodiments of the present disclosure may include gain and offset. Gain has been introduced above and will not be repeated here.

[0127] Because the acquired color includes multiple color vectors, a third color vector and a fourth color vector representing the color of the real object can be determined based on the acquired color of the real object. For example, the first portion of pixels and the second portion of pixels occupied by the real object in the camera image can be determined from the pixels occupied by the real object in the camera image. The value of each color channel of the third color vector can be the average value of the color vector of the first portion of pixels occupied by the real object in the camera image in the same color channel. The value of each color channel of the fourth color vector can be the average value of the color vector of the second portion of pixels occupied by the real object in the camera image in the same color channel.

[0128] Similarly, based on the acquired color of the virtual object, a fifth color vector and a sixth color vector representing the color of the virtual object can be first determined. For example, the first and second portions of pixels occupied by the virtual object in the camera image can be determined from the pixels occupied by the virtual object in the camera image. The first and second portions of pixels can be two portions of pixels in the area where the virtual object is located, adjacent to the area where the real object is located. The first and second portions of pixels can be randomly selected or selected based on the color distribution of the virtual object to represent the primary color of the virtual object. The colors of the first and second portions of pixels can be the same or different. The value of each color channel of the fifth color vector can be the average value of the color vector of the first portion of pixels occupied by the virtual object in the camera image, in the same color channel. The value of each color channel of the sixth color vector can be the average value of the color vector of the second portion of pixels occupied by the virtual object in the camera image, in the same color channel.

[0129] The first portion of pixels occupied by the virtual object in the camera image may be adjacent to the first portion of pixels occupied by the real object in the camera image, and the second portion of pixels occupied by the virtual object in the camera image may be adjacent to the second portion of pixels occupied by the real object in the camera image. Therefore, the third color vector and the fifth color vector may correspond, and the fourth color vector and the sixth color vector may correspond.

[0130] Figure 6 An example of a first portion of pixels and a second portion of pixels occupied by a virtual object and a real object in a camera screen according to an embodiment of the present disclosure is shown.

[0131] like Figure 6 As shown, in the camera image, the curved edge of the real object separates the virtual object from the real object. A first portion of pixels and a second portion of pixels occupied by the virtual object and the real object in the camera image can be selected from both sides of the curved edge.

[0132] Those skilled in the art should understand that in practical applications, there are more ways to determine the third, fourth, fifth, and sixth color vectors. Figure 6 For example, the camera image can be divided into multiple areas according to the horizontal coordinates (or vertical coordinates) of the pixels. Two areas are selected from the areas that simultaneously display virtual objects and real objects. A third color vector is obtained based on the color vectors of the pixels occupied by the real object in the first area, a fourth color vector is obtained based on the color vectors of the pixels occupied by the real object in the second area, a fifth color vector is obtained based on the color vectors of the pixels occupied by the virtual object in the first area, and a sixth color vector is obtained based on the color vectors of the pixels occupied by the virtual object in the second area. The embodiments of the present disclosure do not limit the specific method for determining the third, fourth, fifth, and sixth color vectors.

[0133] Next, for each color channel, a first difference between the values ​​of the third and fourth color vectors in that color channel, as well as a second difference between the values ​​of the fifth and sixth color vectors in that color channel, can be calculated. The gain of that color channel can be determined based on the ratio of the first and second differences. That is, Gain = (color_1_A - color_1_B) / (color_2_A - color_2_B). Here, color_1_A represents the value of the third color vector in a particular color channel, color_1_B represents the value of the fourth color vector in the same color channel, color_2_A represents the value of the fifth color vector in the same color channel, and color_2_B represents the value of the sixth color vector in the same color channel.

[0134] For each color channel, the product of the value of the fifth color vector in that color channel and the gain of that color channel can be calculated. The offset of that color channel is determined based on the difference between the value of the third color vector in that color channel and the product, that is, Offset = color_1_A - color_2_A × Gain.

[0135] Those skilled in the art will appreciate that, when the camera is not color calibrated, color_2_A and color_2_B may be the result of linear processing of the colors in the camera image's color space. When the camera is color calibrated, color_2_A and color_2_B may be the result of linear conversion of the colors in the camera image's color space to the rendering engine's color space.

[0136] In this way, the method of determining the color adjustment parameters is simpler and the efficiency of determining the color adjustment parameters is higher.

[0137] When the color adjustment parameters include gain and offset, the value of any color channel of each pixel whose color changes in the regenerated first image is equal to the product of the value of the same pixel in the same color channel in the previously generated first image and the gain of that color channel, plus the offset of that color channel. That is, color_out = color_in × Gain + Offset, where color_in represents the value of a color in a certain color channel in the rendering engine's color space, and color_out represents the value of that color channel after the color adjustment. Therefore, after the rendering engine applies the color adjustment parameters, each color rendered in the area to be color adjusted is automatically color-adjusted, causing the color of the regenerated first image to change, and the color of the reacquired camera image also changes accordingly, and the color of the virtual object in the reacquired camera image is closer to the color of the real object.

[0138] Those skilled in the art should understand that each color channel may also use the same gain and offset, for example, the average value of the gain of each color channel may be used as the common gain for each color channel, the average value of the offset of each color channel may be used as the common offset for each color channel, etc. The embodiments of the present disclosure do not limit the specific calculation method of the gain and offset.

[0139] The present disclosure also provides a color adjustment device. Figure 7 A schematic diagram illustrating the structure of a color adjustment device according to an embodiment of the present disclosure.

[0140] like Figure 7 As shown, in a possible implementation, the color adjustment device includes:

[0141] A first acquisition module 71 is configured to acquire a camera image, wherein the camera image includes a real object and a virtual object, wherein the virtual object is an object included in a first image displayed on a real screen, the first image being generated by a rendering engine, and the virtual object corresponds to the real object;

[0142] A second acquisition module 72 is used to respectively acquire the colors of the virtual object and the real object in the camera image;

[0143] A first determining module 73 is configured to determine a color adjustment method and color adjustment parameters of the rendering engine according to the acquired color, wherein the color adjustment method is related to a degree of color difference between the virtual object and the real object in the camera image;

[0144] The first generating module 74 is configured to enable the rendering engine to regenerate the first image using the color adjustment method and the color adjustment parameters, and enable the first acquiring module to reacquire the camera image when the real screen displays the new first image.

[0145] In one possible implementation, the device further includes: a third acquisition module, configured to respectively acquire the colors of the virtual object and the real object in the camera image after the first acquisition module reacquires the camera image; the device is further configured to, when the difference in color between the virtual object and the real object in the camera image is greater than a first preset threshold, re-execute the steps of determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired colors and subsequent steps.

[0146] In one possible implementation, the device further includes: a calibration module, configured to perform color calibration on the camera based on the first image on the real screen and the first image in the camera screen before the first acquisition module acquires the camera screen, so that the color of the first image on the real screen is consistent with the color of the first image in the camera screen.

[0147] In one possible implementation, the color space of the rendering engine is a linear color space, and the device further includes: a conversion module, configured to perform linear processing on the acquired colors of the virtual object and the real object before the first determination module determines the color adjustment method and color adjustment parameters of the rendering engine based on the acquired colors, and convert the linearly processed colors into colors in the color space of the rendering engine.

[0148] In one possible implementation, the color adjustment parameters include the gain of each color channel, and determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired color includes: determining a first color vector based on the acquired color of the real object; determining a second color vector based on the acquired color of the virtual object; for each color channel, determining the gain of the color channel based on the ratio of the values ​​of the first color vector and the second color vector in the color channel; in the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the same color channel in the previously generated first image and the gain of the color channel.

[0149] In one possible implementation, the color adjustment parameters include a gain and an offset. Determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired color includes: determining a third color vector and a fourth color vector based on the acquired color of the real object; determining a fifth color vector and a sixth color vector based on the acquired color of the virtual object; calculating, for each color channel, a first difference between the values ​​of the third color vector and the fourth color vector in the color channel, and a second difference between the values ​​of the fifth color vector and the sixth color vector in the color channel, and determining the gain of the color channel based on a ratio of the first difference to the second difference; calculating, for each color channel, a product of the value of the fifth color vector in the color channel and the gain of the color channel, and determining the offset of the color channel based on the difference between the value of the third color vector in the color channel and the product; and in the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the previously generated first image in the same color channel and the gain of the color channel, and then adding the product to the offset of the color channel.

[0150] In a possible implementation, determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired color includes: determining the hue distribution of the real object based on the acquired color of the real object; determining the hue distribution of the virtual object based on the acquired color of the virtual object; when the difference between the hue distribution of the real object and the hue distribution of the virtual object is greater than a second preset threshold, determining that the color adjustment method is global color adjustment; when the difference between the hue distribution of the real object and the hue distribution of the virtual object is less than or equal to the second preset threshold, determining that the color adjustment method is local color adjustment.

[0151] In a possible implementation, the device is a plug-in of a rendering engine.

[0152] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0153] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0154] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0155] An embodiment of the present disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0156] Figure 8 FIG1 shows a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server or a terminal device. Figure 8 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above-described method.

[0157] The electronic device 1900 may further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2003. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0158] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0159] The present disclosure may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0160] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0161] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0162] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions. The electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0163] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0164] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0165] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0166] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0167] While various embodiments of the present disclosure have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A color adjustment method, characterized in that: The method comprises: Performing color calibration on the camera according to the first image on the real screen and the first image in the camera picture so that the color of the first image on the real screen is consistent with the color of the first image in the camera picture; Acquire the camera image, where the camera image includes a real object and a virtual object, where the virtual object is an object included in a first image displayed on a real screen, where the first image is generated by a rendering engine, and the virtual object corresponds to the real object; respectively obtaining the colors of the virtual object and the real object in the camera image; performing linear processing on the acquired colors of the virtual object and the real object, converting the linearly processed colors into colors in the color space of the rendering engine according to a method of converting colors in the color space of the real screen back into colors in the color space of the rendering engine, wherein the color space of the rendering engine is a linear color space; Determining a color adjustment method and color adjustment parameters of the rendering engine according to the acquired color, wherein the color adjustment method is related to a degree of color difference between the virtual object and the real object in the camera image; The rendering engine is enabled to regenerate the first image using the color adjustment method and the color adjustment parameters, and the camera image is reacquired when the real screen displays the new first image.

2. The method according to claim 1, characterized in that After reacquiring the camera image, the method further includes: respectively obtaining the colors of the virtual object and the real object in the camera image; When the color difference between the virtual object and the real object in the camera image is greater than a first preset threshold, re-execute the steps of determining the color adjustment method and color adjustment parameters of the rendering engine according to the acquired color and subsequent steps.

3. The method according to claim 1 or 2, characterized in that The color adjustment parameters include the gain of each color channel, The determining of the color adjustment method and color adjustment parameters of the rendering engine according to the acquired color includes: Determining a first color vector according to the acquired color of the real object; Determining a second color vector according to the acquired color of the virtual object; For each color channel, determining a gain of the color channel according to a ratio of values ​​of the first color vector and the second color vector in the color channel; In the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the same color channel in the previously generated first image and the gain of the color channel.

4. The method according to claim 1 or 2, characterized in that The color adjustment parameters include gain and offset, The determining of the color adjustment method and color adjustment parameters of the rendering engine according to the acquired color includes: determining a third color vector and a fourth color vector according to the acquired color of the real object; determining a fifth color vector and a sixth color vector according to the acquired color of the virtual object; For each color channel, calculating a first difference between the values ​​of the third color vector and the fourth color vector in the color channel, and a second difference between the values ​​of the fifth color vector and the sixth color vector in the color channel, and determining a gain for the color channel based on a ratio of the first difference to the second difference; For each color channel, calculate the product of the value of the fifth color vector in the color channel and the gain of the color channel, and determine the offset of the color channel according to the difference between the value of the third color vector in the color channel and the product; In the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the same color channel in the previously generated first image and the gain of the color channel and the sum of the offset of the color channel.

5. The method according to claim 1 or 2, characterized in that The determining of the color adjustment method and color adjustment parameters of the rendering engine according to the acquired color includes: determining a hue distribution of the real object according to the acquired color of the real object; determining a hue distribution of the virtual object according to the acquired color of the virtual object; When the difference between the hue distribution of the real object and the hue distribution of the virtual object is greater than a second preset threshold, determining that the color adjustment mode is global color adjustment; When the difference between the hue distribution of the real object and the hue distribution of the virtual object is less than or equal to a second preset threshold, it is determined that the color adjustment mode is local color adjustment.

6. The method according to claim 1, characterized in that The method is executed by a plug-in of a rendering engine.

7. A color adjustment device, characterized in that: The device comprises: a calibration module, configured to perform color calibration on the camera according to the first image on the real screen and the first image in the camera picture, so that the color of the first image on the real screen is consistent with the color of the first image in the camera picture; A first acquisition module is configured to acquire the camera image, wherein the camera image includes a real object and a virtual object, wherein the virtual object is an object included in a first image displayed on a real screen, the first image being generated by a rendering engine, and the virtual object corresponds to the real object; A second acquisition module is used to respectively acquire the colors of the virtual object and the real object in the camera image; a conversion module, configured to perform linear processing on the acquired colors of the virtual object and the real object, converting the linearly processed colors into colors in the color space of the rendering engine according to a method of converting colors in the color space of the real screen back into colors in the color space of the rendering engine, wherein the color space of the rendering engine is a linear color space; a first determining module, configured to determine a color adjustment method and color adjustment parameters of the rendering engine according to the acquired color, wherein the color adjustment method is related to a degree of color difference between the virtual object and the real object in the camera image; The first generating module is configured to enable the rendering engine to regenerate the first image using the color adjustment method and the color adjustment parameters, and enable the first acquiring module to reacquire the camera image when the real screen displays the new first image.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method according to any one of claims 1 to 6 when executing the instructions stored in the memory.

9. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising computer readable code, characterized in that When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the method according to any one of claims 1 to 6.

Images