Color adjustment method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to the field of virtual shooting, and provides a color adjustment method and device, electronic equipment, a storage medium and a program product. The method comprises the steps that a camera picture is obtained, the camera picture comprises a real object and a virtual object, the virtual object is an object included in a first image displayed on a real screen, the first image is generated by a rendering engine, and the virtual object corresponds to the real object; respectively acquiring colors of the virtual object and the real object in a camera picture; determining a color adjustment mode and a color adjustment parameter of the rendering engine according to the obtained color; and enabling the rendering engine to regenerate the first image by using the color adjustment mode and the color adjustment parameter, and obtaining the camera picture again when a new first image is displayed on the real screen. According to the color adjustment method, automatic color obtaining is supported, the color adjustment mode and the color adjustment parameters are automatically determined, the labor cost of color adjustment is reduced, and the efficiency and accuracy of color adjustment are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of virtual photography, and in particular to a color adjustment method, device, electronic device, storage medium and program product. Background Art

[0002] Virtual shooting is a technical method used in film, television, commercial and other film and television production. A common virtual shooting scheme is to combine the virtual scene generated by the rendering engine on the computer with the real scene for shooting. For example, a screen is placed in the real scene, and an image including a virtual object is generated by the rendering engine and displayed on the screen, and a real object corresponding to the virtual object is placed in front of the screen. In this way, the virtual object and the real object can appear at the same time in the camera screen.

[0003] Ideally, the colors of virtual objects and real objects in the camera screen should be consistent. However, in the actual camera screen, the virtual objects may differ from the real objects in color, resulting in a sense of disconnection between the virtual objects and the real objects in the camera screen. In order to improve the shooting effect, it is necessary to make the colors of virtual objects and real objects in the camera screen as close as possible. In the prior art, the color of the object in the camera screen is usually obtained manually, and the color adjustment method and color adjustment parameters are determined. On the one hand, this increases the labor cost of color adjustment, and on the other hand, due to the poor accuracy of manual operation, it also reduces the efficiency and accuracy of color adjustment. Summary of the invention

[0004] In view of this, the present disclosure proposes a color adjustment method, device, electronic device, storage medium and program product. The color adjustment method of the embodiment of the present disclosure supports automatic acquisition of color, automatic determination of color adjustment mode and color adjustment parameters, reduces the labor cost of color adjustment, and improves the efficiency and accuracy of color adjustment.

[0005] According to one aspect of the present disclosure, a color adjustment method is provided, the method comprising: acquiring a camera screen, the camera screen comprising a real object and a virtual object, the virtual object being an object included in a first image displayed on a real screen, the first image being generated by a rendering engine, and the virtual object corresponding to the real object; respectively acquiring the colors of the virtual object and the real object in the camera screen; determining a color adjustment method and color adjustment parameters of the rendering engine according to the acquired colors, wherein the color adjustment method is related to the degree of color difference between the virtual object and the real object in the camera screen; causing the rendering engine to regenerate the first image using the color adjustment method and the color adjustment parameters, and reacquiring the camera screen when the real screen displays a new first image.

[0006] In a possible implementation, after reacquiring the camera image, the method further includes: respectively acquiring the colors of the virtual object and the real object in the camera image; when the difference in color between the virtual object and the real object in the camera image is greater than a first preset threshold, re-executing the steps of determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired colors and subsequent steps.

[0007] In a possible implementation, before acquiring the camera screen, the method further includes: performing color calibration on the camera based on the first image on the real screen and the first image in the camera screen, so that the color of the first image on the real screen is consistent with the color of the first image in the camera screen.

[0008] In a possible implementation, the color space of the rendering engine is a linear color space. Before determining the color adjustment method and color adjustment parameters of the rendering engine according to the acquired color, the method further includes: performing linear processing on the acquired color of the virtual object and the acquired color of the real object, and converting the linearly processed color into the color of the color space of the rendering engine.

[0009] In a possible implementation, the color adjustment parameters include the gain of each color channel, and determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired color includes: determining a first color vector based on the acquired color of the real object; determining a second color vector based on the acquired color of the virtual object; for each color channel, determining the gain of the color channel based on the ratio of the values ​​of the first color vector and the second color vector in the color channel; in the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the same color channel in the previously generated first image and the gain of the color channel.

[0010] In a possible implementation, the color adjustment parameters include gain and offset, and the color adjustment method and color adjustment parameters of the rendering engine are determined according to the acquired color, including: determining a third color vector and a fourth color vector according to the acquired color of the real object; determining a fifth color vector and a sixth color vector according to the acquired color of the virtual object; for each color channel, calculating a first difference between the values ​​of the third color vector and the fourth color vector in the color channel, and a second difference between the values ​​of the fifth color vector and the sixth color vector in the color channel, and determining the gain of the color channel according to a ratio of the first difference to the second difference; for each color channel, calculating the product of the value of the fifth color vector in the color channel and the gain of the color channel, and determining the offset of the color channel according to the difference between the value of the third color vector in the color channel and the product; in the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the same color channel in the first image generated last time and the gain of the color channel plus the offset of the color channel.

[0011] In a possible implementation, determining the color adjustment method and color adjustment parameters of the rendering engine according to the acquired color includes: determining the hue distribution of the real object according to the acquired color of the real object; determining the hue distribution of the virtual object according to the acquired color of the virtual object; when the difference between the hue distribution of the real object and the hue distribution of the virtual object is greater than a second preset threshold, determining that the color adjustment method is global color adjustment; when the difference between the hue distribution of the real object and the hue distribution of the virtual object is less than or equal to the second preset threshold, determining that the color adjustment method is local color adjustment.

[0012] In a possible implementation, the method is executed by a plug-in of a rendering engine.

[0013] According to another aspect of the present disclosure, a color adjustment device is provided, the device comprising: a first acquisition module, used to acquire a camera screen, the camera screen comprising a real object and a virtual object, the virtual object being an object included in a first image displayed on a real screen, the first image being generated by a rendering engine, and the virtual object corresponding to the real object; a second acquisition module, used to respectively acquire the colors of the virtual object and the real object in the camera screen; a first determination module, used to determine a color adjustment method and a color adjustment parameter of the rendering engine according to the acquired colors, wherein the color adjustment method is related to the degree of color difference between the virtual object and the real object in the camera screen; and a first generation module, used to enable the rendering engine to regenerate the first image using the color adjustment method and the color adjustment parameter, and when the real screen displays a new first image, enable the first acquisition module to reacquire the camera screen.

[0014] In a possible implementation, the device also includes: a third acquisition module, which is used to respectively acquire the colors of the virtual object and the real object in the camera screen after the first acquisition module re-acquires the camera screen; the device is also used to, when the difference in color between the virtual object and the real object in the camera screen is greater than a first preset threshold, re-execute the steps of determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired color and subsequent steps.

[0015] In a possible implementation, the device also includes: a calibration module, which is used to perform color calibration on the camera based on the first image on the real screen and the first image in the camera screen before the first acquisition module acquires the camera screen, so that the color of the first image on the real screen is consistent with the color of the first image in the camera screen.

[0016] In a possible implementation, the color space of the rendering engine is a linear color space, and the device further includes: a conversion module, configured to perform linear processing on the acquired colors of the virtual object and the real object before the first determination module determines the color adjustment method and color adjustment parameters of the rendering engine according to the acquired colors, and convert the linearly processed colors into colors in the color space of the rendering engine.

[0017] In a possible implementation, the color adjustment parameters include the gain of each color channel, and determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired color includes: determining a first color vector based on the acquired color of the real object; determining a second color vector based on the acquired color of the virtual object; for each color channel, determining the gain of the color channel based on the ratio of the values ​​of the first color vector and the second color vector in the color channel; in the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the same color channel in the previously generated first image and the gain of the color channel.

[0018] In a possible implementation, the color adjustment parameters include gain and offset, and the color adjustment method and color adjustment parameters of the rendering engine are determined according to the acquired color, including: determining a third color vector and a fourth color vector according to the acquired color of the real object; determining a fifth color vector and a sixth color vector according to the acquired color of the virtual object; for each color channel, calculating a first difference between the values ​​of the third color vector and the fourth color vector in the color channel, and a second difference between the values ​​of the fifth color vector and the sixth color vector in the color channel, and determining the gain of the color channel according to a ratio of the first difference to the second difference; for each color channel, calculating the product of the value of the fifth color vector in the color channel and the gain of the color channel, and determining the offset of the color channel according to the difference between the value of the third color vector in the color channel and the product; in the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the same color channel in the first image generated last time and the gain of the color channel plus the offset of the color channel.

[0019] In a possible implementation, determining the color adjustment method and color adjustment parameters of the rendering engine according to the acquired color includes: determining the hue distribution of the real object according to the acquired color of the real object; determining the hue distribution of the virtual object according to the acquired color of the virtual object; when the difference between the hue distribution of the real object and the hue distribution of the virtual object is greater than a second preset threshold, determining that the color adjustment method is global color adjustment; when the difference between the hue distribution of the real object and the hue distribution of the virtual object is less than or equal to the second preset threshold, determining that the color adjustment method is local color adjustment.

[0020] In a possible implementation, the device is a plug-in of a rendering engine.

[0021] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0022] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0023] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0024] According to the color adjustment method of the embodiment of the present disclosure, a camera screen is acquired, the camera screen includes a real object and a virtual object, the virtual object is an object included in a first image displayed on a real screen, the first image is generated by a rendering engine, and the virtual object corresponds to the real object, so that the virtual object in the camera screen can be used as an object to be color adjusted, and the real object in the camera screen can be used as a color adjustment standard; the colors of the virtual object and the real object in the camera screen are acquired respectively, and a color adjustment method and a color adjustment parameter of the rendering engine are determined according to the acquired colors. Since the color adjustment method and the color adjustment parameter are determined according to the colors of the virtual object and the real object in the camera screen, the color adjustment method and the color adjustment parameter can make the color of the virtual object in the camera screen closer to the color of the real object; the color adjustment method is related to the degree of color difference between the virtual object and the real object in the camera screen, so the determined color adjustment method is more matched with the actual application scenario; the rendering engine uses the color adjustment method and the color adjustment parameter to regenerate the first image, and the first image will automatically complete the color adjustment without having to perform color adjustment pixel by pixel. When the new first image is displayed on the real screen, the colors of the virtual object and the real object in the re-acquired camera screen will be closer. In summary, the color adjustment method of the embodiment of the present disclosure supports automatic acquisition of color, automatic determination of color adjustment mode and color adjustment parameters, reduces the labor cost of color adjustment, and improves the efficiency and accuracy of color adjustment.

[0025] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0027] Figure 1 An exemplary application scenario of the color adjustment method according to an embodiment of the present disclosure is shown.

[0028] Figure 2 A schematic diagram showing the process of a color adjustment method according to an embodiment of the present disclosure.

[0029] Figure 3 A schematic diagram showing the process of a color adjustment method according to an embodiment of the present disclosure.

[0030] Figure 4 A schematic diagram showing the process of a color adjustment method according to an embodiment of the present disclosure.

[0031] Figure 5 A schematic diagram showing the process of a color adjustment method according to an embodiment of the present disclosure.

[0032] Figure 6 An example of a first portion of pixels and a second portion of pixels occupied by a virtual object and a real object in a camera screen according to an embodiment of the present disclosure is shown.

[0033] Figure 7 A schematic diagram showing the structure of a color adjustment device according to an embodiment of the present disclosure.

[0034] Figure 8 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0035] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0036] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0037] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0038] The following is an introduction to the terms that may appear in this article.

[0039] Virtual filming: refers to the process of creating and capturing real-time images and videos using digital technology, game engines (such as Unreal Engine or Unity), and technologies such as virtual reality or augmented reality.

[0040] Color space: A mathematical model for describing and representing color that, when used in conjunction with tests on physical devices, results in fixed analog and digital representations of color.

[0041] Color calibration: In virtual shooting, color calibration means that when the camera captures the screen, the on-screen image displayed by the camera screen is consistent with the actual color of the image displayed on the screen.

[0042] Post-Processing Volume (PPV): is a tool used to control the visual effects after rendering. It can apply various visual effects such as color correction, blur, depth of field and halo to specific areas in the game scene, and is often used for global color adjustment.

[0043] Color Correction (CCR): It is an important post-processing effect used to adjust the color and brightness in a specific area to make it consistent with the rest of the scene or stand out in a specific visual style. It is often used for local color adjustment.

[0044] Plugin: A carrier of customizable functions in the rendering engine that can be applied to other projects.

[0045] Figure 1 An exemplary application scenario of the color adjustment method according to an embodiment of the present disclosure is shown.

[0046] like Figure 1 As shown, in an exemplary application scenario, a first device, a camera, a screen, and a real object may be included. A camera generally refers to a device with the ability to capture images, such as a video camera. The camera image can be displayed on the first device. The first device may be provided with a rendering engine (not shown), such as Unreal Engine (UE). The first device may use the rendering engine to render an image including a virtual object, and output it to the screen for display. Virtual objects may correspond to real objects. For example, when the real object is a beach, the virtual object may also be a beach. The embodiments of the present disclosure do not limit the specific types of virtual objects and real objects.

[0047] The types of color spaces of the rendering engine, screen, and camera image may be the same, for example, all are RGB three-channel color spaces, but the numerical ranges of the same color channel in the above three color spaces may be different. For example, in the color space of the rendering engine, the numerical range of red may be 0-100, in the color space of the screen, the numerical range of red may be 0-70, and in the color space of the camera, the numerical range of red may be 0-50. The disclosed embodiments do not limit the type of each color space or the numerical range of each color channel.

[0048] To ensure the accuracy of rendering, the color space of the rendering engine can be set to a linear color space. In a linear color space, the color component is linearly related to the physical intensity of light, and the mixing and transformation of colors follow the rules of linear algebra, which makes color processing and calculation more intuitive and convenient. The display objects of the screen and camera are humans. In order to adapt to human visual sensitivity, the color space of the real screen and the color space of the camera screen can be set to a nonlinear color space.

[0049] The staff can adjust the camera's viewing angle and the position of the real object as needed, so that the virtual object and the real object appear in the camera screen at the same time. Next, the first device can execute the color adjustment method of the embodiment of the present disclosure to determine the color adjustment mode and color adjustment parameters of the rendering engine. After the image rendered by the rendering engine under the new color adjustment mode and color adjustment parameters is output to the screen for display, the color of the virtual object in the camera screen is closer to the color of the real object, and finally the color of the content on the camera screen is adjusted.

[0050] Those skilled in the art should understand that, in actual applications, the rendering function and the color adjustment function may also be implemented by different electronic devices. For example, the second device (not shown) may be used to implement the rendering function, and the first device may be used to implement the color adjustment function. The embodiments of the present disclosure do not limit the specific functions of the first device.

[0051] Figure 2 A schematic diagram showing the process of a color adjustment method according to an embodiment of the present disclosure.

[0052] like Figure 2 As shown, in a possible implementation, the method includes:

[0053] Step S21, acquiring a camera image, the camera image including real objects and virtual objects, the virtual objects being objects included in a first image displayed on a real screen, the first image being generated by a rendering engine, and the virtual objects corresponding to real objects;

[0054] Step S22, respectively obtaining the colors of the virtual object and the real object in the camera image;

[0055] Step S23, determining a color adjustment method and color adjustment parameters of a rendering engine according to the acquired color, wherein the color adjustment method is related to the degree of color difference between the virtual object and the real object in the camera image;

[0056] Step S24, enabling the rendering engine to regenerate the first image using the color adjustment method and the color adjustment parameters, and reacquiring the camera image when the new first image is displayed on the real screen.

[0057] For example, when a first image generated by a rendering engine has been displayed on a real screen, and the first image includes a virtual object, step S21 can be executed to obtain a camera screen. The camera screen may include real objects and virtual objects. Virtual objects correspond to real objects. For example, virtual objects and real objects may be objects of the same type whose colors need to be aligned. Virtual objects and real objects with a corresponding relationship that need to be aligned in color can be determined by object recognition and other related technologies, such as identifying virtual objects and real objects of the same type and adjacent in position, or virtual objects and real objects that need to be aligned in color can be manually selected. Due to the viewing angle of the camera, the camera screen may include all or part of the real object, as well as all or part of the virtual object. The disclosed embodiment does not limit whether the real objects and virtual objects included in the camera screen are complete.

[0058] Next, step S22 may be executed to obtain the colors of the virtual object and the real object in the camera screen respectively. Taking obtaining the color of the virtual object as an example, assuming that the color space of the camera screen is a RGB three-channel color space, the virtual object occupies multiple pixels in the camera screen, and a certain number of pixels may be selected therefrom to obtain the values ​​of the selected pixels in the three color channels. According to the obtained values ​​of each pixel in the three color channels, a color vector may be obtained. The method for obtaining the color of the real object may be the same as the method for obtaining the color of the virtual object, which will not be repeated here. The details of obtaining the colors of the virtual object and the real object in the camera screen are given later.

[0059] The disclosed embodiments do not limit the method of selecting pixels. The pixels may be selected manually or certain selection conditions may be set. A deep learning algorithm may be used to identify pixels that meet the conditions and selection may be made based on the identification results, as long as the selected pixels are the pixels occupied by the object whose color is to be obtained in the camera image.

[0060] Those skilled in the art should understand that the acquired color may also be represented in forms other than color vectors, as long as the acquired color can represent the actual color of the object whose color is to be acquired in the camera image. The embodiments of the present disclosure do not limit the specific form of the acquired color.

[0061] After obtaining the colors of the virtual object and the real object in the camera screen, step S23 may be executed to determine the color adjustment method and color adjustment parameters of the rendering engine according to the obtained colors. The color adjustment method and color adjustment parameters are used to adjust the color of the image output by the rendering engine, so that the color of the image displayed on the real screen also changes accordingly, and finally the color of the virtual object in the camera screen is closer to the color of the real object.

[0062] Exemplarily, the rendering engine may support multiple color adjustment methods, such as the post-processing volume PPV and regional color correction CCR described above. The determined color adjustment method may be related to the degree of color difference between the virtual object and the real object in the camera image. The determined color adjustment method may be one of the multiple color adjustment methods supported by the rendering engine. The rendering engine may use at least one color adjustment parameter, such as gain (Gain) and offset (Offset), etc. in each color adjustment method. The embodiment of the present disclosure does not limit the specific type of the color adjustment method determined in step S23 and the specific type of the color adjustment parameter. The details of determining the color adjustment method and color adjustment parameters of the rendering engine are given later.

[0063] Furthermore, the precondition for determining the color adjustment method and color adjustment parameters of the rendering engine may be that the color difference between the virtual object and the real object in the camera image is large, for example, greater than a first preset threshold. The first preset threshold may be a boundary value at which the color difference between the virtual object and the real object meets the requirements of the application scenario. By adding this precondition, the data processing cost can be further reduced. The value of the first preset threshold can be set according to the requirements of the application scenario, and the embodiment of the present disclosure does not limit the specific value of the first preset threshold.

[0064] After determining the color adjustment method and color adjustment parameters of the rendering engine, step S24 can be performed to enable the rendering engine to regenerate the first image using the determined color adjustment method and color adjustment parameters. The regenerated first image may have the same content as the original first image, and the difference may be the color of the image.

[0065] The rendering engine may output the regenerated first image to the real screen for display. When the real screen displays the new first image, the camera picture may be reacquired. In this case, the color of the virtual object in the reacquired camera picture may be closer to the color of the real object.

[0066] Among them, step S23 and step S24 can be implemented by a pre-written plug-in of the rendering engine. Before executing step S23, the plug-in is started, and step S23 and step S24 can be automatically executed. Steps S21 and S22 can also be implemented by the plug-in (starting the plug-in before executing step S21), or directly implemented by the processor, which is not limited in the embodiments of the present disclosure.

[0067] According to the color adjustment method of the embodiment of the present disclosure, a camera screen is acquired, the camera screen includes a real object and a virtual object, the virtual object is an object included in a first image displayed on a real screen, the first image is generated by a rendering engine, and the virtual object corresponds to the real object, so that the virtual object in the camera screen can be used as an object to be color adjusted, and the real object in the camera screen can be used as a color adjustment standard; the colors of the virtual object and the real object in the camera screen are acquired respectively, and a color adjustment method and a color adjustment parameter of the rendering engine are determined according to the acquired colors. Since the color adjustment method and the color adjustment parameter are determined according to the colors of the virtual object and the real object in the camera screen, the color adjustment method and the color adjustment parameter can make the color of the virtual object in the camera screen closer to the color of the real object; the color adjustment method is related to the degree of color difference between the virtual object and the real object in the camera screen, so the determined color adjustment method is more matched with the actual application scenario; the rendering engine uses the color adjustment method and the color adjustment parameter to regenerate the first image, and the first image will automatically complete the color adjustment without having to perform color adjustment pixel by pixel. When the new first image is displayed on the real screen, the colors of the virtual object and the real object in the re-acquired camera screen will be closer. In summary, the color adjustment method of the embodiment of the present disclosure supports automatic acquisition of color, automatic determination of color adjustment mode and color adjustment parameters, reduces the labor cost of color adjustment, and improves the efficiency and accuracy of color adjustment.

[0068] Figure 3 A schematic diagram showing the process of a color adjustment method according to an embodiment of the present disclosure.

[0069] like Figure 3 As shown, in a possible implementation, after reacquiring the camera image, the method further includes:

[0070] Step S25, respectively obtaining the colors of the virtual object and the real object in the camera image;

[0071] When the color difference between the virtual object and the real object in the camera picture is greater than the first preset threshold, the color adjustment method and color adjustment parameters of the rendering engine are determined according to the acquired color (step S23) and subsequent steps are re-executed.

[0072] For example, if the color of the first image displayed on the real screen in the actual scene is greatly different from the color of the first image displayed on the real screen in the camera picture, or the determined color adjustment parameters are relatively simple, then in the reacquired camera picture, there may still be a large difference between the color of the virtual object and the color of the real object. In this case, the colors of the virtual object and the real object in the reacquired camera picture can be obtained, and the difference in color between the virtual object and the real object in the camera picture can be determined. If the difference is greater than the first preset threshold, it is determined that the color adjustment method and the color adjustment parameters need to be iterated, that is, the color adjustment method and the color adjustment parameters of the rendering engine are determined according to the acquired color and the subsequent steps are re-executed until the color difference between the virtual object and the real object in the camera picture is less than or equal to the first preset threshold.

[0073] Those skilled in the art should understand that the difference in color between the virtual object and the real object in the camera screen can be determined based on the prior art, and the specific method for determining the difference is not limited in the embodiments of the present disclosure. In one example, the color vector of the virtual object (such as the first color vector, the third color vector, and the fourth color vector described below) can be determined by the color of the virtual object in the camera screen, and the color vector of the real object (such as the second color vector, the fifth color vector, and the sixth color vector described below) can be determined by the color of the real object in the camera screen. The difference in color between the virtual object and the real object in the camera screen can be determined by the ratio of the first color vector and the second color vector in the same color channel, or the ratio of the third color vector and the fifth color vector in the same color channel, or the ratio of the fourth color vector and the sixth color vector in the same color channel. The determination method and the associated relationship of the first, second, third, fourth, fifth, and sixth color vectors are given later.

[0074] In this way, the accuracy of color adjustment can be guaranteed.

[0075] Furthermore, the colors in the acquired camera image may be nonlinear. Before determining the color vectors of virtual objects and real objects based on the colors in the acquired camera image, the colors in the acquired camera image can be linearly processed to obtain linear colors, and then the linear colors are used to determine the color vectors of virtual objects and real objects to improve the accuracy of the determined color vectors.

[0076] Figure 4 A schematic diagram showing the process of a color adjustment method according to an embodiment of the present disclosure.

[0077] like Figure 4 As shown, in a possible implementation, before acquiring the camera image, the method further includes:

[0078] Step S26, color calibration is performed on the camera according to the first image on the real screen and the first image in the camera picture, so that the color of the first image on the real screen is consistent with the color of the first image in the camera picture.

[0079] For example, there may be two reasons why there is a color difference between the virtual object displayed on the camera screen and the real object: first, the color of the first image displayed on the real screen in the actual scene is inconsistent with the color of the first image displayed on the real screen in the camera screen, resulting in inconsistent colors between the virtual object on the real screen and the virtual object in the camera screen; second, the color of the real object in the camera screen is inconsistent with the color of the virtual object displayed on the real screen.

[0080] The smaller the color difference between the first image displayed on the real screen in the actual scene and the first image displayed on the real screen in the camera screen, the closer the color of the virtual object and the color of the real object in the re-acquired camera screen are. Based on this, before executing step S21 to acquire the camera screen, the camera calibration can be performed first. The real screen can be photographed with a camera before calibration, and the camera can be color calibrated according to the first image on the real screen and the first image in the camera screen, so that the color of the first image on the real screen is consistent with the color of the first image in the camera screen. Then, the color-calibrated camera is used to execute step S21.

[0081] Since the color space of the camera and the real screen is nonlinear, when the camera is color calibrated, the color of the first image in the camera screen and the color of the first image displayed on the screen can be first obtained, the obtained colors can be linearly processed, and then the colors after linear processing can be calibrated. For example, assuming that the color of the first image obtained in the camera screen is linearly processed to obtain color A, and the color of the first image displayed on the screen is linearly processed to obtain color B, color A can be mapped to color B. Color calibration can be implemented based on existing technologies. For example, a correspondence between the difference between the first image on the real screen and the first image in the camera screen and the camera parameters can be established in advance, and the camera parameters can be adjusted according to the correspondence to achieve calibration. Among them, the adjustable camera parameters may include white balance parameters, contrast, aperture, shutter speed, color saturation, etc. The specific implementation method of color calibration will not be repeated here.

[0082] On the premise that the camera is color calibrated in advance, the colors of virtual objects in the reacquired camera image are closer to those of real objects, which can reduce the number of iterations.

[0083] Figure 5 A schematic diagram showing the process of a color adjustment method according to an embodiment of the present disclosure.

[0084] like Figure 5As shown, in a possible implementation, the color space of the rendering engine is a linear color space.

[0085] Before determining the color adjustment mode and color adjustment parameters of the rendering engine according to the acquired color, the method further includes:

[0086] Step S27, linear processing is performed on the acquired colors of the virtual object and the real object, and the colors after the linear processing are converted into colors in the color space of the rendering engine.

[0087] For example, the first image on the real screen is output to the real screen by the rendering engine. In this process, the color of the first image in the color space of the rendering engine is converted to the color in the color space of the real screen. Therefore, the method of converting the color in the color space of the real screen back to the color in the color space of the rendering engine is also known. On the premise that the color of the first image on the real screen is consistent with the color of the first image in the camera screen by color calibration in advance, the color of the first image in the camera screen can be converted to the color space of the rendering engine by the same conversion method. The color of the real object in the camera screen can also be converted to the color space of the rendering engine based on the same method.

[0088] That is, color_ue_LED_workspace=f(color_ue_LED), color_real_workspace=f(color_real), where color_ue_LED can represent the color of a virtual object after linear processing, and color_real can represent the color of a real object after linear processing. color_ue_LED_workspace can represent the color of a virtual object after conversion to the color space of a rendering engine. color_real_workspace can represent the color of a real object after conversion to the color space of a rendering engine. Linear processing can be based on related technologies to convert colors from nonlinear to linear. Function f() can represent the function used for conversion from the color space of a camera screen to the color space of a rendering engine. When the color space of a camera screen and the color space of a rendering engine are of the same type, the function is used to map the numerical range of the color space of a camera screen to the numerical range of the color space of a rendering engine. Function f() can be implemented based on existing technologies and will not be described in detail here.

[0089] Since the camera is color calibrated in advance, the color of the acquired virtual object can be converted into the color of the color space of the rendering engine. Assuming that there is no error in each step of the color adjustment method, the color adjustment method and color adjustment parameters determined according to the color in the converted color space of the rendering engine are applied to the rendering engine, and the color of the virtual object in the camera screen is consistent with the color of the real object. That is, the color adjustment of the virtual object can be completed by determining the color adjustment method and color adjustment parameters only once, and step S25 can be omitted, further reducing the number of iterations and improving the efficiency of color adjustment.

[0090] The above steps S25-S27 may also be implemented by a pre-written plug-in of the rendering engine.

[0091] The following describes an exemplary method for determining the color adjustment mode according to an embodiment of the present disclosure.

[0092] In a possible implementation, determining a color adjustment method and a color adjustment parameter of a rendering engine according to the acquired color includes:

[0093] Determine the hue distribution of the real object according to the acquired color of the real object;

[0094] Determine the hue distribution of the virtual object according to the acquired color of the virtual object;

[0095] When the difference between the hue distribution of the real object and the hue distribution of the virtual object is greater than a second preset threshold, determining that the color adjustment mode is global color adjustment;

[0096] When the difference between the hue distribution of the real object and the hue distribution of the virtual object is less than or equal to a second preset threshold, it is determined that the color adjustment mode is local color adjustment.

[0097] For example, the color of the real object obtained may include the color vector corresponding to the pixel occupied by the real object in the camera image. Based on the color of the real object obtained, the hue distribution of the real object may be determined, that is, the hit situation of the color of the real object to different hues. Similarly, the color of the virtual object obtained may include the color vector corresponding to the pixel occupied by the virtual object in the camera image. Based on the color of the virtual object obtained, the hue distribution of the virtual object may be determined, that is, the hit situation of the color of the virtual object to different hues. Determining the hue distribution may be implemented based on existing technologies, for example, which will not be repeated here.

[0098] The difference between the hue distribution of the real object and the hue distribution of the virtual object can be calculated based on the prior art. In one example, some hues of the virtual object's color hit may not be hit by the real object's color. Similarly, some hues of the real object's color hit may not be hit by the virtual object's color. The number of the above-mentioned hues that are not hit can be counted, and the ratio of the number to the total number of hues of the virtual object and the real object's color hits can be used as the difference between the hue distribution of the real object and the hue distribution of the virtual object. The embodiments of the present disclosure do not limit the specific calculation method of the hue distribution difference.

[0099] It is understandable that the greater the difference between the hue distribution of the real object and the hue distribution of the virtual object, the more obvious the color difference between the real object and the virtual object in the camera screen will be. Therefore, the second preset threshold can be set according to the application scenario requirements, for example, set to 50%. When the difference between the hue distribution of the real object and the hue distribution of the virtual object is greater than the second preset threshold, it can be confirmed that the color difference between the real object and the virtual object in the camera screen is large, and the effect of using global color adjustment is better. The color adjustment method can be determined as global color adjustment, such as PPV. On the contrary, when the difference between the hue distribution of the real object and the hue distribution of the virtual object is less than or equal to the second preset threshold, it can be confirmed that the color difference between the real object and the virtual object in the camera screen is small, and the effect of using local color adjustment is better. The color adjustment method can be determined as local color adjustment, such as CCR. The disclosed embodiment does not limit the specific value of the second preset threshold.

[0100] Furthermore, after the color adjustment method is determined to be local color adjustment, the area that needs to be color adjusted can also be determined based on the acquired color of the real object and the color of the virtual object. This step can be implemented based on the existing technology, and the embodiment of the present disclosure does not limit the method for determining the specific area of ​​local color adjustment. For example, the pixels occupied by the real object that cause the hue distribution of the real object and the hue distribution of the virtual object to differ can be found, and the area that needs to be color adjusted can be determined based on the position of the pixel, so that the determined area can include the pixel.

[0101] In this way, the determined color adjustment method is more adapted to the actual application scenario.

[0102] In an exemplary application scenario, there may also be multiple real objects and multiple virtual objects in the camera screen, such as virtual object 1 and virtual object 2, and real object 1 and real object 2. Assuming that virtual object 1 corresponds to real object 1, and virtual object 2 corresponds to real object 2, after using the color adjustment method of the embodiment of the present disclosure to make the colors of virtual object 1 and real object 1 in the camera screen consistent, when there is a difference between the hue distribution of real object 2 and the hue distribution of virtual object 2, the color adjustment method can be directly determined to be local color adjustment, and the area for local color adjustment is selected from the area occupied by virtual object 2. In this way, it is possible to avoid the subsequent color adjustment step changing the color of the area that has completed color adjustment.

[0103] An exemplary method for determining color adjustment parameters according to an embodiment of the present disclosure is described below.

[0104] In one possible implementation, the color adjustment parameters include the gain of each color channel.

[0105] The color adjustment method and color adjustment parameters of the rendering engine are determined according to the acquired color, including:

[0106] Determine a first color vector according to the acquired color of the real object;

[0107] Determine a second color vector according to the acquired color of the virtual object;

[0108] For each color channel, determining a gain of the color channel according to a ratio of values ​​of the first color vector and the second color vector in the color channel;

[0109] In the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the same color channel in the first image generated previously and the gain of the color channel.

[0110] For example, the rendering engine renders a texture map in its own color space through the asset's texture, lighting, etc., and then uses the color adjustment parameters to adjust the color of the texture map. Common color adjustment parameters may include gain. The gain can amplify or reduce the value of the color channel through multiplication to control the brightness and contrast of the image, thereby changing the image color. A larger gain will make the image brighter, while a smaller gain will make the image darker. Since color is usually represented by a color vector composed of the values ​​of three color channels, the color adjustment parameters of the embodiment of the present disclosure may include the gain of each color channel.

[0111] Since the acquired color includes multiple color vectors, the first color vector that best represents the color of the real object can be determined based on the acquired color of the real object. Exemplarily, the value of each color channel of the first color vector can be the average value of each color vector of the acquired real object in the same color channel. Similarly, the second color vector that best represents the color of the virtual object can be determined based on the acquired color of the virtual object. Exemplarily, the value of each color channel of the second color vector can be the average value of each color vector of the acquired virtual object in the same color channel.

[0112] Those skilled in the art should understand that, in practical applications, there are more ways to determine the first color vector and the second color vector, such as using the mode of the values ​​of each color channel of the real object as the value of the first color vector in the same color channel, using the mode of the values ​​of each color channel of the virtual object as the value of the second color vector in the same color channel, etc. The embodiments of the present disclosure do not limit the specific method for determining the first color vector and the second color vector.

[0113] Next, for each color channel, the gain of the color channel can be determined according to the ratio of the values ​​of the first color vector and the second color vector in the color channel. That is, Gain = color_1 / color_2. In which, color_1 represents the value of the first color vector in a certain color channel, and color_2 represents the value of the second color vector in the same color channel.

[0114] Those skilled in the art should understand that when the camera is not color calibrated, color_2 may be the result of linear processing of the color in the color space of the camera image. When the camera is color calibrated, color_2 may be the color in the color space of the camera image converted to the color space of the rendering engine after linear processing.

[0115] In this way, the method of determining the color adjustment parameters is simpler and the efficiency of determining the color adjustment parameters is higher.

[0116] When the color adjustment parameters include gain, the value of any color channel of each pixel whose color changes in the regenerated first image is equal to the product of the value of the same pixel in the same color channel in the first image generated last time and the gain of the color channel. That is, color_out = color_in × Gain, where color_in represents the value of a color in a certain color channel in the color space of the rendering engine, and color_out represents the value of the color channel after color adjustment. Therefore, after the rendering engine applies the color adjustment parameters, each color rendered in the area to be color adjusted is automatically color adjusted, so that the color of the regenerated first image changes, the color of the reacquired camera image also changes accordingly, and the color of the virtual object in the reacquired camera image is closer to the color of the real object.

[0117] Those skilled in the art should understand that each color channel may also use the same gain, for example, taking the average value of the gain of each color channel as the common gain of each color channel, etc. The embodiments of the present disclosure do not limit the specific calculation method of the gain.

[0118] Another exemplary method for determining color adjustment parameters according to an embodiment of the present disclosure is described below.

[0119] In a possible implementation, the color adjustment parameters include gain and offset.

[0120] According to the acquired color, determine the color adjustment method and color adjustment parameters of the rendering engine, including:

[0121] Determine a third color vector and a fourth color vector according to the acquired color of the real object;

[0122] Determine a fifth color vector and a sixth color vector according to the acquired color of the virtual object;

[0123] For each color channel, calculate a first difference between the values ​​of the third color vector and the fourth color vector in the color channel, and a second difference between the values ​​of the fifth color vector and the sixth color vector in the color channel, and determine a gain of the color channel according to a ratio of the first difference to the second difference;

[0124] For each color channel, calculate the product of the value of the fifth color vector in the color channel and the gain of the color channel, and determine the offset of the color channel according to the difference between the value of the third color vector in the color channel and the product;

[0125] In the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the same color channel in the previously generated first image and the gain of the color channel plus the offset of the color channel.

[0126] For example, common color adjustment parameters of a rendering engine also include an offset. The offset is used to control the overall brightness offset of an image, which can be achieved by increasing or decreasing the value of a color channel through an addition operation. Based on this, the color adjustment parameters of an embodiment of the present disclosure may include gain and offset. The gain has been introduced above and will not be repeated here.

[0127] Since the acquired color includes multiple color vectors, the third color vector and the fourth color vector that can represent the color of the real object can be determined first according to the acquired color of the real object. Exemplarily, the first part of pixels and the second part of pixels occupied by the real object in the camera screen can be determined from the pixels occupied by the real object in the camera screen. The value of each color channel of the third color vector can be the average value of the color vector of the first part of pixels occupied by the real object in the camera screen in the same color channel. The value of each color channel of the fourth color vector can be the average value of the color vector of the second part of pixels occupied by the real object in the camera screen in the same color channel.

[0128] Similarly, according to the color of the acquired virtual object, the fifth color vector and the sixth color vector that can represent the color of the virtual object can be determined first. Exemplarily, the first part of pixels and the second part of pixels occupied by the virtual object in the camera screen can be determined from the pixels occupied by the virtual object in the camera screen. The first part of pixels and the second part of pixels can be two parts of pixels in the area where the virtual object is located and adjacent to the area where the real object is located. The first part of pixels and the second part of pixels can be randomly selected or selected according to the color distribution of the virtual object to represent the main color of the virtual object. The colors of the first part of pixels and the second part of pixels can be the same or different. The value of each color channel of the fifth color vector can be the average value of the color vector of the first part of pixels occupied by the virtual object in the camera screen in the same color channel. The value of each color channel of the sixth color vector can be the average value of the color vector of the second part of pixels occupied by the virtual object in the camera screen in the same color channel.

[0129] The first part of pixels occupied by the virtual object in the camera screen may be adjacent to the first part of pixels occupied by the real object in the camera screen, and the second part of pixels occupied by the virtual object in the camera screen may be adjacent to the second part of pixels occupied by the real object in the camera screen. Therefore, the third color vector may correspond to the fifth color vector, and the fourth color vector may correspond to the sixth color vector.

[0130] Figure 6 An example of a first portion of pixels and a second portion of pixels occupied by a virtual object and a real object in a camera screen according to an embodiment of the present disclosure is shown.

[0131] like Figure 6 As shown, in the camera image, the curved edge of the real object separates the virtual object from the real object. The first part of pixels and the second part of pixels occupied by the virtual object and the real object in the camera image can be selected from both sides of the curved edge.

[0132] Those skilled in the art should understand that in practical applications, the third, fourth, fifth, and sixth color vectors may be determined in more ways. Figure 6 For example, the camera screen can also be divided into multiple areas according to the horizontal coordinates (or vertical coordinates) of the pixels, and two areas are selected from the areas that simultaneously display virtual objects and real objects, and a third color vector is obtained according to the color vectors of the pixels occupied by the real objects in the first area, a fourth color vector is obtained according to the color vectors of the pixels occupied by the real objects in the second area, a fifth color vector is obtained according to the color vectors of the pixels occupied by the virtual objects in the first area, and a sixth color vector is obtained according to the color vectors of the pixels occupied by the virtual objects in the second area. The embodiments of the present disclosure do not limit the specific method for determining the third, fourth, fifth, and sixth color vectors.

[0133] Next, for each color channel, a first difference between the values ​​of the third color vector and the fourth color vector in the color channel, and a second difference between the values ​​of the fifth color vector and the sixth color vector in the color channel can be calculated. According to the ratio of the first difference and the second difference, the gain of the color channel can be determined. That is, Gain = (color_1_A-color_1_B) / (color_2_A-color_2_B). Among them, color_1_A represents the value of the third color vector in a certain color channel, and color_1_B represents the value of the fourth color vector in the same color channel. Color_2_A represents the value of the fifth color vector in the same color channel, and color_2_B represents the value of the sixth color vector in the same color channel.

[0134] For each color channel, the product of the value of the fifth color vector in the color channel and the gain of the color channel can be calculated, and the offset of the color channel can be determined according to the difference between the value of the third color vector in the color channel and the product, that is, Offset=color_1_A-color_2_A×Gain.

[0135] Those skilled in the art should understand that when the camera is not color calibrated, color_2_A and color_2_B may be the result of linear processing of the colors in the color space of the camera image. When the camera is color calibrated, color_2_A and color_2_B may be the colors in the color space of the rendering engine converted from the colors in the color space of the camera image after linear processing.

[0136] In this way, the method of determining the color adjustment parameters is simpler and the efficiency of determining the color adjustment parameters is higher.

[0137] When the color adjustment parameters include gain and offset, the value of any color channel of each pixel whose color changes in the regenerated first image is equal to the product of the value of the same pixel in the same color channel in the first image generated last time and the gain of the color channel plus the offset of the color channel. That is, color_out = color_in × Gain + Offset, where color_in represents the value of a color in a certain color channel in the color space of the rendering engine, and color_out represents the value of the color channel after color adjustment. Therefore, after the rendering engine applies the color adjustment parameters, each color rendered in the area to be color adjusted is automatically color adjusted, so that the color of the regenerated first image changes, the color of the reacquired camera image also changes accordingly, and the color of the virtual object in the reacquired camera image is closer to the color of the real object.

[0138] Those skilled in the art should understand that each color channel may also use the same gain and offset, for example, using the average value of the gain of each color channel as the common gain of each color channel, using the average value of the offset of each color channel as the common offset of each color channel, etc. The embodiments of the present disclosure do not limit the specific calculation method of the gain and offset.

[0139] The disclosed embodiment also provides a color adjustment device. Figure 7 A schematic diagram showing the structure of a color adjustment device according to an embodiment of the present disclosure.

[0140] like Figure 7 As shown, in a possible implementation, the color adjustment device includes:

[0141] A first acquisition module 71 is used to acquire a camera image, wherein the camera image includes a real object and a virtual object, wherein the virtual object is an object included in a first image displayed on a real screen, wherein the first image is generated by a rendering engine, and the virtual object corresponds to the real object;

[0142] A second acquisition module 72, used to respectively acquire the colors of the virtual object and the real object in the camera image;

[0143] A first determination module 73, configured to determine a color adjustment method and a color adjustment parameter of the rendering engine according to the acquired color, wherein the color adjustment method is related to a degree of color difference between the virtual object and the real object in the camera image;

[0144] The first generating module 74 is used to enable the rendering engine to regenerate the first image using the color adjustment method and the color adjustment parameters, and to enable the first acquiring module to reacquire the camera image when the real screen displays the new first image.

[0145] In a possible implementation, the device also includes: a third acquisition module, which is used to respectively acquire the colors of the virtual object and the real object in the camera screen after the first acquisition module re-acquires the camera screen; the device is also used to, when the difference in color between the virtual object and the real object in the camera screen is greater than a first preset threshold, re-execute the steps of determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired color and subsequent steps.

[0146] In a possible implementation, the device also includes: a calibration module, which is used to perform color calibration on the camera based on the first image on the real screen and the first image in the camera screen before the first acquisition module acquires the camera screen, so that the color of the first image on the real screen is consistent with the color of the first image in the camera screen.

[0147] In a possible implementation, the color space of the rendering engine is a linear color space, and the device further includes: a conversion module, configured to perform linear processing on the acquired colors of the virtual object and the real object before the first determination module determines the color adjustment method and color adjustment parameters of the rendering engine according to the acquired colors, and convert the linearly processed colors into colors in the color space of the rendering engine.

[0148] In a possible implementation, the color adjustment parameters include the gain of each color channel, and determining the color adjustment method and color adjustment parameters of the rendering engine based on the acquired color includes: determining a first color vector based on the acquired color of the real object; determining a second color vector based on the acquired color of the virtual object; for each color channel, determining the gain of the color channel based on the ratio of the values ​​of the first color vector and the second color vector in the color channel; in the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the same color channel in the previously generated first image and the gain of the color channel.

[0149] In a possible implementation, the color adjustment parameters include gain and offset, and the color adjustment method and color adjustment parameters of the rendering engine are determined according to the acquired color, including: determining a third color vector and a fourth color vector according to the acquired color of the real object; determining a fifth color vector and a sixth color vector according to the acquired color of the virtual object; for each color channel, calculating a first difference between the values ​​of the third color vector and the fourth color vector in the color channel, and a second difference between the values ​​of the fifth color vector and the sixth color vector in the color channel, and determining the gain of the color channel according to a ratio of the first difference to the second difference; for each color channel, calculating the product of the value of the fifth color vector in the color channel and the gain of the color channel, and determining the offset of the color channel according to the difference between the value of the third color vector in the color channel and the product; in the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the same color channel in the first image generated last time and the gain of the color channel plus the offset of the color channel.

[0150] In a possible implementation, determining the color adjustment method and color adjustment parameters of the rendering engine according to the acquired color includes: determining the hue distribution of the real object according to the acquired color of the real object; determining the hue distribution of the virtual object according to the acquired color of the virtual object; when the difference between the hue distribution of the real object and the hue distribution of the virtual object is greater than a second preset threshold, determining that the color adjustment method is global color adjustment; when the difference between the hue distribution of the real object and the hue distribution of the virtual object is less than or equal to the second preset threshold, determining that the color adjustment method is local color adjustment.

[0151] In a possible implementation, the device is a plug-in of a rendering engine.

[0152] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0153] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0154] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0155] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0156] Figure 8 1 is a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server or a terminal device. Figure 8 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0157] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2003. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0158] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0159] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0160] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0161] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0162] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0163] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0164] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0165] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0166] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0167] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A color adjustment method, characterized in that: The method comprises: Acquire a camera picture, wherein the camera picture includes a real object and a virtual object, wherein the virtual object is an object included in a first image displayed on a real screen, wherein the first image is generated by a rendering engine, and the virtual object corresponds to the real object; Respectively obtaining the colors of the virtual object and the real object in the camera picture; Determining a color adjustment method and a color adjustment parameter of the rendering engine according to the acquired color, wherein the color adjustment method is related to the degree of color difference between the virtual object and the real object in the camera picture; The rendering engine is enabled to regenerate the first image using the color adjustment method and the color adjustment parameters, and the camera picture is reacquired when the real screen displays the new first image.

2. The method according to claim 1, characterized in that After reacquiring the camera image, the method further includes: Respectively obtaining the colors of the virtual object and the real object in the camera picture; When the color difference between the virtual object and the real object in the camera picture is greater than a first preset threshold, re-execute the steps of determining the color adjustment method and color adjustment parameters of the rendering engine according to the acquired color and subsequent steps.

3. The method according to claim 1, characterized in that Before acquiring the camera image, the method further includes: The camera is color calibrated according to the first image on the real screen and the first image in the camera picture, so that the color of the first image on the real screen is consistent with the color of the first image in the camera picture.

4. The method according to claim 3, characterized in that The color space of the rendering engine is a linear color space. Before determining the color adjustment mode and color adjustment parameters of the rendering engine according to the acquired color, the method further includes: Linear processing is performed on the acquired colors of the virtual object and the real object, and the colors after the linear processing are converted into colors in the color space of the rendering engine.

5. The method according to any one of claims 1 to 4, characterized in that The color adjustment parameters include the gain of each color channel, The step of determining the color adjustment method and color adjustment parameters of the rendering engine according to the acquired color includes: Determine a first color vector according to the acquired color of the real object; Determine a second color vector according to the acquired color of the virtual object; For each color channel, determining a gain of the color channel according to a ratio of values ​​of the first color vector and the second color vector in the color channel; In the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the same color channel in the first image generated previously and the gain of the color channel.

6. The method according to any one of claims 1 to 4, characterized in that The color adjustment parameters include gain and offset, The step of determining the color adjustment method and color adjustment parameters of the rendering engine according to the acquired color includes: Determine a third color vector and a fourth color vector according to the acquired color of the real object; Determine a fifth color vector and a sixth color vector according to the acquired color of the virtual object; For each color channel, calculate a first difference between the values ​​of the third color vector and the fourth color vector in the color channel, and a second difference between the values ​​of the fifth color vector and the sixth color vector in the color channel, and determine a gain of the color channel according to a ratio of the first difference to the second difference; For each color channel, calculate the product of the value of the fifth color vector in the color channel and the gain of the color channel, and determine the offset of the color channel according to the difference between the value of the third color vector in the color channel and the product; In the regenerated first image, the value of any color channel of each pixel whose color changes is equal to the product of the value of the same pixel in the same color channel in the previously generated first image and the gain of the color channel plus the offset of the color channel.

7. The method according to any one of claims 1 to 4, characterized in that The step of determining the color adjustment method and color adjustment parameters of the rendering engine according to the acquired color includes: Determining the hue distribution of the real object according to the acquired color of the real object; Determining a hue distribution of the virtual object according to the acquired color of the virtual object; When the difference between the hue distribution of the real object and the hue distribution of the virtual object is greater than a second preset threshold, determining that the color adjustment mode is global color adjustment; When the difference between the hue distribution of the real object and the hue distribution of the virtual object is less than or equal to a second preset threshold, it is determined that the color adjustment mode is local color adjustment.

8. The method according to any one of claims 1 to 7, characterized in that The method is executed by a plug-in of a rendering engine.

9. A color adjustment device, characterized in that: The device comprises: A first acquisition module is used to acquire a camera picture, wherein the camera picture includes a real object and a virtual object, wherein the virtual object is an object included in a first image displayed on a real screen, wherein the first image is generated by a rendering engine, and the virtual object corresponds to the real object; A second acquisition module, used to respectively acquire the colors of the virtual object and the real object in the camera picture; A first determination module, configured to determine a color adjustment method and a color adjustment parameter of the rendering engine according to the acquired color, wherein the color adjustment method is related to a degree of color difference between the virtual object and the real object in the camera picture; The first generating module is used to enable the rendering engine to regenerate the first image using the color adjustment method and the color adjustment parameters, and to enable the first acquiring module to reacquire the camera image when the real screen displays the new first image.

10. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method described in any one of claims 1 to 8 when executing the instructions stored in the memory.

11. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product comprising computer readable code, or a non-volatile computer readable storage medium carrying computer readable code, characterized in that: When the computer readable code runs in a processor of an electronic device, the processor in the electronic device executes the method according to any one of claims 1 to 8.

Images