Color Correction Method, Device and Storage Medium for Extended Reality Shooting

Abstract

The present disclosure relates to a color correction method, apparatus, and storage medium for extended reality shooting, including: performing color correction on a target image to be presented on a screen according to a first color mapping table, where the first color mapping table represents a mapping relationship between the input color of the screen and the target color to be displayed on the screen, and the target color is a color that can be obtained by performing color acquisition on the image to be presented on the screen via an image acquisition device; obtaining a captured image according to the target image after color correction; performing color correction on an extended image according to a second color mapping table, where the second color mapping table represents a mapping relationship between the color of the captured image and the composite color of the extended image; and synthesizing the captured image and the extended image after color correction to obtain an extended reality image. Thus, color matching between the actual captured image and the extended image can be achieved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of extended reality shooting, and in particular, to a color correction method, device, and storage medium for extended reality shooting. Background Art

[0002] In traditional film and television production, to meet the shooting requirements, a large amount of time and manpower are required to select shooting locations, produce props, and build shooting scenes. However, virtual production (abbreviation: VP) technology can use virtual scenes rendered by a rendering engine to replace real sets, reducing the dependence of film and television shooting on locations and sets, and greatly reducing the shooting cost. Moreover, for some special effects that originally required post-production, with the ability of real-time rendering, virtual production can see the finished film effect during the shooting stage, bringing post-production forward and improving the production efficiency. Due to these advantages, virtual production has been increasingly applied to film and television production in recent years.

[0003] With the continuous progress of display and control technologies, virtual production is increasingly applied to film and television production, especially extended reality (abbreviation: XR) technology, which has achieved amazing shooting effects in many projects. XR shooting can stitch the real scene images captured by a camera with the virtual images in a rendering engine to create an augmented reality experience of overlapping virtual and real scenes.

[0004] However, XR shooting also faces difficulties in many aspects such as technology and systems, making the large-scale application of XR shooting still face many challenges. How to achieve color matching between the real shooting part (real scene images) and the extended part (extended images) is one of the difficult technologies. Summary of the Invention

[0005] In view of this, the present disclosure provides a color correction method, device, and storage medium for extended reality shooting, so as to achieve color matching between the real shooting part and the extended part.

[0006] According to a first aspect of the present disclosure, there is provided a color correction method for extended reality shooting, including: performing color correction on a target picture to be presented on a screen according to a first color mapping table, where the first color mapping table represents a mapping relationship between an input color of the screen and a target color to be displayed on the screen, and the target color is a color that can be obtained by collecting the color of the picture to be presented on the screen through an image acquisition device; obtaining a shooting picture according to the target picture after color correction; performing color correction on an extended picture according to a second color mapping table, where the second color mapping table represents a mapping relationship between the color of the shooting picture and the composite color of the extended picture; and synthesizing the shooting picture and the extended picture after color correction to obtain an extended reality picture.

[0007] In a possible implementation, performing color correction on a target picture to be presented on a screen according to a first color mapping table includes: determining a target color to be displayed on the screen; solving a first target input color according to the first color mapping table and the target color; and adjusting the color of the target picture to the first target input color.

[0008] In a possible implementation, before performing color correction on a target picture to be presented on a screen according to a first color mapping table, the color correction method further includes: generating a first color sampling point by using a standard color generator as a first input color sampling value; rendering the first color sampling point on the screen, and collecting, by using the image acquisition device, the color obtained after color conversion of the first color sampling point through the screen as a first color collection value; and analyzing the first input color sampling value and the first color collection value to construct the first color mapping table for representing the mapping relationship between the first input color sampling value and the first color collection value, where the mapping relationship between the first input color sampling value and the first color collection value corresponds to the mapping relationship between the input color and the target color.

[0009] In a possible implementation, obtaining a shooting picture according to the target picture after color correction includes: rendering the target picture after color correction on the screen; and shooting the rendered picture and a real scene picture to obtain the shooting picture.

[0010] In a possible implementation, performing color correction on an extended picture according to a second color mapping table includes: solving a target composite color according to the color of the shooting picture and the second color mapping table; and adjusting the color of the extended picture to the target composite color.

[0011] In a possible implementation, before color-correcting the extended screen according to the second color mapping table, the color correction method further includes: generating second color sampling points by using a standard color generator as second input color sampling values; color-correcting the second color sampling points by using the first color mapping table; rendering the color-corrected second color sampling points on the screen, and collecting, by using the image acquisition device, the colors obtained after color conversion of the color-corrected second color sampling points passing through the screen as second color collection values; analyzing the second input color sampling values and the second color collection values to construct the second color mapping table for representing the mapping relationship between the second input color sampling values and the second color collection values, where the mapping relationship between the second input color sampling values and the second color collection values corresponds to the mapping relationship between the color of the captured screen and the composite color of the extended screen.

[0012] According to a second aspect of the present disclosure, there is provided a color correction device for extended reality shooting, including: a first color correction module, configured to color-correct a target screen to be presented on a screen according to a first color mapping table, where the first color mapping table represents the mapping relationship between the input color of the screen and the target color to be displayed on the screen, and the target color is a color that can be obtained by collecting the color of the screen to be presented by using an image acquisition device; a first obtaining module, configured to obtain a captured screen according to the color-corrected target screen; a second color correction module, configured to color-correct an extended screen according to a second color mapping table, where the second color mapping table represents the mapping relationship between the color of the captured screen and the composite color of the extended screen; and a second obtaining module, configured to composite the captured screen and the color-corrected extended screen to obtain an extended reality screen.

[0013] In a possible implementation, the first color correction module is configured to: determine the target color to be displayed on the screen; solve a first target input color according to the first color mapping table and the target color; and adjust the color of the target screen to the first target input color.

[0014] In a possible implementation, the above color correction device further includes: a first construction module configured to construct the first color mapping table, where the first construction module is configured to: generate first color sampling points using a standard color generator as first input color sampling values; render the first color sampling points on the screen, and use the image acquisition device to collect the colors obtained after the color conversion of the first color sampling points passing through the screen as first color acquisition values; analyze the first input color sampling values and the first color acquisition values to construct the first color mapping table representing the mapping relationship between the first input color sampling values and the first color acquisition values, where the mapping relationship between the first input color sampling values and the first color acquisition values corresponds to the mapping relationship between the input color and the target color.

[0015] In a possible implementation, the first acquisition module is configured to: render the target screen after color correction on the screen; capture the rendered screen and the real scene screen to obtain the captured screen.

[0016] In a possible implementation, the second color correction module is configured to: solve the target composite color according to the color of the captured screen and the second color mapping table; adjust the color of the extended screen to the target composite color.

[0017] In a possible implementation, the above color correction device further includes: a second construction module configured to construct the second color mapping table, where the second construction module is configured to: generate second color sampling points using a standard color generator as second input color sampling values; perform color correction on the second color sampling points using the first color mapping table; render the second color sampling points after color correction on the screen, and use the image acquisition device to collect the colors obtained after the color conversion of the second color sampling points after color correction passing through the screen as second color acquisition values; analyze the second input color sampling values and the second color acquisition values to construct the second color mapping table representing the mapping relationship between the second input color sampling values and the second color acquisition values, where the mapping relationship between the second input color sampling values and the second color acquisition values corresponds to the mapping relationship between the color of the captured screen and the composite color of the extended screen.

[0018] According to a third aspect of the present disclosure, there is provided a color correction device for extended reality shooting, including: a processor; a memory for storing instructions executable by the processor; wherein, the processor is configured to implement the above color correction method when executing the instructions stored in the memory.

[0019] According to a fourth aspect of the present disclosure, there is provided a non-volatile computer-readable storage medium having computer program instructions stored thereon, wherein when the computer program instructions are executed by a processor, the above-described color correction method is implemented.

[0020] According to the color correction method, device and storage medium for extended reality shooting of the present disclosure, different color mapping tables are used to correct the on-screen image and the extended image respectively, the corrected on-screen image is rendered, the rendered image and the real scene image are collected to obtain the captured image, and the captured image and the corrected extended image are fused to obtain the XR image. It is possible to ensure color matching between the real captured image and the extended image, and perform accurate color matching within an effective color expression range, improving the robustness of color matching.

[0021] Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings included in and constituting a part of this specification illustrate exemplary embodiments, features, and aspects of the present disclosure, and are used to explain the principles of the present disclosure.

[0023] Figure 1 The flowchart showing the color correction method for extended reality shooting according to an embodiment of the present disclosure.

[0024] Figure 2 The system architecture diagram showing the color correction method for extended reality shooting according to an embodiment of the present disclosure.

[0025] Figure 3 The block diagram showing the color correction device for extended reality shooting according to an embodiment of the present disclosure.

[0026] Figure 4 The structural schematic diagram showing an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the drawings denote elements having the same or similar functions. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0028] The term "exemplary" used herein means "serving as an example, embodiment, or illustration". Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

[0029] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0030] In related technologies, the color matching scheme between the real shot image and the extended image taken by XR aims to accurately correct the color of the real shot image, so that a better matching effect can be obtained by directly superimposing the extended image on the accurately corrected real shot image. Among them, the color correction system for the screen and image acquisition device can be used to ensure the color accuracy of the real shot image.

[0031] Although the above-mentioned related technologies can indeed solve the color matching problem between the real shot and the extended picture in some scenarios, the effect is not robust. The reason is that the overall color system of the screen and image acquisition device has various characteristics and cannot be perfectly corrected in many cases. For example, if the color gamut is not enough, the real shot and the extended picture after color correction will have color differences. The colors of the extended picture and the real shot after color correction do not match, resulting in color errors at the virtual and real boundaries.

[0032] To this end, a color correction method and device for extended reality shooting of the present invention are proposed, so that accurate color matching can be performed within the effective color expression range while ensuring color matching between the real shot picture and the extended picture, thereby improving the robustness of color matching.

[0033] The following combination Figures 1 to 4 The color correction method and device for augmented reality shooting according to an exemplary embodiment of the present disclosure are described in detail.

[0034] Figure 1 A flow chart of a color correction method for augmented reality photography according to an embodiment of the present disclosure is shown. Figure 1 As shown, the color correction method may include:

[0035] Step S110, color correction is performed on a target image to be presented on the screen according to a first color mapping table, wherein the first color mapping table represents a mapping relationship between an input color of the screen and a target color to be displayed on the screen, and the target color is a color that can be obtained by performing color capture on the image to be presented on the screen via an image capture device.

[0036] In this embodiment, the target picture is the picture to be presented on the screen, and the picture to be presented on the screen is also called the upper screen picture, so the target picture is actually the target upper screen picture.

[0037] The first color mapping table related to the colors that can be obtained by a color acquisition device such as a camera for color acquisition of the screen image to be presented is used for color correction of the screen image. Thus, this first color mapping table is related to the color system of the overall combination of the screen and the image acquisition device. Therefore, as Figure 2 shown, in the shooting stage, before rendering the target image (the image to be displayed on the screen) on the screen (the virtual shooting screen), the target image is first color-corrected using the first color mapping table (color mapping table 1) (via the color correction system) to correct the input color of the screen to the input color required for the image acquisition device to capture the target image presented on the screen.

[0038] In a possible implementation, the first color mapping table includes a three-dimensional lookup table. Exemplarily, this three-dimensional lookup table is an RGB color lookup table.

[0039] Step S120: Obtain the shooting image based on the color-corrected target image.

[0040] In this embodiment, as Figure 2 shown, after color-correcting the image to be displayed on the screen using the first color mapping table, the color-corrected image to be displayed on the screen is input to the screen (the virtual shooting screen), and the color-corrected target image can be displayed on the screen. For example, through a rendering device, the data of the color-corrected target image is rendered to the screen, and the rendered image and the real scene image (the real scene) are captured via the image acquisition device (such as via a camera), and thus a combined virtual and real shooting image can be obtained.

[0041] Since the target image has been color-corrected according to the first color mapping table before being rendered on the screen to correct the input color of the screen to the input color required for the image acquisition device to capture the target image presented on the screen, the screen can present the target image captured by the image acquisition device using the corrected input color, and thus a shooting image with accurate colors can be obtained. This shooting image is also called the actual shooting image.

[0042] Step S130: Color-correct the extended image according to a second color mapping table, where the second color mapping table represents the mapping relationship between the colors of the shooting image and the combined colors of the extended image.

[0043] As described above, after the color correction in step S120, there are parts with accurate color correction and parts with inaccurate color correction in the captured image, such as the parts with inaccurate color correction caused by insufficient color gamut range. In the parts with accurate color correction, the captured image of these parts can be directly synthesized (fused) with the extended image, and an XR image with better color matching between the captured image and the extended image should be obtained. However, in the parts with inaccurate color correction, directly synthesizing the captured image of these parts with the extended image may result in an XR image with unmatched colors between the captured image and the extended image.

[0044] In this embodiment, the extended image can be color-corrected according to the second color mapping table representing the mapping relationship between the color of the captured image and the color of the extended image. In this way, in the parts with inaccurate color correction of the captured image, using this second color correction table, the color of the extended image can be corrected to match the color of the captured image of these parts. And in the parts with accurate color correction of the captured image, using this second color correction table, the color of the extended image can be maintained to match the color of the captured image of these parts.

[0045] Step S140, synthesize the captured image and the color-corrected extended image to obtain an extended reality image.

[0046] In this embodiment, after the color correction in step S130, whether it is the parts with accurate color correction in the captured image or the parts with inaccurate color correction in the captured image, such as the parts with insufficient color gamut range, the color of the extended image matches the color of the captured image of these parts. Therefore, any suitable method in the related art can be used to synthesize the captured image and the extended image with matching colors to obtain an XR image with matching colors between the captured image and the extended image.

[0047] According to the color correction method for extended reality shooting in this embodiment, different color mapping tables are used to correct the on-screen image and the extended image respectively, render the corrected on-screen image, collect the rendered image and the real scene image to obtain the captured image, and fuse the captured image and the corrected extended image to obtain the XR image.

[0048] Therefore, the color matching of the XR scene is completed in two steps, the color mapping for the on-screen image before it is displayed (color correction of the on-screen image using the first mapping table) and the color mapping for the extended image (color correction of the extended image using the second mapping table), taking into account both the accuracy and matching degree of the colors in the XR scene.

[0049] Among them, the accuracy of the color of the real-shot image is ensured via the first color mapping table, and the color matching degree between the real-shot image and the extended image is ensured via the second color mapping table. Specifically speaking, in this embodiment, the color correction of the overall combination of the screen and the image acquisition device is performed, which can ensure the accurate color of the captured image by the image acquisition device, so as to achieve the basic alignment of the color with the extended image, and further make the colors between the real-shot image and the extended image basically match. In addition, within some color intervals where the color correction of the overall combination of the screen and the image acquisition device is not ideal enough, by using a color mapping (the second color mapping table) applied to the extended image, the color matching between the extended image and the real-shot image can be achieved, so as to ensure that the XR captured image is normal and does not show any flaws.

[0050] Therefore, this embodiment can provide an improved and robust color matching scheme. Under the color system of the overall combination of diverse screens and image acquisition devices, it can ensure the color matching between the real-shot image and the extended image, and at the same time achieve accurate matching within the effective color expression range.

[0051] In a possible implementation manner, step S110 may include: determining the target color to be displayed on the screen; solving the first target input color according to the first color mapping table and the target color; adjusting the color of the target image to the first target input color.

[0052] In this embodiment, since the mapping relationship between the input color of the screen and the target color to be presented on the screen can be determined according to the first color mapping table, and the color system of the screen and the image acquisition device will change the color of the target image, therefore, in order to enable the image acquisition device to finally capture the target image presented by the screen, it is necessary to solve the corresponding input color according to the first color mapping table and the color of the target image as the first target input color. Among them, the screen uses this input color to present the corresponding image. For example, the rendering device can perform rendering based on the response data of the first target input color to present the corresponding image on the screen, and the image obtained by the image acquisition device capturing this corresponding image is the target image.

[0053] After solving the input color, the color of the target image can be adjusted to the first target input color. In this way, the input color of the screen is adjusted to this first target input color. Therefore, the image obtained by the image acquisition device capturing the image finally presented by the screen is actually the target image, so that the same optical response as the original target image can be obtained, and further a captured image with accurate color can be obtained.

[0054] In a possible implementation, a first color mapping table can be pre-constructed. The first color mapping table can be pre-constructed in the following way: Use a standard color generator to generate first color sampling points as first input color sampling values; Render the first color sampling points on the screen, and use the color acquisition device to acquire the colors obtained after the color conversion of the first color sampling points through the screen as first color acquisition values; Analyze the first input color sampling values and the first color acquisition values to construct the first color mapping table representing the mapping relationship between the first input color sampling values and the first color acquisition values, where the mapping relationship between the first input color sampling values and the first color acquisition values corresponds to the mapping relationship between the input color and the target color.

[0055] In this embodiment, as Figure 2 shown, in the initial screen calibration stage, a standard color generator is used to generate each color sampling point (first color sampling point) of the standard sampling color to obtain the input color sampling value color_samples1 (first input color sampling value), so as to ensure that the input color conforms to the standard definition and is unique. Among them, the standard color generator can be implemented based on software. For example, the color values of each color sampling point are generated through software according to the required rules. The each color sampling point is rendered on the virtual shooting screen through the on-screen interface, and the color after the screen rendering is the color obtained after the color conversion of each color sampling point through the screen. At the same time, the input color sampling value color_samples1 is sent to the color analysis system.

[0056] Then, as Figure 2 shown, in the initial screen calibration stage, the color of the screen presented is acquired through an image acquisition device such as a camera to acquire the screen-rendered image to obtain the color acquisition value color_measure1 (first color acquisition value). The color acquisition value color_measure1 is sent to the color analysis system. Then, through the color analysis system, statistical analysis is performed on the color sampling value color_samples1 and the color acquisition value color_measure1, and a function representing the mapping relationship between the color sampling value color_samples1 and the color acquisition value color_measure1 can be constructed as the color mapping table color_map1 (first color mapping table).

[0057] That is to say, in the initial screen calibration stage, according to the screen getting brighter, darker, redder, greener, etc., through the relationship between the color sampling value color_samples1 and the color acquisition value color_measure1, dense sampling at multiple points can be performed, and the color mapping relationship (i.e., the first color mapping table) between the color sampling value color_samples1 and the color acquisition value color_measure1 should be able to be constructed. In this way, after the first color mapping relationship is constructed, it can be specified what color should be input to the screen when the image acquisition device captures the picture presented on the screen, that is, the color to which the on-screen picture is to be adjusted.

[0058] Exemplarily, compare the color sampling value color_samples1 with the color acquisition value color_measure1. If the color acquisition value color_measure1 is greater than the color sampling value color_samples1, then construct a color mapping table color_map1 for reducing the color sampling value color_samples1; if the color acquisition value color_measure1 is less than the color sampling value color_samples1, then construct a color mapping table color_map1 for increasing the color sampling value color_samples1.

[0059] The color mapping table color_map1 satisfies color_samples1 = H_color(color_map1(color_samples1)), where H_color() represents the function of the color system of the overall screen and image acquisition device on the on-screen picture, and it is the inverse function of the color mapping table color_map1. In this way, through the dual action of the color mapping table color_map1 and the inverse function of the color mapping table color_map1, the color change of the on-screen picture caused by the color system of the overall screen and image acquisition device can be offset for the target picture corresponding to color_samples1.

[0060] Combined with Figure 2It can be understood that in this embodiment, the color correction of the on-screen image is first performed using the color mapping table color_map1. In this way, the color_samples1 obtained by solving based on the color of the target image and color_map1 is input to the virtual shooting screen. After color_samples1 is rendered on the screen, the rendered image is then captured by the image capture system to obtain H_color(color_map1(color_samples1)). Thus, the color change of the on-screen image caused by the overall color system of the screen and the image acquisition device is offset, so that the same optical response as the original on-screen image can be obtained, and the color of the captured image is accurate. Therefore, the virtual shooting screen and the image acquisition device can be controlled so that the captured image displayed by the image acquisition device when shooting the image presented on the virtual shooting screen can display the color as desired.

[0061] According to this embodiment, in the initial screen calibration stage, the standard color generator is used as the input (color_samples1), and the image acquisition device is used to measure the output (color_measure1) to ensure the accuracy of the calculated and analyzed color mapping table. Based on this, the first color mapping table is determined. In the shooting stage, the color correction of the target image to be presented on the screen is performed according to the first color mapping table, and the captured image is obtained using the color-corrected target image. In this way, the accuracy of the color of the captured image can be ensured.

[0062] In a possible implementation, generating the first color sampling points using the standard color generator as the first input color sampling values includes: obtaining the arrangement mode of the first color sampling points corresponding to the target image; and causing the standard color generator to generate the corresponding first color sampling points according to the arrangement mode.

[0063] In this embodiment, the arrangement mode of the first color sampling points may include, but is not limited to, the number and / or distribution position of the first color sampling points (for example, the distribution position in the screen display image). The arrangement mode of the first color sampling points can be preset according to the type and / or scene of the on-screen image for color sampling. For example, if the on-screen image of a certain virtual scene is a grassland displayed at a certain position, then multiple green sampling points that can cover the grassland can be arranged at that position.

[0064] Thus, more refined screen calibration can be performed.

[0065] In a possible implementation, the arrangement mode is related to the accuracy and time required for color correction.

[0066] In this embodiment, the denser the arrangement of the first color sampling points, the higher the accuracy of the first color mapping table constructed using the dense first color sampling points. Correspondingly, the higher the accuracy of color correction. Conversely, the sparser the arrangement of the first color sampling points, the lower the accuracy of the first color mapping table constructed using the sparse first color sampling points. Correspondingly, the lower the accuracy of color correction. Additionally, the denser the arrangement of the first color sampling points, the more time it takes to construct the first color mapping table based on the dense first color sampling points. Conversely, the sparser the arrangement of the first color sampling points, the less time it takes to construct the first color mapping table based on the sparse first color sampling points. Therefore, when constructing the first color mapping table, the arrangement method of the first color sampling points can be determined according to the accuracy and / or time required for color correction.

[0067] Thus, it is possible to reduce the correction time while ensuring a certain correction accuracy.

[0068] In a possible implementation manner, step S130 may include: solving for the target composite color according to the color of the captured image and the second color mapping table; adjusting the color of the extended image to the target composite color.

[0069] In this embodiment, since the mapping relationship between the color of the captured image and the color of the extended image can be determined according to the second color mapping table, and the color systems of the screen and the image acquisition device will change the color of the target image, resulting in the color of the captured image may not match the color of the extended image. Therefore, in order to obtain an XR image with the colors of the real captured image and the extended image matching, it is necessary to solve for the corresponding color of the extended image according to the second color mapping table and the color of the captured image as the target composite color.

[0070] After solving for the target composite color, the color of the extended image can be adjusted to the target composite color. In this way, the color of the extended image is adjusted to the target composite color. Thus, after correcting the color of the extended image to the target composite color and fusing the corrected extended image and the captured image, an XR image with the colors of the real captured image and the extended image matching can be obtained.

[0071] In a possible implementation, a second color mapping table can be pre-constructed. The following method can be used to pre-construct the second color mapping table: Use a standard color generator to generate second color sampling points as second input color sampling values; Use the first color mapping table to perform color correction on the second color sampling points; Render the color-corrected second color sampling points on the screen, and use the image acquisition device to collect the color obtained after the color conversion of the color-corrected second color sampling points through the screen as the second color acquisition value; Analyze the second input color sampling value and the second color acquisition value to construct the second color mapping table representing the mapping relationship between the second input color sampling value and the second color acquisition value, where the mapping relationship between the second input color sampling value and the second color acquisition value corresponds to the mapping relationship between the color of the captured picture and the composite color of the extended picture.

[0072] In this embodiment, as Figure 2 shown, in the post-calibration screen measurement stage, use a standard color generator to generate each color sampling point (second color sampling point) of the standard sampling color to obtain the input color sampling value color_samples2 (second input color sampling value), so as to ensure that the input color conforms to the standard definition and is unique. The input color sampling value color_samples2 is sent to the color analysis system.

[0073] Then, as Figure 2 shown, in the post-calibration screen measurement stage, via the color correction system, use the first color mapping table (color mapping table 1) constructed in the initial screen calibration stage to perform color correction on each color sampling point generated by the standard color generator. Each color sampling point after color correction is rendered on the virtual shooting screen via the on-screen interface. The color after screen rendering is the color obtained after the color conversion of each color sampling point after color correction through the screen. Collect the color of the picture presented on the screen via an image acquisition device such as a camera to collect the picture after screen rendering to obtain the color acquisition value color_measure2 (second color acquisition value). The color acquisition value color_measure2 is sent to the color analysis system.

[0074] Next, as Figure 2 shown, in the post-calibration screen measurement stage, via the color analysis system, perform statistical analysis on the color sampling value color_samples2 and the color acquisition value color_measure2, and a function representing the mapping relationship between the color sampling value color_samples2 and the color acquisition value color_measure2 can be constructed as the color mapping table color_map2 (second color mapping table).

[0075] That is to say, in the post-school test screen stage, according to the screen brightening, dimming, turning red, turning green, etc., through the relationship between the color sampling value color_samples2 and the color acquisition value color_measure2, dense sampling of multiple points can be carried out, and the color mapping relationship (i.e., the second color mapping table) between the color sampling value color_samples2 and the color acquisition value color_measure2 should be able to be constructed. In this way, after the second color mapping relationship is constructed, the color of the extended screen can be specified.

[0076] Exemplarily, the color sampling value color_samples2 and the color acquisition value color_measure2 are compared. If the color acquisition value color_measure2 is greater than the color sampling value color_samples2, a color mapping table color_map2 for increasing the color sampling value color_samples2 is constructed; if the color acquisition value color_measure2 is less than the color sampling value color_samples2, a color mapping table color_map2 for decreasing the color sampling value color_samples2 is constructed.

[0077] The color mapping table color_map2 satisfies color_measure2 = color_map2(color_samples2). In this way, since in the part where the color correction is accurate, color_map2 is almost a unit mapping and does not change the color of the extended screen, the accuracy of the color in the part where the color correction is accurate can be guaranteed; at the same time, in the part where the color correction is inaccurate (such as areas with insufficient color gamut, etc.), color_map2 can pull up the color of the extended screen to the color actually displayed by the overall color system of the screen and the image acquisition device (the color of the captured screen), so that the color matching between the part of the captured screen with inaccurate color correction and the extended screen can be achieved, and thus there will be no revealing of flaws.

[0078] According to this embodiment, in the post-school test screen stage, a standard color generator is used as the input (color_samples2), and an image acquisition device is used to measure the output (color_measure2) to ensure the accuracy of the calculated and analyzed color mapping table. Based on this, the second color mapping table is determined. In the shooting stage, the extended screen is color-corrected according to the second color mapping table, and an XR screen is obtained by using the color-corrected extended screen and the captured screen. In this way, the color matching degree between the captured screen and the extended screen can be guaranteed.

[0079] Exemplarily, in combination with Figure 2, in the initial screen calibration stage, a first color mapping table, i.e., color mapping table 1, is pre-constructed; in the post-calibration screen testing stage, a second color mapping table, i.e., color mapping table 2, is pre-constructed; in the shooting stage, first, the color mapping table 1 is used to perform color correction processing on the on-screen image, and then it is rendered on the screen via the on-screen interface. The image acquisition device acquires the rendered image and the real scene to obtain the captured image. At the same time, the color mapping table 2 is also used to perform color correction processing on the extended image, and then the corrected extended image is fused with the captured image. In this way, an XR image that takes into account both the color accuracy and matching degree in the XR scene can be obtained.

[0080] In a possible implementation, generating the second input color sampling values by using a standard color generator to generate second color sampling points includes: obtaining the arrangement mode of the second color sampling points corresponding to the extended image; and making the standard color generator generate corresponding second color sampling points according to the arrangement mode.

[0081] In this embodiment, the arrangement mode of the second color sampling points may include, but is not limited to, the number and / or distribution position of the second color sampling points. The arrangement mode of the second color sampling points can be preset according to the type of the extended image and / or the scene sampling color. Thereby, more refined screen calibration can be performed.

[0082] In a possible implementation, the arrangement mode is related to the accuracy and time required for color correction.

[0083] In this embodiment, the denser the second color sampling points are arranged, the higher the accuracy of the second color mapping table constructed by using the dense second color sampling points, and correspondingly, the higher the accuracy of color correction; conversely, the sparser the second color sampling points are arranged, the lower the accuracy of the second color mapping table constructed by using the sparse second color sampling points, and correspondingly, the lower the accuracy of color correction. In addition, the denser the second color sampling points are arranged, the more time it takes to construct the second color mapping table based on the dense second color sampling points; conversely, the sparser the second color sampling points are arranged, the less time it takes to construct the second color mapping table based on the sparse second color sampling points. Therefore, when constructing the second color mapping table, the arrangement mode of the second color sampling points can be determined according to the accuracy and / or time required for color correction.

[0084] Thereby, it is possible to reduce the correction time while ensuring a certain correction accuracy.

[0085] Figure 3 Shows a block diagram of a color correction device for extended reality shooting according to an exemplary embodiment of the present disclosure. As Figure 3As shown, the color correction device 300 may include a first color correction module 310, a first acquisition module 320, a second color correction module 330, and a second acquisition module 340.

[0086] The first color correction module 310 is configured to perform color correction on a target image to be presented on the screen according to a first color mapping table, where the first color mapping table represents a mapping relationship between an input color of the screen and a target color to be displayed on the screen, and the target color is a color that can be obtained by color acquisition of the image to be presented on the screen via an image acquisition device. The first acquisition module 320 is connected to the first color correction module 310 and is configured to obtain a captured image according to the target image after color correction.

[0087] The second color correction module 330 is configured to perform color correction on an extended image according to a second color mapping table, where the second color mapping table represents a mapping relationship between the color of the captured image and the composite color of the extended image. The second acquisition module 340 is connected to the first acquisition module 320 and the second color correction module 330 and is configured to composite the captured image and the extended image after color correction to obtain an extended reality image.

[0088] In a possible implementation manner, the first color correction module 310 is configured to: determine a target color to be displayed on the screen; solve a first target input color according to the first color mapping table and the target color; and adjust the color of the target image to the first target input color.

[0089] In a possible implementation manner, the above color correction device 300 may further include: a first construction module (not shown) configured to construct the first color mapping table, where the first construction module is configured to: use a standard color generator to generate first color sampling points as first input color sampling values; render the first color sampling points on the screen, and use the image acquisition device to acquire the color obtained after color conversion of the first color sampling points by the screen as first color acquisition values; analyze the first input color sampling values and the first color acquisition values to construct the first color mapping table representing the mapping relationship between the first input color sampling values and the first color acquisition values, where the mapping relationship between the first input color sampling values and the first color acquisition values corresponds to the mapping relationship between the input color and the target color.

[0090] In a possible implementation manner, the first acquisition module 320 is configured to: render the target image after color correction on the screen; and capture the rendered image and the real scene image to obtain the captured image.

[0091] In a possible implementation, the second color correction module 330 is configured to: solve a target composite color according to the color of the captured image and the second color mapping table; and adjust the color of the extended image to the target composite color.

[0092] In a possible implementation, the above color correction device may further include: a second construction module (not shown) for constructing the second color mapping table, where the second construction module is configured to: generate second color sampling points using a standard color generator as second input color sampling values; perform color correction on the second color sampling points using the first color mapping table; render the color-corrected second color sampling points on the screen, and use the image acquisition device to collect the color obtained after color conversion of the color-corrected second color sampling points passing through the screen as second color acquisition values; analyze the second input color sampling values and the second color acquisition values to construct the second color mapping table representing the mapping relationship between the second input color sampling values and the second color acquisition values, where the mapping relationship between the second input color sampling values and the second color acquisition values corresponds to the mapping relationship between the color of the captured image and the composite color of the extended image.

[0093] In some embodiments, the functions or modules included in the device provided in the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0094] The embodiments of the present disclosure also propose a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the above color correction method is implemented. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0095] The embodiments of the present disclosure also propose a color correction device for extended reality shooting, including: a processor; and a memory for storing instructions executable by the processor; where the processor is configured to implement the above color correction method when executing the instructions stored in the memory.

[0096] Figure 4 A schematic structural diagram of an electronic device according to an embodiment of the present disclosure is shown. Please refer to Figure 4, the electronic device 60 may include a processor 61 and a memory 62. Exemplarily, the processor 61, the memory 62, and each part are interconnected through a bus 63. The memory 62 stores computer-executable instructions; the processor 61 executes the computer-executable instructions stored in the memory 62, so that the processor 61 executes the color correction method for extended reality shooting as shown in the above method embodiments.

[0097] The embodiments of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to technologies in the market, or to enable other ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A color correction method for extended reality shooting, characterized in that Including: Performing color correction on a target image to be presented on a screen according to a first color mapping table, where the first color mapping table represents a mapping relationship between an input color of the screen and a target color to be displayed on the screen, and the target color is a color that can be obtained by collecting the color of the image to be presented on the screen through an image acquisition device; Obtaining a captured image according to the target image after color correction; Performing color correction on an extended image according to a second color mapping table, where the second color mapping table represents a mapping relationship between the color of the captured image and the composite color of the extended image; Composite the captured image and the extended image after color correction to obtain an extended reality image.

2. The color correction method according to claim 1, wherein Performing color correction on a target image to be presented on a screen according to a first color mapping table includes: Determining a target color to be displayed on the screen; Solving a first target input color according to the first color mapping table and the target color; Adjusting the color of the target image to the first target input color.

3. The color correction method according to claim 1, characterized in that, Before performing color correction on a target image to be presented on a screen according to a first color mapping table, the color correction method further includes: Using a standard color generator to generate a first color sampling point as a first input color sampling value; Rendering the first color sampling point on the screen, and using the image acquisition device to collect the color obtained after the color conversion of the first color sampling point through the screen as a first color acquisition value; Analyzing the first input color sampling value and the first color acquisition value to construct the first color mapping table for representing the mapping relationship between the first input color sampling value and the first color acquisition value, where the mapping relationship between the first input color sampling value and the first color acquisition value corresponds to the mapping relationship between the input color and the target color.

4. The color correction method according to claim 1, wherein Obtaining a captured image according to the target image after color correction includes: Rendering the target image after color correction on the screen; Capturing the rendered image and the real scene image to obtain the captured image.

5. The color correction method according to claim 1, wherein Performing color correction on an extended image according to a second color mapping table includes: Solving a target composite color according to the color of the captured image and the second color mapping table; Adjusting the color of the extended image to the target composite color.

6. The color correction method according to claim 1, characterized in that, Before performing color correction on an extended image according to a second color mapping table, the color correction method further includes: Using a standard color generator to generate a second color sampling point as a second input color sampling value; Performing color correction on the second color sampling point using the first color mapping table; Rendering the second color sampling point after color correction on the screen, and using the image acquisition device to collect the color obtained after the color conversion of the second color sampling point after color correction through the screen as a second color acquisition value; Analyze the second input color sampling value and the second color acquisition value to construct the second color mapping table for representing the mapping relationship between the second input color sampling value and the second color acquisition value, wherein the mapping relationship between the second input color sampling value and the second color acquisition value corresponds to the mapping relationship between the color of the captured image and the composite color of the extended image.

7. A color correction device for extended reality shooting, characterized in that, Comprising: A first color correction module for performing color correction on a target image to be presented on the screen according to a first color mapping table, wherein the first color mapping table represents the mapping relationship between the input color of the screen and the target color to be displayed on the screen, and the target color is a color that can be obtained by color acquisition of the image to be presented on the screen by an image acquisition device; A first acquisition module for obtaining a captured image according to the target image after color correction; A second color correction module for performing color correction on an extended image according to a second color mapping table, wherein the second color mapping table represents the mapping relationship between the color of the captured image and the composite color of the extended image; A second acquisition module for synthesizing the captured image and the extended image after color correction to obtain an extended reality image.

8. The color correction device according to claim 7, wherein The first color correction module is configured to: Determine the target color to be displayed on the screen; Solve for a first target input color according to the first color mapping table and the target color; Adjust the color of the target image to the first target input color.

9. A color correction device for extended reality shooting, characterized in that, Comprising: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to implement the color correction method according to any one of claims 1 to 6 when executing the instructions stored in the memory.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that, The computer program instructions, when executed by the processor, implement the color correction method according to any one of claims 1 to 6.

Images