Color correction method, device, storage medium and equipment based on extended reality

Abstract

The present application discloses a color correction method, device, storage medium and equipment based on extended reality, which relates to the field of extended display technology, and its main purpose is to solve the problem of obvious color difference when existing extended reality images are combined. It includes: obtaining the target extended area of ​​the extended image to be synthesized by extended reality; calling the first color mapping relationship corresponding to the target extended area, the first color mapping relationship stores the extended correction color corresponding to the target extended area, the extended correction color is obtained by interpolating the extended color of the target extended area based on the second color mapping relationship and the third color mapping relationship, the second color mapping relationship is constructed based on the display color and the acquisition color of the display area, and the third color mapping relationship is obtained after inverse mapping based on the display color and the acquisition color; correcting the target extended area based on the first color mapping relationship.

Description

Technical Field

[0001] The present application relates to the field of extended display technology, and in particular to a color correction method, device, storage medium and equipment based on extended reality. Background Art

[0002] Extended Reality (XR) virtual shooting is a technology that uses extended reality technology for film and television production, including virtual reality (VR), augmented reality (AR) and mixed reality (MR). Through the integrated application of these technologies, users can create immersive scenes in the virtual world and achieve three-dimensional visual effects. Among them, XR virtual shooting can present 3D digital scenes in real time through LED screens, providing users with an immersive experience for film and television creation.

[0003] At present, in order to obtain a better virtual-real fusion effect, the color matching between the real shot and the extended picture is very important. Usually, a color conversion function is used to adjust the color of the real shot and the extended picture. However, the color conversion function still causes a large color difference when the virtual and real are combined, and the boundary is more obvious when the virtual and real are combined, which cannot meet the high realism requirements of the virtual and real combination, and greatly affects the shooting effect of the extended realization. Summary of the invention

[0004] In view of this, the present application provides a color correction method, device, storage medium and equipment based on extended reality, the main purpose of which is to solve the problem of obvious color differences when combining existing extended reality images.

[0005] According to one aspect of the present application, a color correction method based on extended reality is provided, comprising:

[0006] Acquire a target extended area of ​​an extended image to be synthesized into an extended reality;

[0007] Retrieve a first color mapping relationship corresponding to the target extended area, wherein the first color mapping relationship stores an extended correction color corresponding to the target extended area, wherein the extended correction color is obtained by interpolating the extended color of the target extended area based on a second color mapping relationship and a third color mapping relationship, wherein the second color mapping relationship is constructed based on a display color and a collection color of a display area of ​​an LED display screen, and the third color mapping relationship is obtained by inverse mapping the display color and the collection color;

[0008] The target extended area is corrected based on the first color mapping relationship, so as to perform extended reality image synthesis based on the corrected extended image.

[0009] Furthermore, before calling the first color mapping relationship corresponding to the target extension area, the method further includes:

[0010] Acquire an extended color of the extended area, and perform a first interpolation calculation on the extended color based on the third color mapping relationship to obtain a first color value;

[0011] Performing a second interpolation calculation on the first color value based on the second color mapping relationship to obtain an extended correction color;

[0012] A color pair is constructed based on the extended color and the extended corrected color to generate the first color mapping relationship.

[0013] Furthermore, before obtaining the extended color of the extended area, the method further includes:

[0014] Acquire regional display images obtained by photographing multiple display areas in the LED display screen by a collection device, wherein one display area corresponds to one regional display image;

[0015] Determine the display color of the display area, and determine the acquisition color of the image displayed in the area. The display color is the color displayed by the LED display screen in the display area through the color generator;

[0016] Constructing a color pair based on the display color and the collected color, and generating the second color mapping relationship;

[0017] An inverse mapping calculation is performed on the second color mapping relationship to generate the third color mapping relationship.

[0018] Further, the determining the acquisition color of the regional display image includes:

[0019] Determine the position of the area center point based on the area outline of the target display area;

[0020] The center point color is extracted from the area display image based on the position of the area center point, and the center point color is determined as the acquisition color.

[0021] Furthermore, the display image of the display area includes a plurality of pixel points, and after performing inverse mapping calculation on the second color mapping relationship to generate the third color mapping relationship, the method further includes:

[0022] Binding and configuring the third color mapping relationship with each pixel in the display area to retrieve the third color mapping relationship according to the pixel;

[0023] or,

[0024] Acquire a central pixel point in the displayed image, and bind and configure the third color mapping relationship with the central pixel point;

[0025] The third color mapping relationship is interpolated or fitted according to the associated pixel points of the central pixel point to obtain a fourth color mapping relationship of the associated pixel point, and the fourth color mapping relationship is bound and configured with the associated pixel point to call the third color mapping relationship according to the central pixel point and to call the fourth color mapping relationship according to the associated pixel point.

[0026] Further, the correcting the target extension area based on the first color mapping relationship includes:

[0027] Determine an extended correction color matching the target extended area based on the first color mapping relationship;

[0028] Correcting the color displayed by the color generator in the extended image based on the extended correction color; or

[0029] The color displayed in the extended image is corrected based on the difference between the target extended area and the extended correction color.

[0030] Furthermore, after correcting the target extension area based on the first color mapping relationship, the method further includes:

[0031] Acquire a photographed image obtained by virtually photographing the corrected display image, wherein the display image is obtained by performing color cast correction based on the third color mapping relationship;

[0032] The captured image is synthesized with the extended image to obtain an extended reality image.

[0033] Furthermore, the method further comprises:

[0034] When a change in the target display area for correcting the displayed image is detected, a third color mapping relationship matching the changed display area is retrieved;

[0035] The display image is color corrected based on the matched third color mapping relationship, and the display area after color correction is resynthesized with the extended image to obtain a changed extended reality image.

[0036] Furthermore, before calling the third color mapping relationship for changing the display area matching, the method further includes:

[0037] Obtain real-time station information through positioning equipment;

[0038] When it is detected that the changed station position of the real-time station information does not match the station position of the target display area, it is determined that the target display area is changed, and the changed display area is determined based on the changed station position.

[0039] Furthermore, before acquiring the display image corrected based on the third color mapping relationship, the method further includes:

[0040] Acquire the station position through a positioning device, and determine at least one display area matching the station position based on a preset angle-area mapping relationship, wherein the preset angle-area mapping relationship includes a mapping relationship between different station positions and display areas at different angles;

[0041] A third color mapping relationship matching the display area is retrieved, and color cast correction is performed on the display area based on the third color mapping relationship.

[0042] According to one aspect of the present application, a color correction device based on extended reality is provided, comprising:

[0043] An acquisition module, used to acquire a target extended area of ​​an extended image to be synthesized into an extended reality;

[0044] A calling module, used for calling a first color mapping relationship corresponding to the target extended area, wherein the first color mapping relationship stores an extended correction color corresponding to the target extended area, and the extended correction color is obtained by interpolating the extended color of the target extended area based on a second color mapping relationship and a third color mapping relationship, wherein the second color mapping relationship is constructed based on a display color and a collection color of a display area of ​​an LED display screen, and the third color mapping relationship is obtained by inverse mapping the display color and the collection color;

[0045] A correction module is used to correct the target extended area based on the first color mapping relationship, so as to perform extended reality image synthesis based on the corrected extended image.

[0046] According to one aspect of the present application, a computer-readable storage medium is provided, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to perform operations corresponding to the color correction method based on augmented reality as described above.

[0047] According to one aspect of the present application, a terminal is provided, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus;

[0048] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above-mentioned color correction method based on extended reality.

[0049] By means of the above technical solution, the technical solution provided by the embodiment of the present application has at least the following advantages:

[0050] The present application provides a color correction method, apparatus, storage medium and device based on extended reality. Compared with the prior art, the embodiments of the present application construct a first color mapping relationship including extended correction colors of each extended area, so as to perform color correction on the target extended area based on the first color mapping relationship. Since the extended correction color in the first color mapping relationship is constructed based on the extended color, the display color of the display area and the collected color, it is possible to integrate the color information of the display area during the color correction of the extended area, thereby ensuring the consistency and accuracy of the extended image and the captured image obtained by capturing the display image as the real image when performing color synthesis, greatly reducing the boundary difference between the virtual and the real, achieving more accurate seamless virtual and real fusion, meeting the flexibility, accuracy and realism requirements of the virtual and the real, and thus improving the shooting effect of the extended reality.

[0051] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0053] Figure 1 A flow chart of a color correction method based on extended reality provided by an embodiment of the present application is shown;

[0054] Figure 2 A schematic diagram of a display area provided in an embodiment of the present application is shown;

[0055] Figure 3 A flow chart of another color correction method based on extended reality provided by an embodiment of the present application is shown;

[0056] Figure 4 A schematic diagram of a multi-angle matching display area provided in an embodiment of the present application is shown;

[0057] Figure 5A schematic diagram of a user station provided in an embodiment of the present application is shown;

[0058] Figure 6 A block diagram of a color correction device based on extended reality provided by an embodiment of the present application is shown;

[0059] Figure 7 A schematic diagram of the structure of a terminal provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0060] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0061] The present application embodiment provides a color correction method based on augmented reality, such as Figure 1 As shown, the method includes:

[0062] 101. Obtain a target extension area of ​​an extended image to be synthesized into an extended reality.

[0063] In the embodiment of the present application, the current execution subject may be a server or display control device that performs extended reality synthesis on the extended image generated by the LED display screen. The LED display screen is a screen that displays images by controlling semiconductor light emitting diodes. It is composed of tens of thousands to hundreds of thousands of semiconductor light emitting diode pixels arranged evenly to form a display screen for displaying various information such as text, graphics, images, animations, market conditions, videos, and video signals. Among them, the extended image is an image displayed on the screen by a color generator on the LED display screen. At this time, the color in the image can be represented based on the red (Red), green (Green), and blue (Blue) (hereinafter referred to as RGB) color channel values. Therefore, the target extended area is the smallest unit of the image color extracted in the extended image, including but not limited to the screen size area, the box structure area, the module unit area, and the lamp bead distribution area, etc., and the implementation of this application does not make specific limitations.

[0064] It should be noted that the extended image in the embodiment of the present application is an image generated according to different virtual and real scenes, and is displayed through the LED screen. The image content is not specifically limited in the embodiment of the present application. In addition, when obtaining the target extended area of ​​the extended image, the target extended area can be one or more, and the embodiment of the present application does not specifically limit it.

[0065] 102. Retrieve a first color mapping relationship corresponding to the target extension area.

[0066] In an embodiment of the present application, after the current execution subject obtains the target extension area, it calls up the pre-configured first color mapping relationship. At this time, a target extension area is bound to a matching first color mapping relationship. Among them, the first color mapping relationship stores the extended correction color corresponding to the target extension area. Specifically, the extended correction color is obtained by interpolating the extended color of the target extension area based on the second color mapping relationship and the third color mapping relationship. At this time, the second color mapping relationship is constructed based on the display color and the collected color of the display area of ​​the LED display screen, and the third color mapping relationship is obtained after inverse mapping the display color and the collected color. In an embodiment of the present application, the LED display screen includes multiple display areas. The display area is used to characterize the smallest unit for extracting screen color, including but not limited to the screen size area, the box structure area, the module unit area, and the lamp bead distribution area. In an embodiment of the present application, the display area and the extension area may be the same or different. The display area is preferably as follows Figure 2 The box structure area shown is not specifically limited in the embodiments of the present application. In addition, the extended color is the color in the area of ​​the extended image in the LED screen for extended reality shooting, and the acquisition color is determined based on the regional display image obtained by the acquisition device shooting the display area. Specifically, the current execution subject pre-configures the acquisition device at the position where the user may be, and uses the acquisition device to shoot the target display area to obtain a regional display image. At this time, the regional display image is used to characterize the image obtained by shooting the target display area, and then, the acquisition color is extracted based on the regional display image. In addition, the display color is the color displayed by the LED display screen through the color generator in the target display area. If the target display area is the same as the target extended area, the display color is the same as the extended color, which is not specifically limited in the embodiments of the present application.

[0067] 103. Correct the target extended area based on the first color mapping relationship, so as to perform extended reality image synthesis based on the corrected extended image.

[0068] In the embodiment of the present application, since the first color mapping relationship includes the extended correction color corresponding to the target extended area, the current execution subject uses this extended correction color to perform color correction on the target extended area. Specifically, since the color displayed in the LED screen is obtained by the change of the RGB color channel value of each pixel in the extended image and their superposition with each other to achieve different color effects, therefore, when performing correction, the RGB value of each pixel can be corrected based on the matching extended correction color. At this time, the RGB channel value of the extended image to be displayed in the target extended area can be directly used to cover or replace the RGB channel value of the extended image to be displayed in the target extended area to achieve the correction purpose. The embodiment of the present application does not make specific limitations.

[0069] The embodiment of the present application provides a color correction method based on extended reality. Compared with the prior art, the embodiment of the present application constructs a first color mapping relationship including extended correction colors of each extended area, so as to perform color correction on the target extended area based on the first color mapping relationship. Since the extended correction color in the first color mapping relationship is constructed based on the extended color, the display color of the display area and the collected color, it is possible to integrate the color information of the display area during the color correction of the extended area, thereby ensuring the consistency and accuracy of the extended image and the captured image obtained by capturing the display image as the real image when performing color synthesis, greatly reducing the boundary difference between the virtual and the real, achieving more accurate seamless virtual and real fusion, meeting the flexibility, accuracy and realism requirements of the virtual and the real, and thus improving the shooting effect of the extended reality.

[0070] The present application embodiment provides another color correction method based on extended reality, such as Figure 3 As shown, the method includes:

[0071] 201. Acquire regional display images obtained by photographing multiple display areas in the LED display screen with a collection device.

[0072] In order to improve the image fusion effect in virtual and real shooting, for the upper screen picture displayed on the LED display screen as a real image and combined with the extended image, the color cast correction of the upper screen picture can be processed by constructing a second color mapping relationship. Among them, the current execution subject pre-acquires the regional display images corresponding to different display areas. Specifically, the acquisition device is placed at the selected shooting position. At this time, since the shooting position is the position where the user may stand, the display area can be different for the same shooting position and different shooting angles. Therefore, when shooting the display area, shooting the display area at one angle obtains a corresponding regional display image, and shooting the display area at multiple angles obtains multiple corresponding regional display images, that is, one display area corresponds to one regional display image, and one shooting position matches multiple display areas, such as Figure 4 As shown, a display area is bound to a color mapping relationship, which is not specifically limited in the present embodiment. In addition, the acquisition device can be a device with a shooting function, including but not limited to a camera or a video camera, which is not specifically limited in the present embodiment.

[0073] It should be noted that, in the embodiments of the present application, in order to ensure effective shooting of the display area, the position of each standing position should correspond to the maximum range of the display area, that is, when the acquisition device is in the shooting position, the shooting range can completely cover each display area. The shooting position can also be determined according to stage scheduling or any shooting position. The embodiments of the present application do not specifically limit the shooting position.

[0074] 202. Determine a display color of the display area, and determine a collection color of an image displayed in the area.

[0075] In order to realize the correction of the display color in the upper screen image of the LED screen by using the color captured by the camera, thereby improving the effectiveness of the color cast correction, the current execution subject determines the display color of the display area and the acquisition color of the regional display image obtained by the capture. Among them, the display color can be the color displayed by the LED display screen in the display area through the color generator, which can be represented based on the RGB channel value, and the embodiment of the present application does not make specific limitations.

[0076] In another embodiment of the present application, for further definition and explanation, the step of determining the acquisition color of the regional display image includes:

[0077] Determine the position of the area center point based on the area outline of the target display area;

[0078] The center point color is extracted from the area display image based on the position of the area center point, and the center point color is determined as the acquisition color.

[0079] In order to quickly extract the color information in the regional display image, so as to reduce the data processing pressure of constructing a color mapping relationship for multiple regional display images and ensure the effectiveness of the corrected color information, the current execution subject first determines the regional center point position of the target display area when determining the acquisition color. Specifically, since the display area can include a screen size area, a box structure area, a module unit area, and a lamp bead distribution area, etc., the shape of each display area can be a regular shape or an irregular shape, preferably a regular form. At this time, the regional center point position can be determined based on the regional outline of the target display area, for example, the regional center point position is determined based on the regional outline of the box structure area, and the embodiment of the present application does not make specific limitations. After determining the position of the regional center point, extract the center point color of the center point of this region. At this time, since the regional display image is in image form, the center point position extracted in the regional display image can be represented as a center point pixel, that is, the corresponding center point color is the color of the center pixel point, which can be represented as an RGB channel value and determined as the acquisition color of the entire target display area. The embodiment of the present application does not make specific limitations.

[0080] 203. Construct a color pair based on the display color and the collected color to generate the second color mapping relationship.

[0081] In order to improve the accuracy of color cast correction, the current execution entity constructs the display color and the collected color into a color pair, that is, a target display area corresponds to a regional display image and a display color, a regional display image extracts a collected color, and the display color and the collected color are constructed into a color pair. At this time, the mapping relationship of the color pair can be directly generated as a second color mapping relationship.

[0082] 204. Perform inverse mapping calculation on the second color mapping relationship to generate the third color mapping relationship.

[0083] In order to realize the color cast correction of the upper screen image as the real image quickly and effectively based on the third color mapping relationship, after the display color and the collected color are constructed into a color pair, the color pair (second color mapping relationship) is inversely mapped and calculated to obtain the third color mapping relationship. At this time, the inverse mapping calculation is an inverse LUT. LUT (Look-Up Table) is a mapping component that maps input color values ​​to output color values, which is used for image color mapping and adjustment. Correspondingly, the inverse LUT is to achieve a specific effect by reversing the LUT to obtain the third color mapping relationship, which is not specifically limited in the embodiments of the present application. Among them, the solution method for LUT or inverse LUT can adopt triangulation, neural network fitting, affine transformation, etc., which is not specifically limited in the embodiments of the present application.

[0084] It should be noted that since a shooting position can capture regional display images of the display area at multiple angles, when the third color mapping relationship is represented by a mapping list, a shooting position can be configured with a color mapping table of the display area corresponding to multiple angles, and the embodiment of the present application does not make specific limitations.

[0085] In another embodiment of the present application, for further definition and explanation, after performing inverse mapping calculation on the second color mapping relationship to generate the third color mapping relationship, the method further includes:

[0086] The third color mapping relationship is bound and configured with each pixel in the display area.

[0087] Since the display image of the display area includes multiple pixel points, in order to reduce the image processing speed and ensure the correction of screen display color cast at different viewing angles, the current executing entity, after obtaining the third color mapping relationship of the display image serving as the upper screen, binds this third color mapping relationship to all the pixel points in the display image of the display area, so that after determining the target display area, the matching third color mapping relationship is directly called to correct the color values ​​in all the pixel points of the display image. The embodiment of the present application does not make specific limitations.

[0088] In another embodiment of the present application, for further definition and explanation, after performing inverse mapping calculation on the second color mapping relationship to generate the third color mapping relationship, the method further includes:

[0089] Acquire a central pixel point in the displayed image, and bind and configure the third color mapping relationship with the central pixel point;

[0090] The third color mapping relationship is interpolated or fitted according to the associated pixel points of the central pixel point to obtain a fourth color mapping relationship of the associated pixel point, and the fourth color mapping relationship is bound and configured with the associated pixel point to call the third color mapping relationship according to the central pixel point and to call the fourth color mapping relationship according to the associated pixel point.

[0091] Since the display image of the display area includes multiple pixels, in order to achieve the purpose of color cast correction for the display image of the display area, so as to meet the requirements of the LED display screen to display a more immersive picture, the current execution subject applies the third color mapping relationship after obtaining the third color mapping relationship, first obtains the central pixel of the display image of the display area, and at this time, binds and configures the central pixel with the obtained third color mapping relationship, so that after determining the display area, the central pixel of the display image is corrected for color cast according to the third color mapping relationship. For other corresponding pixels, the current execution subject determines the associated pixel of the central pixel, and interpolates or fits the color mapping relationship according to the correlation of each associated pixel, obtains the associated color mapping relationship corresponding to each associated pixel, and performs corresponding binding configuration in turn, so as to call the corresponding associated color mapping relationship according to the associated pixel to correct the color of the associated pixel, which is not specifically limited in the embodiment of the present application.

[0092] It should be noted that the interpolation calculation in the embodiment of the present application can be trilinear interpolation, tetrahedral interpolation, etc., and the fitting calculation can be linear regression fitting, quadratic regression fitting, etc., which are not specifically limited in the embodiment of the present application. In addition, for the associated pixel points, they can be extended to the surrounding area with the central pixel point as the center, including the distance, proximity, etc. as the correlation, as the parameters for interpolation or fitting of the associated pixel points determined according to the extended distance, so as to obtain the associated color mapping relationship matching all the pixel points in the display area, which is not specifically limited in the embodiment of the present application.

[0093] 205 . Acquire an extended color of the extended area, and perform a first interpolation calculation on the extended color based on the third color mapping relationship to obtain a first color value.

[0094] 206. Perform a second interpolation calculation on the first color value based on the second color mapping relationship to obtain an extended correction color.

[0095] 207. Construct a color pair based on the extended color and the extended corrected color to generate the first color mapping relationship.

[0096] In order to achieve the purpose of efficient color correction of the extended image in the extended reality, after obtaining the extended color of the extended area, the extended color is interpolated based on the third color mapping relationship to obtain the first color value. At this time, since the corresponding colors in the extended color or the third color mapping relationship are all represented by RGB color channel values, the first interpolation calculation is performed on the extended color, that is, the RGB channel value of the extended color is input into the third color mapping table for interpolation calculation to obtain the first color value. Furthermore, using the same calculation method, the obtained first color value is input into the second color mapping table for interpolation calculation to obtain the extended correction color used to construct the first color mapping relationship. At this time, the extended color of the original extended area and the extended correction color obtained by interpolation calculation are used to construct a color pair to obtain the first color mapping relationship. The embodiment of the present application does not make specific limitations.

[0097] It should be noted that the interpolation calculation in the embodiment of the present application can adopt trilinear interpolation, tetrahedral interpolation, etc., without specific limitation. In addition, when the current execution end determines the extended area of ​​the LED display screen, it can determine multiple extended areas. Therefore, when constructing the first color mapping relationship, each extended area corresponds to at least one matching first color mapping relationship.

[0098] 208. Obtain a target extension area of ​​an extended image to be synthesized into an extended reality.

[0099] In the embodiment of the present application, step 208 is the same as step 101 and will not be repeated here.

[0100] 209. Retrieve a first color mapping relationship corresponding to the target extension area.

[0101] In the embodiment of the present application, step 209 is the same as step 102 and will not be repeated here.

[0102] 210. Determine an extended correction color that matches the target extended area based on the first color mapping relationship.

[0103] 211a. Correct the color displayed by the color generator in the extended image based on the extended correction color.

[0104] In a specific implementation scenario, in order to achieve the purpose of effectively and accurately performing color correction on an extended image, when performing correction, the current executing entity first determines a matching extended correction color based on a first color mapping relationship, so as to correct the color displayed in the extended image through a color generator of an LED screen. At this time, the extended correction color can be used to directly replace the color in the extended image, for example, directly replacing the RGB color channel value of the extended correction color with the RGB color channel value to be displayed by the color generator. This embodiment of the present application does not make specific limitations.

[0105] In another embodiment of the present application, step 211b parallel to step 211a is to correct the color displayed in the extended image based on the difference between the target extended area and the extended correction color.

[0106] In a specific implementation scenario, in order to achieve the purpose of effectively and accurately correcting the color of the extended image, when performing the correction, the current execution subject first determines the matching extended correction color based on the first color mapping relationship, so as to correct the color displayed in the extended image through the color generator of the LED screen. At this time, since the color displayed in the LED screen is represented by the RGB color channel value, the difference between the RGB color channel value of the target extended area and the extended correction color can be calculated first, and then the RGB color channel value displayed by the color generator in the extended area on the LED screen can be corrected based on the difference. For example, the RGB color channel value displayed in the extended image is increased or decreased based on the difference to complete the correction, which is not specifically limited in the embodiments of the present application.

[0107] 212. Acquire a captured image obtained by virtually capturing the corrected display image.

[0108] In order to meet the flexible combination requirements of the display image and the extended image in the extended reality scene and ensure color accuracy and consistency, after the correction of the extended image is completed, the captured image obtained by shooting the corrected display image displayed on the LED screen is obtained. At this time, the display image is obtained by color cast correction based on the third color mapping relationship, and the embodiment of the present application does not make specific limitations. In addition, the display image includes a complete image obtained by combining multiple target display areas that have completed color cast correction with other display areas, and then the display image is virtually photographed using a shooting device to obtain a captured image for extended display synthesis.

[0109] 213. Synthesize the captured image and the extended image to obtain an extended reality image.

[0110] In order to achieve the realism of the fusion effect between the extended and real images and ensure the color consistency of the synthesized extended reality image, so as to show a seamless visualization effect, the current execution subject synthesizes the captured image obtained after the color cast correction with the corrected extended image to obtain the extended reality image, so as to achieve the realistic effect of transition in the synthesized image. Among them, the synthesis method may include but is not limited to overlay or splicing, which is not specifically limited in the embodiments of the present application.

[0111] In another embodiment of the present application, for further definition and explanation, the steps further include:

[0112] When a change in the target display area for correcting the displayed image is detected, a third color mapping relationship matching the changed display area is retrieved;

[0113] The display image is color corrected based on the matched third color mapping relationship, and the display area after color correction is resynthesized with the extended image to obtain a changed extended reality image.

[0114] In order to meet the flexibility of camera / camcorder mobile shooting and the synthesis requirements of extended reality to ensure the timeliness of extended reality, the current execution subject detects in real time whether the target display area has changed. When it is determined that the target display area has changed, the current execution subject re-calls the third color mapping relationship that matches the changed display area, so as to re-color correct the display image of the changed target display area based on the third color mapping relationship, and then re-synthesize the display area and the extended image based on the color correction to obtain the changed extended reality image. Among them, re-color correcting the display image of the changed target display area based on the third color mapping relationship is the same as the aforementioned method of color cast correction using the third color mapping relationship, which will not be repeated here.

[0115] Correspondingly, in another embodiment, the current execution entity can also detect whether the target expansion area has changed, so as to call the first color mapping relationship that matches the target expansion area to re-correct the changed expansion area when the target expansion area has changed, so as to re-synthesize based on the corrected expansion image, which is not specifically limited in the embodiments of the present application.

[0116] It should be noted that, since the extended area and the display area can be the same or different, in the same scenario, the acquisition of the extended area or the detection of the change of the extended area can be carried out in the same way as the determination or detection of the display area, and the embodiments of the present application do not make specific limitations. In another different scenario, since the extended image can be generated in advance based on the requirements of the extended implementation, the extended area can be changed based on the change of the extended content in the extended image, or based on the change of the area displayed by the designated LED display screen, therefore, it can be determined by the image generation system based on the change of the extended image content, and the embodiments of the present application do not make specific limitations.

[0117] In another embodiment of the present application, for further definition and explanation, before the step of calling the third color mapping relationship for changing the display area matching, the method further includes:

[0118] Obtain real-time station information through positioning equipment;

[0119] When it is detected that the changed station position of the real-time station information does not match the station position of the target display area, it is determined that the target display area is changed, and the changed display area is determined based on the changed station position.

[0120] In order to enable users to experience the effect of immersive extended reality virtual shooting on the LED screen during movement, thereby improving the effectiveness of color cast correction, the current execution subject first determines whether the display area corresponding to the user's moving position has changed when correcting the moved display area using the third color mapping relationship. Specifically, the current execution subject can obtain the real-time station information of the user's movement based on the positioning device (camera, infrared sensor). Specifically, after obtaining the real-time position information, determine this real-time station information, that is, change the station position. At this time, if the changed station position can match the original target display area, color cast correction does not need to be performed again. If the changed station position cannot match the original target display area, it means that the user has moved a large distance. Therefore, based on this change in station position, the display area is changed, and color cast correction can be re-performed based on the changed display area, which greatly improves the effectiveness of color cast correction of the LED screen during the user's movement.

[0121] It should be noted that in the embodiment of the present application, since the user can see different display areas according to different viewing angles at one standing position, one standing position can be pre-configured with multiple angles and the display area corresponding to each angle. For example, after the camera is in the shooting position (the same as the standing position), different angles are adjusted to shoot the display area of ​​the LED display screen, thereby establishing a corresponding matching relationship, so as to quickly determine the display area for color cast correction during the user's movement.

[0122] In another embodiment of the present application, for further definition and explanation, before the step of obtaining the display image corrected based on the third color mapping relationship, the method further includes:

[0123] Acquire the station position through a positioning device, and determine at least one display area matching the station position based on a preset angle-area mapping relationship;

[0124] A third color mapping relationship matching the display area is retrieved, and color cast correction is performed on the display area based on the third color mapping relationship.

[0125] In order to meet the color cast correction requirements of the LED display screen's extended reality combined with multiple scenes, for the scene where the user moves in front of the LED screen, the current execution subject can collect the standing position through the positioning device when acquiring the target display area. The standing position is the position where the user stands in front of the LED display screen, which can be the same as the aforementioned shooting position, such as Figure 5 As shown, the positioning device can be a camera or an infrared sensor. When it is determined that the user is standing at the station position, the target display area seen at different angles at this station position is determined based on the preset angle area mapping relationship. At this time, the preset angle area mapping relationship includes the mapping relationship between different station positions and target display areas at different angles, which is not specifically limited in the embodiment of the present application. Then, a third color mapping relationship matching this display area is retrieved, and the display area is corrected for color cast based on the third color mapping relationship to obtain a display image to be combined with augmented reality after correction.

[0126] It should be noted that the preset angle area mapping relationship can be pre-configured, that is, different angles are shot in advance at the standing position, and the display area that can be shot at multiple angles is used as the target display area bound to this standing position. In addition, different angles can be configured based on different immersive space experience effects, such as starting with 0 degrees straight and matching a display area with every 30 degrees of rotation, etc. Figure 5 As shown, the embodiments of the present application are not specifically limited.

[0127] The embodiment of the present application provides another color correction method based on extended reality, which can integrate the color information of the display area when correcting the color of the extended area, thereby ensuring the consistency and accuracy of the color synthesis of the extended image and the captured image obtained by capturing the display image as the real image, greatly reducing the boundary difference between the virtual and the real, and achieving more accurate seamless virtual and real fusion, meeting the flexibility, accuracy and realism requirements of the virtual and the real, thereby improving the shooting effect of the extended reality.

[0128] Furthermore, as a response to the above Figure 1 The implementation of the method shown in the figure, the embodiment of the present application provides a color correction device based on extended reality, such as Figure 6 As shown, the device comprises:

[0129] An acquisition module 31 is used to acquire a target expansion area of ​​an extended image to be synthesized into an extended reality;

[0130] A calling module 32 is used to call a first color mapping relationship corresponding to the target extended area, wherein the first color mapping relationship stores an extended correction color corresponding to the target extended area, and the extended correction color is obtained by interpolating the extended color of the target extended area based on a second color mapping relationship and a third color mapping relationship, wherein the second color mapping relationship is constructed based on the display color and the collected color of the display area of ​​the LED display screen, and the third color mapping relationship is obtained by inverse mapping the display color and the collected color;

[0131] The correction module 33 is used to correct the target extended area based on the first color mapping relationship, so as to perform extended reality image synthesis based on the corrected extended image.

[0132] Furthermore, the device further comprises: a calculation module, a generation module,

[0133] The acquisition module is further used to acquire an extended color of the extended area, and perform a first interpolation calculation on the extended color based on the third color mapping relationship to obtain a first color value;

[0134] The calculation module is used to perform a second interpolation calculation on the first color value based on the second color mapping relationship to obtain an extended correction color;

[0135] The generating module is used to construct a color pair based on the extended color and the extended corrected color to generate the first color mapping relationship.

[0136] Furthermore, the device further comprises: a determination module,

[0137] The acquisition module is further used to acquire regional display images obtained by photographing multiple display areas in the LED display screen by a collection device, and one display area corresponds to one regional display image;

[0138] The determination module is used to determine the display color of the display area and determine the acquisition color of the image displayed in the area. The display color is the color displayed by the LED display screen in the display area through the color generator;

[0139] The generating module is further used to construct a color pair based on the display color and the collected color to generate the second color mapping relationship;

[0140] The generating module is further used to perform inverse mapping calculation on the second color mapping relationship to generate the third color mapping relationship.

[0141] Furthermore, the determination module is specifically used to determine the position of the area center point based on the area outline of the target display area; extract the center point color from the area display image based on the area center point position, and determine the center point color as the acquisition color.

[0142] Further, the determination module, wherein the display image of the display area includes a plurality of pixel points, is specifically used to bind and configure the third color mapping relationship with each pixel point in the display area so as to call the third color mapping relationship according to the pixel point; or, obtain a central pixel point in the display image, and bind and configure the third color mapping relationship with the central pixel point; interpolate or fit the third color mapping relationship according to the associated pixel points of the central pixel point to obtain a fourth color mapping relationship of the associated pixel points, and bind and configure the fourth color mapping relationship with the associated pixel points so as to call the third color mapping relationship according to the central pixel point and call the fourth color mapping relationship according to the associated pixel points.

[0143] Furthermore, the correction module is specifically used to determine an extended correction color that matches the target extended area based on the first color mapping relationship; correct the color displayed by the color generator in the extended image based on the extended correction color; or correct the color displayed in the extended image based on the difference between the target extended area and the extended correction color.

[0144] Furthermore, the device further comprises: a synthesis module,

[0145] The acquisition module is further used to acquire a photographed image obtained by virtually photographing the corrected display image, wherein the display image is obtained by performing color cast correction based on the third color mapping relationship;

[0146] The synthesis module is used to synthesize the captured image and the extended image to obtain an extended reality image.

[0147] Further,

[0148] The calling module is further used to call a third color mapping relationship matching the changed display area when detecting that the target display area for correcting the displayed image is changed;

[0149] The synthesis module is further used to perform color correction on the display image based on the matched third color mapping relationship, and re-synthesize the display area after color correction with the extended image to obtain a changed extended reality image.

[0150] Further,

[0151] The acquisition module is also used to acquire real-time station information through a positioning device;

[0152] The determination module is further configured to determine that the target display area is changed when it is detected that the changed station position of the real-time station information does not match the station position of the target display area, and determine the changed display area based on the changed station position.

[0153] Further,

[0154] The acquisition module is further used to acquire the station position through the positioning device, and determine at least one display area matching the station position based on a preset angle-area mapping relationship, wherein the preset angle-area mapping relationship includes a mapping relationship between different station positions and display areas at different angles;

[0155] The calling module is further used to call a third color mapping relationship that matches the display area, and perform color cast correction on the display area based on the third color mapping relationship.

[0156] The embodiment of the present application provides a color correction device based on extended reality. Compared with the prior art, the embodiment of the present application constructs a first color mapping relationship including extended correction colors of each extended area, so as to perform color correction on the target extended area based on the first color mapping relationship. Since the extended correction color in the first color mapping relationship is constructed based on the extended color, the display color of the display area and the collected color, it is possible to integrate the color information of the display area during the color correction of the extended area, thereby ensuring the consistency and accuracy of the extended image and the captured image obtained by capturing the display image as the real image when performing color synthesis, greatly reducing the boundary difference between the virtual and the real, achieving more accurate seamless virtual and real fusion, meeting the flexibility, accuracy and realism requirements of the virtual and the real, and thus improving the shooting effect of the extended reality.

[0157] According to an embodiment of the present application, a computer-readable storage medium is provided, wherein the storage medium stores at least one executable instruction, and the computer-executable instruction can execute the color correction method based on extended reality in any of the above method embodiments.

[0158] Figure 7 A schematic diagram of the structure of a terminal provided according to an embodiment of the present application is shown, and the specific embodiment of the present application does not limit the specific implementation of the terminal.

[0159] like Figure 7 As shown, the terminal may include: a processor (processor) 402 , a communication interface (Communications Interface) 404 , a memory (memory) 406 , and a communication bus 408 .

[0160] The processor 402 , the communication interface 404 , and the memory 406 communicate with each other via a communication bus 408 .

[0161] The communication interface 404 is used to communicate with other devices such as clients or other servers.

[0162] The processor 402 is used to execute the program 410, and specifically can execute the relevant steps in the above-mentioned embodiment of the color correction method based on extended reality.

[0163] Specifically, the program 410 may include program codes, which include computer operation instructions.

[0164] The processor 402 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the terminal may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0165] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0166] The program 410 may be specifically configured to enable the processor 402 to perform the following operations:

[0167] Acquire a target extended area of ​​an extended image to be synthesized into an extended reality;

[0168] Retrieve a first color mapping relationship corresponding to the target extended area, wherein the first color mapping relationship stores an extended correction color corresponding to the target extended area, wherein the extended correction color is obtained by interpolating the extended color of the target extended area based on a second color mapping relationship and a third color mapping relationship, wherein the second color mapping relationship is constructed based on a display color and a collection color of a display area of ​​an LED display screen, and the third color mapping relationship is obtained by inverse mapping the display color and the collection color;

[0169] The target extended area is corrected based on the first color mapping relationship, so as to perform extended reality image synthesis based on the corrected extended image.

[0170] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0171] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A color correction method based on extended reality, characterized in that: include: Acquire a target extended area of ​​an extended image to be synthesized into an extended reality; Retrieve a first color mapping relationship corresponding to the target extended area, wherein the first color mapping relationship stores an extended correction color corresponding to the target extended area, wherein the extended correction color is obtained by interpolating the extended color of the target extended area based on a second color mapping relationship and a third color mapping relationship, wherein the second color mapping relationship is constructed based on a display color and a collection color of a display area of ​​an LED display screen, and the third color mapping relationship is obtained by inverse mapping the display color and the collection color; The target extended area is corrected based on the first color mapping relationship, so as to perform extended reality image synthesis based on the corrected extended image.

2. The method according to claim 1, characterized in that Before calling the first color mapping relationship corresponding to the target extension area, the method further includes: Acquire an extended color of the extended area, and perform a first interpolation calculation on the extended color based on the third color mapping relationship to obtain a first color value; Performing a second interpolation calculation on the first color value based on the second color mapping relationship to obtain an extended correction color; A color pair is constructed based on the extended color and the extended corrected color to generate the first color mapping relationship.

3. The method according to claim 2, characterized in that Before obtaining the extended color of the extended area, the method further includes: Acquire regional display images obtained by photographing multiple display areas in the LED display screen by a collection device, wherein one display area corresponds to one regional display image; Determine the display color of the display area, and determine the acquisition color of the image displayed in the area. The display color is the color displayed by the LED display screen in the display area through the color generator; Constructing a color pair based on the display color and the collected color, and generating the second color mapping relationship; An inverse mapping calculation is performed on the second color mapping relationship to generate the third color mapping relationship.

4. The method according to claim 3, characterized in that Determining the acquisition color of the regional display image includes: Determine the position of the area center point based on the area outline of the target display area; The center point color is extracted from the area display image based on the position of the area center point, and the center point color is determined as the acquisition color.

5. The method according to claim 3, characterized in that: The display image of the display area includes a plurality of pixels. After performing inverse mapping calculation on the second color mapping relationship to generate the third color mapping relationship, the method further includes: Binding and configuring the third color mapping relationship with each pixel in the display area to retrieve the third color mapping relationship according to the pixel; or, Acquire a central pixel point in the displayed image, and bind and configure the third color mapping relationship with the central pixel point; The third color mapping relationship is interpolated or fitted according to the associated pixel points of the central pixel point to obtain a fourth color mapping relationship of the associated pixel point, and the fourth color mapping relationship is bound and configured with the associated pixel point to call the third color mapping relationship according to the central pixel point and to call the fourth color mapping relationship according to the associated pixel point.

6. The method according to claim 1, characterized in that The correcting the target extension area based on the first color mapping relationship includes: Determine an extended correction color matching the target extended area based on the first color mapping relationship; Correcting the color displayed by the color generator in the extended image based on the extended correction color; or, The color displayed in the extended image is corrected based on the difference between the target extended area and the extended correction color.

7. The method according to claim 1, characterized in that After correcting the target extension area based on the first color mapping relationship, the method further includes: Acquire a photographed image obtained by virtually photographing the corrected display image, wherein the display image is obtained by performing color cast correction based on the third color mapping relationship; The captured image is synthesized with the extended image to obtain an extended reality image.

8. The method according to claim 7, characterized in that The method further comprises: When a change in the target display area for correcting the displayed image is detected, a third color mapping relationship matching the changed display area is retrieved; The display image is color corrected based on the matched third color mapping relationship, and the display area after color correction is resynthesized with the extended image to obtain a changed extended reality image.

9. The method according to claim 8, characterized in that Before calling the third color mapping relationship for changing the display area matching, the method further includes: Obtain real-time station information through positioning equipment; When it is detected that the changed station position of the real-time station information does not match the station position of the target display area, it is determined that the target display area is changed, and the changed display area is determined based on the changed station position.

10. The method according to claim 7, characterized in that Before acquiring the display image corrected based on the third color mapping relationship, the method further includes: Acquire the station position through a positioning device, and determine at least one display area matching the station position based on a preset angle-area mapping relationship, wherein the preset angle-area mapping relationship includes a mapping relationship between different station positions and display areas at different angles; A third color mapping relationship matching the display area is retrieved, and color cast correction is performed on the display area based on the third color mapping relationship.

11. A color correction device based on extended reality, characterized in that: include: An acquisition module, used to acquire a target extended area of ​​an extended image to be synthesized into an extended reality; A calling module, used for calling a first color mapping relationship corresponding to the target extended area, wherein the first color mapping relationship stores an extended correction color corresponding to the target extended area, and the extended correction color is obtained by interpolating the extended color of the target extended area based on a second color mapping relationship and a third color mapping relationship, wherein the second color mapping relationship is constructed based on a display color and a collection color of a display area of ​​an LED display screen, and the third color mapping relationship is obtained by inverse mapping the display color and the collection color; A correction module is used to correct the target extended area based on the first color mapping relationship, so as to perform extended reality image synthesis based on the corrected extended image.

12. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 10.

Images