Data processing method, device, apparatus, and computer storage medium

Abstract

Embodiments of the present application provide a data processing method, device and equipment and a computer storage medium. The data processing method comprises: determining a reference acquisition device and a compensation acquisition device in a plurality of acquisition devices, the reference acquisition device being an acquisition device that has realized virtual-real color alignment; obtaining a color mapping relationship between the reference acquisition device and the compensation acquisition device; the color mapping relationship representing a mapping relationship between a reference color value obtained by the reference acquisition device after picture acquisition and a compensation color value obtained by the compensation acquisition device after picture acquisition for the same display color; and based on the color mapping relationship, performing image color compensation in the process of virtual-real picture acquisition by the compensation acquisition device to obtain a compensated image. Embodiments of the present application can realize virtual-real color alignment in the compensation acquisition device, thereby improving the shooting quality of virtual shooting.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method, apparatus, electronic device, computer storage medium, and computer program product. Background Technology

[0002] XR (Extended Reality) virtual filming involves projecting a rendered scene image, obtained through a virtual engine, onto an LED screen. Actors then use the LED screen as a background for their performances. Image capture devices (such as cameras) simultaneously capture images of both the actors and the LED screen. The captured camera footage (images) is then combined with the rendered scene image, placing real actors within a virtual environment to achieve the effect of shooting outdoor scenes or science fiction backdrops in a studio.

[0003] Due to factors such as those within the image acquisition equipment, there are inherent differences between the color values ​​in the original rendered scene image obtained by the virtual rendering engine and the color values ​​in the final image acquired by the image acquisition equipment, resulting in some color distortion. Therefore, color calibration is typically performed on both the original rendered scene image and the final image acquired by the image acquisition equipment to eliminate these differences. Furthermore, to avoid color discrepancies between the actual scene and the virtual scene captured by the image acquisition equipment, further color adjustments are made to the virtual rendering engine to achieve color alignment between the real and virtual scenes, thereby improving the shooting effect.

[0004] In actual shooting, due to shooting requirements, different image acquisition devices are usually set up in different locations to capture images. These images are then edited and post-processed to produce a final product that meets the requirements. However, in multi-device acquisition scenarios, there are often image acquisition devices that cannot achieve true-to-life color alignment. Summary of the Invention

[0005] In view of this, embodiments of this application provide a data processing scheme to at least partially solve the above-mentioned problems.

[0006] According to a first aspect of the embodiments of this application, a data processing method is provided, including:

[0007] A reference acquisition device and a compensation acquisition device are identified among multiple acquisition devices. The reference acquisition device is an acquisition device that has achieved virtual and real color alignment. The compensation acquisition device is any acquisition device other than the reference acquisition device.

[0008] Obtain the color mapping relationship between the reference acquisition device and the compensation acquisition device; the color mapping relationship represents the mapping relationship between the reference color value obtained by the reference acquisition device after image acquisition for the same display color and the compensation color value obtained by the compensation acquisition device after image acquisition.

[0009] Based on the color mapping relationship, color compensation is performed on the acquired image during the virtual and real image acquisition process of the compensation acquisition device to obtain the compensated image.

[0010] According to a second aspect of the embodiments of this application, a data processing apparatus is provided, comprising:

[0011] The device determination module is used to determine the reference acquisition device and the compensation acquisition device among multiple acquisition devices. The reference acquisition device is an acquisition device that has achieved virtual and real color alignment; the compensation acquisition device is other acquisition devices besides the reference acquisition device.

[0012] The mapping relationship acquisition module is used to acquire the color mapping relationship between the reference acquisition device and the compensation acquisition device; the color mapping relationship represents the mapping relationship between the reference color value obtained by the reference acquisition device after image acquisition and the compensation color value obtained by the compensation acquisition device after image acquisition for the same display color.

[0013] The compensation module is used to perform color compensation on the acquired image during the virtual and real image acquisition process of the compensation acquisition device based on the color mapping relationship, so as to obtain the compensated image.

[0014] According to a third aspect of the present application, an electronic device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, wherein the executable instruction causes the processor to perform an operation corresponding to the method described in the first aspect.

[0015] According to a fourth aspect of the embodiments of this application, a computer storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0016] According to the data processing scheme provided in this application embodiment, a reference acquisition device that has achieved virtual-real color alignment and a compensation acquisition device that has not yet achieved virtual-real alignment are determined from multiple acquisition devices used for virtual shooting. Then, the color mapping relationship between the reference acquisition device and the compensation acquisition device is obtained. Based on this color mapping relationship, color compensation is performed on the acquired image obtained by the compensation acquisition device from virtual-real image acquisition, thereby obtaining a compensated image with virtual-real color alignment. In this application embodiment, by utilizing the color mapping relationship between the reference acquisition device and the compensation acquisition device, the difference in spectral understanding between the reference acquisition device and the compensation acquisition device is applied to the compensation acquisition device, thereby effectively reducing the color difference between virtual content and real-world content in the image acquired by the compensation acquisition device. Virtual-real color alignment is also achieved in the compensation acquisition device, improving the shooting quality of virtual shooting.

[0017] In addition, compared to the method of performing virtual rendering engine color calibration and color adjustment steps separately for each of the multiple image acquisition devices to achieve virtual-real color alignment, the virtual-real alignment scheme provided in this application embodiment only needs to perform virtual rendering engine color calibration and adjustment steps for the reference acquisition device, without having to repeat the virtual rendering engine color calibration and adjustment steps for the other devices. Therefore, through this application embodiment, the shooting process can be made simpler and the shooting difficulty can be reduced.

[0018] Furthermore, many real-world shooting scenarios require simultaneous shooting with multiple devices. The aforementioned method of performing color calibration and adjustment operations for each individual device using virtual rendering can only achieve virtual-to-real alignment for one device at a given shooting moment, thus failing to meet the aforementioned requirements. In contrast, the solution in this application first achieves virtual-to-real alignment for the reference acquisition device, and then performs color compensation for the compensation acquisition device based on the color mapping relationship between the reference and compensation acquisition devices, thereby achieving virtual-to-real alignment for the compensation acquisition device. This better meets the aforementioned requirements, and the embodiments of this application are applicable to a wider range of scenarios. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0020] Figure 1 This is a flowchart of the steps of a data processing method according to an embodiment of this application;

[0021] Figure 2This is a schematic diagram of the virtual shooting link corresponding to the embodiments of this application;

[0022] Figure 3 This is a schematic diagram of the data processing flow according to an embodiment of this application;

[0023] Figure 4 This is a structural block diagram of a data processing apparatus according to an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0026] The specific implementation of the embodiments of this application will be further described below with reference to the accompanying drawings.

[0027] Reference Figure 1 , Figure 1 This is a flowchart illustrating the steps of a data processing method according to an embodiment of this application. Specifically, the data processing method provided in this embodiment includes the following steps:

[0028] Step 102: Determine the reference acquisition device and the compensation acquisition device among the multiple acquisition devices. The reference acquisition device is the acquisition device that has achieved virtual and real color alignment; the compensation acquisition device is the other acquisition device besides the reference acquisition device.

[0029] Specifically, in the virtual shooting process, multiple image acquisition devices are usually set up to capture images separately, and then the images captured by each image acquisition device are edited and post-processed to generate a product that meets the requirements.

[0030] During the shooting process, due to factors such as internal characteristics of the image acquisition equipment, there is a certain difference between the color values ​​in the original rendered scene image rendered by the virtual rendering engine and the color values ​​in the final image acquired by the image acquisition equipment, resulting in some color distortion. Therefore, color calibration is usually performed on both the color values ​​in the original rendered scene image rendered by the virtual rendering engine and the color values ​​in the final image acquired by the image acquisition equipment to eliminate the aforementioned differences. Furthermore, to avoid color differences between the actual scene and the virtual scene in the image captured by the image acquisition equipment, further adjustments are made to the parameters of the virtual rendering engine to achieve color alignment between the virtual and real scenes and improve the shooting effect. In other words, given a fixed image acquisition device, to ensure color alignment between the virtual and real scenes in the final image, the virtual rendering engine is typically calibrated first, and then its parameters are adjusted based on the color calibration to adjust the color values ​​of the rendered image, achieving color alignment between the virtual and real scenes.

[0031] During color calibration and virtual-real color alignment, a specific image acquisition device can be selected from multiple image acquisition devices (for ease of description, the selected image acquisition device is referred to as the reference acquisition device in this embodiment) to perform color calibration and virtual-real color alignment processing. This ensures that the image acquired by the reference acquisition device is color-aligned. In other words, regarding the reference acquisition device in this embodiment, color calibration has been performed beforehand on the color values ​​in the original rendered scene image rendered by the virtual rendering engine and the color values ​​in the image finally acquired by the device. Furthermore, color adjustment has been performed on the color values ​​in the original rendered scene image rendered by the virtual rendering engine, achieving color alignment between the color values ​​in the original rendered scene image rendered by the virtual rendering engine and the color values ​​in the actual scene. In summary, the reference acquisition device is a device that has already achieved virtual-real color alignment. The specific process of virtual-real color alignment of the reference acquisition device can be implemented based on relevant technologies.

[0032] Because different image acquisition devices have different understandings of the spectrum due to physical factors such as their internal photosensitive elements, after the virtual rendering engine is calibrated and aligned based on the benchmark acquisition device, there may still be color misalignment issues between the virtual and real images captured by the remaining acquisition devices (that is, the image acquisition devices that were not selected among the multiple image acquisition devices mentioned above).

[0033] Step 104: Obtain the color mapping relationship between the reference acquisition device and the compensation acquisition device.

[0034] Specifically, the color mapping relationship obtained in step 104 can characterize the mapping relationship between the reference color value obtained by the reference acquisition device after image acquisition and the compensated color value obtained by the compensation acquisition device after image acquisition for the same display color.

[0035] Step 106: Based on the color mapping relationship, color compensation is performed on the acquired image during the virtual and real image acquisition process of the compensation acquisition device to obtain the compensated image.

[0036] Specifically, as mentioned above, color calibration compensation can be performed based on a reference acquisition device. That is, the alignment of real and virtual colors can be achieved according to the understanding of the spectrum by the internal photosensitive elements of the reference acquisition device. The color mapping relationship between the reference acquisition device and the compensation acquisition device obtained in step 104 can characterize the degree of difference in their understanding of the spectrum. Therefore, after obtaining the aforementioned color mapping relationship, color compensation of the acquired image can be performed during the acquisition of real and virtual images by the compensation acquisition device, thereby obtaining a compensated image with real-virtual alignment.

[0037] XR virtual shooting involves projecting a rendered scene image, obtained through a virtual engine, onto a screen, and then deploying physical objects (such as actors and props) in front of the screen. An image acquisition device simultaneously captures images of both the physical objects and the rendered scene image on the screen. Therefore, it can be understood that the virtual-real image capture in this embodiment refers to the process of simultaneously capturing images of the rendered scene image (virtual object) displayed on the screen and the physical objects deployed in the actual space using an image acquisition device.

[0038] According to the data processing method provided in this application embodiment, a reference acquisition device that has achieved virtual-real color alignment and a compensation acquisition device that has not yet achieved virtual-real alignment are determined from multiple acquisition devices used for virtual shooting. Then, the color mapping relationship between the reference acquisition device and the compensation acquisition device is obtained. Based on this color mapping relationship, color compensation is performed on the acquired image obtained by the compensation acquisition device from virtual-real image acquisition, thereby obtaining a compensated image with virtual-real color alignment. In this application embodiment, by utilizing the color mapping relationship between the reference acquisition device and the compensation acquisition device, the difference in spectral understanding between the reference acquisition device and the compensation acquisition device is applied to the compensation acquisition device, effectively reducing the color difference between virtual content and real-scene content in the image acquired by the compensation acquisition device. Virtual-real color alignment is also achieved in the compensation acquisition device, improving the shooting quality of virtual shooting.

[0039] In addition, compared to the method of performing virtual rendering engine color calibration and color adjustment steps separately for each of the multiple image acquisition devices to achieve virtual-real color alignment, the virtual-real alignment scheme provided in this application embodiment only needs to perform virtual rendering engine color calibration and adjustment steps for the reference acquisition device, without having to repeat the virtual rendering engine color calibration and adjustment steps for the other devices. Therefore, through this application embodiment, the shooting process can be made simpler and the shooting difficulty can be reduced.

[0040] Furthermore, many real-world shooting scenarios require simultaneous shooting with multiple devices. The aforementioned method of performing color calibration and adjustment operations for each individual device using virtual rendering can only achieve virtual-to-real alignment for one device at a given shooting moment, thus failing to meet the aforementioned requirements. In contrast, the solution in this application first achieves virtual-to-real alignment for the reference acquisition device, and then performs color compensation for the compensation acquisition device based on the color mapping relationship between the reference and compensation acquisition devices, thereby achieving virtual-to-real alignment for the compensation acquisition device. This better meets the aforementioned requirements, and the embodiments of this application are applicable to a wider range of scenarios.

[0041] The data processing method of this embodiment can be executed by any suitable electronic device with data processing capabilities, including but not limited to: servers, PCs, etc.

[0042] Optionally, in some embodiments, the process of obtaining the color mapping relationship between the reference acquisition device and the compensation acquisition device may specifically include:

[0043] Acquire a baseline acquisition image, which is an image obtained by a baseline acquisition device capturing the target object; acquire a compensation acquisition image, which is an image obtained by a compensation acquisition device capturing the target object; acquire the baseline color value of the target object in the baseline acquisition image and the compensation color value of the target object in the compensation acquisition image; calculate the color mapping relationship between the baseline acquisition device and the compensation acquisition device based on the baseline color value and the compensation color value.

[0044] Specifically, the color mapping relationship represents the mapping between the reference color value obtained by the reference acquisition device and the compensated color value obtained by the compensation acquisition device for the same display color. Therefore, a target object with one or more color values ​​can be preset, and then the target object can be captured by both the reference acquisition device and the compensation acquisition device. The aforementioned color mapping relationship can then be obtained by using the color values ​​of the target object in the reference acquisition image and the compensation acquisition image, respectively.

[0045] Specifically, the color mapping relationship can be represented in the form of a color mapping lookup table, which can contain the correspondence between different base color values ​​and different compensation color values. After obtaining the color values ​​of the target object in the base acquisition image and the compensation acquisition image, the correspondence between the color values ​​of the target object in the base color image and the color values ​​of the target object in the compensation acquisition image can be used. If the color values ​​of the target object do not include all color values, the correspondence between other color values ​​besides the target object's color values ​​can be calculated using interpolation or other methods. This color mapping lookup table is then used as the color mapping relationship. Alternatively, the color mapping relationship can also be represented in the form of a matrix. For example, a base color matrix can be formed using the color values ​​of the target object in the base acquisition image, and a compensation color matrix can be formed using the color values ​​of the target object in the compensation color image. Then, the mapping matrix between the base color matrix and the compensation color matrix can be obtained, and this mapping matrix can be used as the aforementioned color mapping relationship.

[0046] The above process of calculating color mapping relationships establishes a benchmark comparison condition for color value mapping by introducing the same target object to calculate the color mapping relationship between the benchmark acquisition device and the compensation acquisition device. This ensures that the final calculated color mapping relationship is a mapping relationship for the same display color, thus improving the accuracy of the color mapping relationship calculation.

[0047] In this embodiment, the color values ​​of the target object and the number of color values ​​are not limited and can be customized according to actual conditions. For example, multiple target objects can be pre-set, each with a single color value, and different target objects may have different color values. In this case, the process of obtaining the color mapping relationship between the reference acquisition device and the compensation acquisition device can be executed separately for each target object, thus obtaining a color mapping relationship applicable to multiple different color values. In this method, since each target object has a single color value, the color complexity of the image obtained in a single mapping relationship calculation is low, and the noise is small, which helps improve the accuracy of the color mapping relationship calculation. Alternatively, a single target object can be pre-set, containing multiple different color values. In this case, the process of obtaining the color mapping relationship between the reference acquisition device and the compensation acquisition device can be executed once for this target object, thus obtaining a color mapping relationship applicable to multiple different color values. In this method, since the target object contains multiple color values, executing the above mapping relationship calculation process once yields color mapping relationships for multiple different colors, thus helping to improve the computational efficiency of the color mapping relationship calculation.

[0048] In addition, in the scenario of XR virtual shooting, the target object in the above embodiments of this application can be a virtual object rendered to the display screen by a virtual rendering engine, or a real-world object set in the actual scene, or it can include both virtual and real-world objects at the same time.

[0049] Optionally, in some embodiments, the target object may include: a real-world object and a virtual object displayed on the screen; correspondingly, the process of obtaining the reference color value of the target object in the reference acquisition image and the compensated color value of the target object in the compensated acquisition image may specifically include:

[0050] Obtain the baseline real-world color value of the real-world object in the baseline acquisition image, and the compensated real-world color value of the real-world object in the compensated acquisition image; obtain the baseline virtual color value of the virtual object in the baseline acquisition image, and the compensated virtual color value of the virtual object in the compensated acquisition image.

[0051] Correspondingly, the process of calculating the color mapping relationship between the reference acquisition device and the compensation acquisition device based on the reference color value and the compensation color value may specifically include:

[0052] Based on the baseline real-scene color value and the compensated real-scene color value, the real-scene color mapping relationship between the baseline acquisition device and the compensated acquisition device is calculated; based on the baseline virtual color value and the compensated virtual color value, the virtual color mapping relationship between the baseline acquisition device and the compensated acquisition device is calculated.

[0053] By integrating real-world color mapping relationships and virtual color mapping relationships, the color mapping relationship between the baseline acquisition device and the compensation acquisition device is obtained.

[0054] Specifically, regarding the color mapping relationship represented by a color mapping relationship lookup table, the specific process of fusing real-world color mapping relationships and virtual color mapping relationships can be as follows: First, determine the target reference real-world color value and the target reference virtual color value with the same value, as the target reference color value; second, determine the target compensation real-world color value corresponding to the target reference real-world color value from the real-world color mapping relationship lookup table, and determine the target compensation virtual color value corresponding to the target reference virtual color value from the virtual color mapping relationship lookup table; third, fuse the target compensation real-world color value and the target compensation virtual color value to obtain a color value fusion result; fourth, based on the above target reference color value and the above color value fusion result, construct a color mapping relationship lookup table as the color mapping relationship between the reference acquisition device and the compensation acquisition device. In this embodiment, the specific fusion method used when fusing the target compensation real-world color value and the target compensation virtual color value is not limited. For example, it can be an averaging method, or a weighted summation method based on preset weights, etc.

[0055] Regarding the color mapping relationship represented by a matrix, the specific process of fusing the real-world color mapping relationship and the virtual color mapping relationship can be as follows: Element-value fusion processing is performed on two matrix elements with the same position coordinates in both the real-world and virtual color mapping matrices, and the fusion result is used as the matrix element located at that position coordinate in the final color mapping matrix. In this embodiment, the specific processing method for the above element-value fusion processing is not limited. For example, it can be an averaging process, or a weighted summation process based on preset weights, etc.

[0056] In the embodiments described above, objects serving as benchmark comparison conditions are set in both the real-world and virtual scenes. This divides the color mapping relationship acquisition process into two branches: a real-world scene branch and a virtual scene branch, obtaining the color mapping relationship for each branch. Then, the color mapping relationships for each branch are fused and calculated to obtain the final color mapping relationship. This process considers both the color differences between images captured in the real-world scene and the color differences between images captured in the virtual scene. Therefore, the final color mapping relationship is better suited to XR virtual shooting scenarios that combine real and virtual elements, improving the accuracy of the color mapping relationship calculation.

[0057] Optionally, in some embodiments, the process of obtaining the reference color value of the target object in the reference acquisition image and the compensated color value of the target object in the compensated acquisition image may specifically include:

[0058] Determine whether the baseline acquired image and the compensated acquired image meet the preset image quality conditions respectively;

[0059] If both the baseline acquisition image and the compensation acquisition image meet the image quality requirements, then the baseline color value of the target object in the baseline acquisition image and the compensation color value of the target object in the compensation acquisition image are obtained respectively.

[0060] Furthermore, after determining whether the baseline acquired image and the compensated acquired image meet the preset image quality conditions, the data processing method may further include:

[0061] If the reference acquisition image does not meet the image quality requirements, return to the step of acquiring the reference acquisition image, and perform subsequent color mapping relationship calculation steps based on the reacquired reference acquisition image;

[0062] If the compensated image does not meet the image quality requirements, the process returns to the step of acquiring the compensated image, and subsequent color mapping relationship calculation steps are performed based on the reacquired compensated image.

[0063] Specifically, during image acquisition, factors such as the acquisition angle and ambient light can affect the quality of the acquired images. For example, the image brightness may be too high (above a preset brightness threshold) or too low (below a preset brightness threshold). Furthermore, for target objects containing multiple color values, the brightness values ​​of individual pixels in the acquired image may be too concentrated (with small differences in brightness values), or the acquired image may contain a limited number of color values. If subsequent calculations are performed based on these poor-quality images, the accuracy of the calculated color mapping relationship may be poor, or even impossible to obtain.

[0064] Therefore, in the above embodiments of this application, after obtaining the reference acquisition image and the compensation acquisition image, the image quality can be determined first. When both the reference acquisition image and the compensation acquisition image meet the image quality conditions, the subsequent calculation operation is then performed. This can effectively avoid the problem of low accuracy of calculation results caused by poor image quality. In other words, through the above process, the calculation accuracy of color mapping relationship can be effectively improved.

[0065] Optionally, in some embodiments, the process of performing color compensation on the acquired image during the acquisition of virtual and real images by the compensation acquisition device, based on color mapping relationships, to obtain the compensated image, may include:

[0066] Acquire the original compensated virtual-real image; the original compensated virtual-real image is the image obtained by the compensation acquisition device after color calibration of the virtual rendering engine according to the color calibration mapping relationship corresponding to the compensation acquisition device;

[0067] Based on the color mapping relationship, the color values ​​of each pixel in the original compensated virtual and real images are corrected and compensated to obtain the compensated image.

[0068] Specifically, regarding image acquisition devices, due to internal factors and other influences, there is a certain difference between the color values ​​in the original rendered scene image obtained by the virtual rendering engine and the color values ​​in the final image acquired by the image acquisition device. The color calibration mapping relationship corresponding to the image acquisition device represents the mapping relationship between the color values ​​in the original rendered scene image obtained by the virtual rendering engine and the color values ​​in the final image acquired by the image acquisition device. Specifically, the color calibration mapping relationship corresponding to the compensation acquisition device represents the mapping relationship between the color values ​​in the original rendered scene image obtained by the virtual rendering engine and the color values ​​in the final acquired image.

[0069] Specifically, the calculation process for the color calibration mapping relationship corresponding to the compensation acquisition device may include: inputting the original rendered color into the shooting system containing the compensation acquisition device to obtain the captured image, that is: rendering the original rendered color to the LED display screen through the broadcast control processor, and then capturing the LED display screen to obtain the captured image through the compensation acquisition device; and then constructing a forward LUT (Look Up Table) based on the original rendered color value and the color value of the captured image to obtain the color calibration mapping relationship f(x) corresponding to the compensation acquisition device.

[0070] The above process first uses the color calibration mapping relationship corresponding to the compensation acquisition device to perform color calibration on the virtual rendering engine. That is, using the color calibration mapping relationship corresponding to the compensation acquisition device (such as the forward LUT mentioned above), the calibrated rendering color corresponding to the original rendering color is found, thereby calibrating the original rendering color to obtain the calibrated rendering color, which is then rendered to the display screen for display. Next, the compensation acquisition device is used to capture virtual and real images of the display screen and physical objects placed outside the display screen to obtain the original compensated virtual and real image. Then, the color values ​​of each pixel in the original compensated virtual and real image are adjusted through the color mapping relationship between the reference acquisition device and the compensation acquisition device (e.g., by looking up a table or multiplying by a transformation matrix), thereby obtaining the virtual and real aligned compensated image. In the virtual image rendering stage of the above process, the color mapping relationship between the reference acquisition device and the compensation acquisition device is not introduced. Instead, the original color calibration scheme for the virtual rendering engine is maintained. Therefore, while achieving virtual and real alignment of the compensation acquisition device, the complexity of shooting operations caused by parameter changes to the virtual rendering engine during the shooting process can be effectively reduced.

[0071] Optionally, in other embodiments of this application, the process of performing color compensation on the acquired image during the acquisition of virtual and real images by the compensation acquisition device based on color mapping relationships to obtain the compensated image may also include:

[0072] Acquire the compensated virtual and real images captured by the compensation acquisition device as the compensated image;

[0073] Among them, the compensated virtual-real image is the image obtained by the compensation acquisition device after color calibration of the virtual rendering engine according to the color calibration mapping relationship and color mapping relationship corresponding to the compensation acquisition device, and then the virtual-real image is acquired.

[0074] Specifically, the process of obtaining the compensated image with virtual-real alignment described above involves, after obtaining the color mapping relationship between the reference acquisition device and the compensation acquisition device, color calibration of the original rendered color is performed using the color calibration mapping relationship corresponding to the compensation acquisition device during virtual image rendering in the virtual rendering engine. The calibrated rendered color is then adjusted again using the aforementioned color mapping relationship to obtain the adjusted rendered color, which is then rendered onto the display screen for display. The compensation acquisition device then performs virtual-real image acquisition on the display screen and physical objects placed outside the display screen to obtain the compensated image. This process introduces the color mapping relationship between the reference acquisition device and the compensation acquisition device during the virtual image rendering stage. The advantage of this is that the image obtained by the compensation acquisition device during subsequent virtual-real image acquisition is the compensated image with virtual-real alignment. In other words, after the compensation acquisition device outputs the acquired image, no further image post-processing is required, thus reducing the data processing volume of the data processing scheme provided in this application embodiment.

[0075] Optionally, in some embodiments, the process of correcting and compensating the color values ​​of each pixel in the original compensated virtual-real image according to the color mapping relationship to obtain the compensated image may further include:

[0076] The original compensated virtual and real images are downsampled to obtain the sampled image;

[0077] Based on the color mapping relationship, the color values ​​of each pixel in the sampled image are corrected and compensated to obtain the compensated sampled image;

[0078] Calculate the first pixel value difference between the compensated sampled image and the reference virtual-real image, and the second pixel value difference between the original compensated virtual-real image and the reference virtual-real image; the reference virtual-real image is the image obtained by the reference acquisition device through virtual-real image acquisition;

[0079] If the difference between the first pixel value and the difference between the second pixel value meet the preset difference condition, the color values ​​of each pixel in the original compensated virtual and real image are corrected and compensated according to the color mapping relationship to obtain the compensated image.

[0080] Specifically, from a theoretical perspective, compared to the original compensated image, using the calculated color mapping relationship to correct and compensate the color values ​​of each pixel in the original compensated image usually results in better alignment of the real and virtual images. However, due to factors such as computational accuracy, the compensation effect cannot usually be quantitatively known before the calculated color mapping relationship is used for image color compensation.

[0081] To address the aforementioned issue and avoid the problem that the alignment of the real and virtual images in the final compensated image is not significantly improved after calculating the color mapping relationship, in the above embodiments of this application, after obtaining the original compensated real and virtual image, the original compensated real and virtual image is first downsampled to obtain a sampled image with smaller data volume (i.e., lower resolution). Then, the subsequent image color value modification compensation operation is performed based on the sampled image, and the color difference data between the sampled image after color value correction compensation and the reference real and virtual image acquired by the reference acquisition device is quantitatively calculated. Afterwards, the magnitude of the color difference data determines whether to perform color value modification compensation on the complete original compensated real and virtual image. In the above embodiments of this application, downsampling reduces the amount of image data processing, thereby objectively quantifying the improvement in image quality before and after performing color value correction compensation with a smaller computational load.

[0082] Compared with the traditional method that relies on the subjective experience of the photographer to decide whether to perform color value correction and compensation, the above embodiments of this application realize the quantitative display of the image quality improvement effect, which helps the operator to decide whether to perform color value correction and compensation based on the actual situation.

[0083] In addition, in this embodiment, the first pixel value difference is used to characterize the degree of difference between the values ​​of each pixel point between the compensated sampled image and the reference virtual-real image, and the second pixel value difference is used to characterize the degree of difference between the values ​​of each pixel point between the original compensated virtual-real image and the reference virtual-real image. The preset difference condition can be whether the difference between the first pixel value difference and the second pixel value difference is large enough (for example, whether it is greater than a preset threshold). When the difference between the first pixel value difference and the second pixel value difference is large enough (for example, greater than the preset threshold), it indicates that the compensation effect is good. At this time, the actual compensation operation can be performed, that is, according to the color mapping relationship, the color values ​​of each pixel point in the original compensated virtual-real image are corrected and compensated to obtain the compensated image.

[0084] Furthermore, regardless of whether it is the original compensated virtual-real image, the compensated post-sampled image, or the reference virtual-real image, the image usually contains a large number of pixels, and the color values ​​of each pixel may not be the same. In view of the above, in the embodiments of this application, when calculating the pixel value difference, a reference object (which can be a region or one or more pixels) can be selected in advance from the virtual-real image, and the reference pixel corresponding to the reference object in the original compensated virtual-real image, the compensated post-sampled image, or the reference virtual-real image can be determined according to the position of the reference object in the image, or by object detection. Then, the first pixel value difference and the second pixel value difference are calculated based on the color values ​​of the reference pixels in the above three images.

[0085] For example, an original compensated virtual-real image can be acquired simultaneously using a compensation acquisition device, and a reference virtual-real image can be acquired using a reference acquisition device. The original compensated virtual-real image is downsampled to obtain a sampled image with lower resolution. Correction and compensation are then performed using color mapping relationships to obtain a compensated sampled image. Since the resolution of the compensated sampled image is lower than that of the reference virtual-real image, the correspondence between each pixel in the compensated sampled image and each region in the reference virtual-real image can be determined. For example, a pixel in the compensated sampled image corresponds to a region within a preset range surrounding a pixel at the same position in the reference virtual-real image. The difference between the pixel value of that pixel in the compensated sampled image and the average pixel value of the corresponding region in the reference virtual-real image is calculated. Finally, the differences corresponding to all pixels (or reference pixels) in the compensated sampled image are statistically analyzed (e.g., calculating the average, standard deviation, or other statistical values ​​of the differences) to obtain the first pixel value difference. Those skilled in the art should understand that the method for calculating the first pixel value difference is not limited to this. For example, the pixel value of a pixel in the compensated sampled image can be compared with the pixel value of pixels at the same or similar positions in the reference virtual-real image to calculate the difference, and then the difference can be statistically analyzed to obtain the first pixel value difference. A second pixel difference can be obtained in a similar manner.

[0086] In one application example, for the same shooting scene (e.g., the rendered scene image displayed on the screen is the same), a small number of original compensated virtual and real images and a reference virtual and real image can be collected first. The above judgment is then performed. If the difference between the first pixel value and the second pixel value meets the preset difference condition, the color values ​​of each pixel in the original compensated virtual and real image collected in that shooting scene are corrected and compensated. Otherwise, the correction and compensation of the original compensated virtual and real image collected in that shooting scene can be abandoned. After changing the shooting scene, the above process can be repeated.

[0087] See Figure 2 , Figure 2 This is a schematic diagram of the virtual shooting link corresponding to the embodiments of this application. The following is in conjunction with... Figure 2 A brief description of the specific execution process of the data processing method provided in the embodiments of this application is as follows:

[0088] The first step is to render a virtual object containing multiple different color values ​​(i.e., rich colors) using a virtual rendering engine, and then display the virtual object on the screen. For example, Figure 2 The virtual objects in the game include: virtual color swatches of different colors, as well as virtual objects such as tables and chairs.

[0089] The second step, corresponding to the first step, involves placing real-world objects with different color values ​​in the actual scene. For example, Figure 2 The real-world objects and virtual objects correspond to each other. The real-world objects also include: physical color cards of different colors, as well as physical items such as tables and chairs.

[0090] The third step involves adjusting the equipment parameters of the benchmark acquisition device to capture images, thereby obtaining benchmark acquisition images containing the aforementioned virtual and real-world objects. Next, the acquired benchmark acquisition images are evaluated for image quality (e.g., whether the image is too bright or too dark, or whether the color value richness in the image meets preset requirements, etc.), and benchmark acquisition images that meet the quality requirements are transmitted to the server. Figure 2 As shown, for example, the reference acquired image can be transmitted to the server in the form of an SDI (Serial Digital Interface) signal.

[0091] The fourth step involves adjusting the device parameters of the compensation acquisition equipment to capture images, thereby obtaining compensated acquisition images containing the aforementioned virtual and real-world objects. Next, the acquired compensated acquisition images are evaluated for image quality (e.g., whether the image is too bright or too dark, or whether the color richness in the image meets preset requirements, etc.), and compensated acquisition images that meet the quality requirements are transmitted to the server. Figure 2 As shown, for example, the compensated acquired image can be transmitted to the server in the form of an SDI signal.

[0092] The fifth step involves configuring the image data received by the server using a monitor or a mobile device connected to the service, specifying which image is the baseline image and which is the compensation image. Then, the server calculates the color mapping relationship between the baseline and compensation images based on the received baseline and compensation images. Figure 2 As shown, for example, color mapping relationships can be presented in the form of LUTs or matrices.

[0093] The sixth step involves performing color compensation on the acquired image during the acquisition of virtual and real images using the compensation acquisition device, based on the color mapping relationship calculated in the fifth step, to obtain a compensated image with better virtual-real alignment.

[0094] The seventh step, in order to help operators understand the image compensation effect more objectively and quantitatively, can also output the pixel value difference between the image acquired by the compensation acquisition device before color calibration compensation and the image acquired by the reference acquisition device, as well as the pixel value difference between the image acquired by the compensation acquisition device and the image acquired by the reference acquisition device after color calibration compensation.

[0095] See Figure 3 , Figure 3 This is a schematic diagram of the data processing flow according to an embodiment of this application. The following is further combined with... Figure 3A brief description of the specific steps and flow of the data processing method provided in this application example:

[0096] First, establish the physical link; for details, please refer to [link / reference needed]. Figure 2 The corresponding first and second steps are as follows: Afterwards, image acquisition is performed using a reference acquisition device to obtain a reference acquisition image. Further, to ensure image quality, the reference acquisition image can be assessed to determine if it meets the quality requirements. If not, the process returns to re-acquiring the image using the reference acquisition device. If it meets the quality requirements, image acquisition is performed using a compensation acquisition device to obtain a compensation acquisition image. Again, to ensure image quality, the compensation acquisition image can be assessed to determine if it meets the quality requirements. If not, the process returns to re-acquiring the image using the compensation acquisition device. If the compensation acquisition image meets the quality requirements, the color mapping relationship between the reference acquisition image and the compensation acquisition image can be calculated. After obtaining the color mapping relationship, color compensation can be performed on the acquired image during the virtual and real image acquisition process using the compensation acquisition device, resulting in a compensated image. Regarding color compensation operations, this application provides two different correction and compensation methods: First, color calibration compensation is performed in advance in the virtual rendering engine, that is, a color mapping relationship is attached to the virtual rendering engine, and the virtual rendering engine is color-calibrated according to the color calibration mapping relationship and color mapping relationship corresponding to the compensation acquisition device, and then the virtual and real images are acquired by the compensation acquisition device to obtain the final compensated image; Second, a color mapping relationship is attached to the compensation acquisition device, and the above color mapping relationship is not introduced in the virtual image rendering stage. Instead, after the original acquired image is acquired by the compensation acquisition device, the color values ​​of each pixel in the original acquired image are corrected and compensated according to the color mapping relationship to obtain the compensated image.

[0097] See Figure 4 , Figure 4 This is a structural block diagram of a data processing apparatus according to an embodiment of the present application. The apparatus includes:

[0098] The device determination module 402 is used to determine the reference acquisition device and the compensation acquisition device among multiple acquisition devices. The reference acquisition device is the acquisition device that has achieved virtual and real color alignment; the compensation acquisition device is the other acquisition device besides the reference acquisition device.

[0099] The mapping relationship acquisition module 404 is used to acquire the color mapping relationship between the reference acquisition device and the compensation acquisition device. The color mapping relationship represents the mapping relationship between the reference color value obtained by the reference acquisition device after image acquisition and the compensation color value obtained by the compensation acquisition device after image acquisition for the same display color.

[0100] The compensation module 406 is used to perform color compensation on the acquired image during the acquisition of virtual and real images by the compensation acquisition device based on the color mapping relationship, so as to obtain the compensated image.

[0101] Optionally, in some embodiments, the mapping relationship acquisition module 404 is specifically used for:

[0102] Acquire a baseline acquisition image, which is an image obtained by a baseline acquisition device capturing the target object.

[0103] Acquire compensated acquisition images, which are images obtained by the compensation acquisition device from the target object;

[0104] The baseline color value of the target object in the baseline acquisition image and the compensation color value of the target object in the compensation acquisition image are obtained respectively.

[0105] Based on the reference color value and the compensation color value, the color mapping relationship between the reference acquisition device and the compensation acquisition device is calculated.

[0106] Optionally, in some embodiments, the target object includes: a real-world object and a virtual object displayed on the screen;

[0107] The mapping relationship acquisition module 404, when performing the steps of acquiring the reference color value of the target object in the reference acquisition image and the compensation color value of the target object in the compensation acquisition image, is specifically used for:

[0108] Obtain the baseline real-world color value of the real-world object in the baseline acquisition image, and the compensated real-world color value of the real-world object in the compensated acquisition image; obtain the baseline virtual color value of the virtual object in the baseline acquisition image, and the compensated virtual color value of the virtual object in the compensated acquisition image.

[0109] The mapping relationship acquisition module 404, when performing the step of calculating the color mapping relationship between the reference acquisition device and the compensation acquisition device based on the reference color value and the compensation color value, is specifically used for:

[0110] Based on the baseline real-scene color value and the compensated real-scene color value, the real-scene color mapping relationship between the baseline acquisition device and the compensated acquisition device is calculated; based on the baseline virtual color value and the compensated virtual color value, the virtual color mapping relationship between the baseline acquisition device and the compensated acquisition device is calculated.

[0111] By integrating real-world color mapping relationships and virtual color mapping relationships, the color mapping relationship between the baseline acquisition device and the compensation acquisition device is obtained.

[0112] Optionally, in some embodiments, the mapping relationship acquisition module 404, when performing the steps of acquiring the reference color value of the target object in the reference acquisition image and the compensation color value of the target object in the compensation acquisition image, is specifically used for:

[0113] Determine whether the baseline acquired image and the compensated acquired image meet the preset image quality conditions respectively;

[0114] If both the baseline acquisition image and the compensation acquisition image meet the image quality requirements, then the baseline color value of the target object in the baseline acquisition image and the compensation color value of the target object in the compensation acquisition image are obtained respectively.

[0115] Optionally, in some embodiments, the compensation module 406 is specifically used for:

[0116] Acquire the original compensated virtual-real image; the original compensated virtual-real image is the image obtained by the compensation acquisition device after color calibration of the virtual rendering engine according to the color calibration mapping relationship corresponding to the compensation acquisition device;

[0117] Based on the color mapping relationship, the color values ​​of each pixel in the original compensated virtual and real images are corrected and compensated to obtain the compensated image.

[0118] Optionally, in some embodiments, the compensation module 406 is specifically used for:

[0119] Acquire the compensated virtual and real images captured by the compensation acquisition device as the compensated image;

[0120] Among them, the compensated virtual-real image is the image obtained by the compensation acquisition device after color calibration of the virtual rendering engine according to the color calibration mapping relationship and color mapping relationship corresponding to the compensation acquisition device, and then the virtual-real image is acquired.

[0121] Optionally, in some embodiments, when performing the step of correcting and compensating the color values ​​of each pixel in the original compensated virtual-real image according to the color mapping relationship to obtain the compensated image, the compensation module 406 is specifically used for:

[0122] The original compensated virtual and real images are downsampled to obtain the sampled image;

[0123] Based on the color mapping relationship, the color values ​​of each pixel in the sampled image are corrected and compensated to obtain the compensated sampled image;

[0124] Calculate the first pixel value difference between the compensated sampled image and the reference virtual-real image, and the second pixel value difference between the original compensated virtual-real image and the reference virtual-real image; the reference virtual-real image is the image obtained by the reference acquisition device through virtual-real image acquisition;

[0125] If the difference between the first pixel value and the difference between the second pixel value meet the preset difference condition, the color values ​​of each pixel in the original compensated virtual and real image are corrected and compensated according to the color mapping relationship to obtain the compensated image.

[0126] The data processing device of this embodiment is used to implement the corresponding data processing methods in the aforementioned mapping relationship calculation method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. Furthermore, the functional implementation of each module in the data processing device of this embodiment can be referred to the description of the corresponding part in the aforementioned method embodiments, which will also not be repeated here.

[0127] Reference Figure 5 This document illustrates a schematic diagram of an electronic device according to an embodiment of this application. The specific embodiments of this application do not limit the specific implementation of the electronic device.

[0128] like Figure 5 As shown, the control terminal may include: a processor 502, a communications interface 504, a memory 506, and a communications bus 508.

[0129] in:

[0130] The processor 502, communication interface 504, and memory 506 communicate with each other via communication bus 508.

[0131] Communication interface 504 is used to communicate with other electronic devices or servers.

[0132] The processor 502 is used to execute program 510, specifically to perform the relevant steps in the above data processing method embodiment.

[0133] Specifically, program 510 may include program code that includes computer operation instructions.

[0134] The processor 502 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The smart device may include one or more processors of the same type, such as one or more CPUs; or it may include processors of different types, such as one or more CPUs and one or more ASICs.

[0135] Memory 506 is used to store program 510. Memory 506 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0136] Program 510 may include multiple computer instructions, and specifically, program 510 may use multiple computer instructions to cause processor 502 to execute the operations corresponding to the methods described in the foregoing multiple method embodiments.

[0137] The specific implementation of each step in program 510 can be found in the corresponding steps and units described in the above method embodiments, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0138] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in any of the foregoing method embodiments. The computer storage medium includes, but is not limited to, compact disc read-only memory (CD-ROM), random access memory (RAM), floppy disk, hard disk, or magneto-optical disk.

[0139] This application also provides a computer program product, including computer instructions that instruct a computing device to perform an operation corresponding to any of the methods in the above-described multiple method embodiments.

[0140] Furthermore, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used for training the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0141] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0142] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA)). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., Random Access Memory (RAM), Read-Only Memory (ROM), Flash Memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0143] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0144] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A data processing method, comprising: A reference acquisition device and a compensation acquisition device are identified among multiple acquisition devices, wherein the reference acquisition device is an acquisition device that has achieved virtual and real color alignment; The compensation acquisition device is any acquisition device other than the reference acquisition device; Obtain the color mapping relationship between the baseline acquisition device and the compensation acquisition device; The color mapping relationship represents the mapping relationship between the reference color value obtained by the reference acquisition device after capturing the image of the target object and the compensated color value obtained by the compensation acquisition device after capturing the image of the target object for the same display color. The target objects include real-world objects placed in the real scene and virtual objects displayed on the screen; Based on the color mapping relationship, color compensation is performed on the acquired image during the virtual and real image acquisition process of the compensation acquisition device to obtain the compensated image.

2. The method according to claim 1, wherein, The acquisition of the color mapping relationship between the reference acquisition device and the compensation acquisition device includes: Acquire a reference acquisition image, which is an image obtained by a reference acquisition device capturing the target object; Acquire a compensated acquisition image, wherein the compensated acquisition image is an image obtained by a compensation acquisition device capturing the target object; The reference color value of the target object in the reference acquisition image and the compensation color value of the target object in the compensation acquisition image are obtained respectively. Based on the reference color value and the compensation color value, the color mapping relationship between the reference acquisition device and the compensation acquisition device is calculated.

3. The method according to claim 2, wherein, The step of obtaining the reference color value of the target object in the reference acquisition image and the compensated color value of the target object in the compensated acquisition image includes: Obtain the reference real-world color value of the real-world object in the reference acquisition image, and the compensated real-world color value of the real-world object in the compensated acquisition image; obtain the reference virtual color value of the virtual object in the reference acquisition image, and the compensated virtual color value of the virtual object in the compensated acquisition image; The step of calculating the color mapping relationship between the reference acquisition device and the compensation acquisition device based on the reference color value and the compensation color value includes: Based on the reference real-scene color value and the compensated real-scene color value, the real-scene color mapping relationship between the reference acquisition device and the compensated acquisition device is calculated; based on the reference virtual color value and the compensated virtual color value, the virtual color mapping relationship between the reference acquisition device and the compensated acquisition device is calculated. By fusing the real-world color mapping relationship and the virtual color mapping relationship, a color mapping relationship between the reference acquisition device and the compensation acquisition device is obtained.

4. The method according to claim 2, wherein, The step of obtaining the reference color value of the target object in the reference acquisition image and the compensated color value of the target object in the compensated acquisition image includes: Determine whether the baseline acquired image and the compensated acquired image meet the preset image quality conditions; If both the reference acquisition image and the compensated acquisition image meet the image quality conditions, then the reference color value of the target object in the reference acquisition image and the compensated color value of the target object in the compensated acquisition image are obtained respectively.

5. The method according to any one of claims 1-4, wherein, Based on the color mapping relationship, the process of performing color compensation on the acquired image during the acquisition of virtual and real images by the compensation acquisition device to obtain the compensated image includes: Acquire the original compensated virtual-real image; the original compensated virtual-real image is the image obtained by the compensation acquisition device after color calibration of the virtual rendering engine according to the color calibration mapping relationship corresponding to the compensation acquisition device; Based on the color mapping relationship, the color values ​​of each pixel in the original compensated virtual-real image are corrected and compensated to obtain the compensated image.

6. The method according to any one of claims 1-4, wherein, Based on the color mapping relationship, the process of performing color compensation on the acquired image during the acquisition of virtual and real images by the compensation acquisition device to obtain the compensated image includes: Acquire the compensated virtual and real images captured by the compensation acquisition device as the compensated image; The compensated virtual-real image is an image obtained by the compensation acquisition device after color calibration of the virtual rendering engine according to the color calibration mapping relationship corresponding to the compensation acquisition device and the color mapping relationship, and then capturing the virtual-real scene.

7. The method according to claim 5, wherein, The step of correcting and compensating the color values ​​of each pixel in the original compensated virtual-real image according to the color mapping relationship to obtain the compensated image includes: The original compensated virtual-real image is downsampled to obtain a sampled image; Based on the color mapping relationship, the color values ​​of each pixel in the sampled image are corrected and compensated to obtain the compensated sampled image; Calculate the first pixel value difference between the compensated sampled image and the reference virtual-real image, and the second pixel value difference between the original compensated virtual-real image and the reference virtual-real image; the reference virtual-real image is an image obtained by a reference acquisition device through virtual-real image acquisition; If the difference between the first pixel value and the difference between the second pixel value meet the preset difference condition, the color values ​​of each pixel in the original compensated virtual-real image are corrected and compensated according to the color mapping relationship to obtain the compensated image.

8. A data processing apparatus, comprising: The device determination module is used to determine the reference acquisition device and the compensation acquisition device among multiple acquisition devices. The reference acquisition device is an acquisition device that has achieved virtual and real color alignment. The compensation acquisition device is any acquisition device other than the reference acquisition device; The mapping relationship acquisition module is used to acquire the color mapping relationship between the reference acquisition device and the compensation acquisition device; The color mapping relationship represents the mapping relationship between the reference color value obtained by the reference acquisition device after capturing the image of the target object and the compensated color value obtained by the compensation acquisition device after capturing the image of the target object for the same display color. The target objects include real-world objects placed in the real scene and virtual objects displayed on the screen; The compensation module is used to perform color compensation on the acquired image during the virtual and real image acquisition process of the compensation acquisition device based on the color mapping relationship, so as to obtain the compensated image.

9. An electronic device, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation corresponding to the method as described in any one of claims 1-7.

10. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1-7.

11. A computer program product comprising computer instructions that instruct a computing device to perform the method as described in any one of claims 1-7.

Images