Data processing method, device and equipment and computer storage medium

Abstract

The embodiment of the invention provides a data processing method and device, equipment and a computer storage medium. The data processing method comprises the following steps: determining a reference acquisition device and a compensation acquisition device in a plurality of acquisition devices, wherein the reference acquisition device is an acquisition device which realizes virtual and real color alignment; acquiring a color mapping relation between the reference acquisition equipment and the compensation acquisition equipment; the color mapping relationship represents a mapping relationship between a reference color value obtained after the reference acquisition equipment performs picture acquisition and a compensation color value obtained after the compensation acquisition equipment performs picture acquisition for the same display color; and on the basis of the color mapping relationship, performing acquired image color compensation in the process of performing virtual and real image acquisition by the compensation acquisition equipment to obtain a compensated image. According to the embodiment of the invention, virtual and real color alignment can be realized in the compensation acquisition equipment, and the shooting quality of virtual shooting is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technologies, and in particular, to a data processing method, apparatus, electronic device, computer storage medium, and computer program product. Background Art

[0002] XR (Extended Reality) virtual shooting is to project the rendered scene map obtained by virtual engine rendering onto an LED screen for display. Then, the actor uses the LED screen as the background for performance. An image acquisition device (such as a camera) simultaneously shoots the actor and the LED screen. After that, the captured camera image (acquired image) is synthesized with the rendered scene map, so as to place the real actor in the virtual scene, achieving the effect of shooting outdoor scenes or science fiction backgrounds in a studio.

[0003] Due to the influence of internal factors of the image acquisition device and other aspects, there is a certain difference between the color values in the original rendered scene map obtained by virtual rendering engine rendering and the color values in the image finally acquired by the image acquisition device, that is, there is a certain color distortion. Therefore, usually, color calibration is required for the color values in the original rendered scene map and the color values in the image finally acquired by the image acquisition device to eliminate the above differences. Further, in order to avoid color differences between the actual scene and the virtual scene in the captured image of the image acquisition device, further color adjustment is also performed on the virtual rendering engine to achieve virtual-real color alignment and improve the shooting effect.

[0004] In the actual shooting process, considering the shooting requirements, image acquisition devices are usually set at different positions, and the images are shot separately by multiple image acquisition devices. Then, post-processing such as editing is performed on the images shot by each image acquisition device to generate products that meet the requirements. However, in the multi-device acquisition scenario, there are usually image acquisition devices that cannot achieve virtual-real color alignment. Summary of the Invention

[0005] In view of this, the embodiments of the present application provide a data processing solution to at least partially solve the above problems.

[0006] According to a first aspect of the embodiments of the present application, a data processing method is provided, including:

[0007] Determine a reference acquisition device and a compensation acquisition device among multiple acquisition devices, where the reference acquisition device is an acquisition device that has achieved virtual-real color alignment; the compensation acquisition device is other acquisition devices except the reference acquisition device;

[0008] Obtain the color mapping relationship between the reference acquisition device and the compensation acquisition device; the color mapping relationship represents the mapping relationship between the reference color value obtained by the reference acquisition device after capturing the picture for the same display color and the compensation color value obtained by the compensation acquisition device after capturing the picture.

[0009] Based on the color mapping relationship, perform color compensation on the captured image during the process of the compensation acquisition device capturing the virtual and real pictures to obtain the compensated image.

[0010] According to the second aspect of the embodiments of the present application, there is provided a data processing device, including:

[0011] A device determination module, configured to determine the reference acquisition device and the compensation acquisition device among multiple acquisition devices, where the reference acquisition device is an acquisition device that has achieved virtual-real color alignment; the compensation acquisition device is other acquisition devices except the reference acquisition device;

[0012] A mapping relationship acquisition module, configured to obtain the color mapping relationship between the reference acquisition device and the compensation acquisition device; the color mapping relationship represents the mapping relationship between the reference color value obtained by the reference acquisition device after capturing the picture for the same display color and the compensation color value obtained by the compensation acquisition device after capturing the picture.

[0013] A compensation module, configured to perform color compensation on the captured image during the process of the compensation acquisition device capturing the virtual and real pictures based on the color mapping relationship to obtain the compensated image.

[0014] According to the third aspect of the embodiments of the present application, there is provided an electronic device, including: a processor, a memory, a communication interface, and a communication bus, where the processor, the memory, and the communication interface complete communication with each other through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operations corresponding to the method in the first aspect.

[0015] According to the fourth aspect of the embodiments of the present application, there is provided a computer storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the method described in the first aspect.

[0016] According to the data processing solution provided by the embodiments of the present application, a reference acquisition device that has achieved virtual-real color alignment and a compensation acquisition device that has not achieved virtual-real alignment are determined from multiple acquisition devices for virtual shooting. Then, the color mapping relationship between the reference acquisition device and the compensation acquisition device is obtained. Furthermore, based on this color mapping relationship, color compensation is performed on the acquisition images obtained by the compensation acquisition device for virtual-real scene acquisition, so as to obtain a compensated image with virtual-real color alignment. In the embodiments of the present application, by means of the color mapping relationship between the reference acquisition device and the compensation acquisition device, the difference in spectral understanding between the reference acquisition device and the compensation acquisition device is applied to the compensation acquisition device, thereby effectively reducing the color difference between the virtual content and the real scene content in the images acquired by the compensation acquisition device, and virtual-real color alignment is also achieved in the compensation acquisition device, improving the shooting quality of virtual shooting.

[0017] In addition, compared with the method of separately performing the color calibration and color adjustment steps of the virtual rendering engine for each individual device among multiple image acquisition devices to achieve virtual-real color alignment, for the virtual-real alignment solution provided by the embodiments of the present application, it is only necessary to perform the virtual rendering engine color calibration and adjustment steps for the reference acquisition device, and there is no need to repeatedly perform the virtual rendering engine color calibration and adjustment steps for the remaining devices. Therefore, through the embodiments of the present application, the shooting process can be made simpler and the shooting difficulty can be lower.

[0018] Moreover, in many actual shooting scenarios, there is a need for simultaneous shooting with multiple devices. However, the above method of separately performing the color calibration and adjustment operations of the virtual rendering engine for each individual device can only achieve the virtual-real alignment effect of one device at a specific shooting moment, and thus cannot meet the above requirements. On the contrary, the solution of the embodiments of the present application can first achieve the virtual-real alignment of the reference acquisition device, and then, based on the color mapping relationship between the reference acquisition device and the compensation acquisition device, perform color compensation on the compensation acquisition device, thereby achieving the virtual-real alignment of the compensation acquisition device, better meeting the above requirements, and the applicable scenarios of the embodiments of the present application are more extensive. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0020] Figure 1 FIG. is a flowchart of the steps of a data processing method according to an embodiment of the present application;

[0021] Figure 2Schematic diagram of the virtual shooting link corresponding to the embodiment of the present application;

[0022] Figure 3 Schematic flow chart of data processing according to the embodiment of the present application;

[0023] Figure 4 Block diagram of the structure of a data processing device according to the embodiment of the present application;

[0024] Figure 5 Schematic diagram of the structure of an electronic device according to the embodiment of the present application. Specific implementation manners

[0025] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art shall fall within the protection scope of the embodiments of the present application.

[0026] The following further illustrates the specific implementation of the embodiments of the present application in conjunction with the accompanying drawings of the embodiments of the present application.

[0027] Referring to Figure 1 , Figure 1 is a step flow chart of a data processing method according to the embodiment of the present application. Specifically, the data processing method provided in this embodiment includes the following steps:

[0028] Step 102, determine the reference acquisition device and the compensation acquisition device among multiple acquisition devices, where the reference acquisition device is the acquisition device that has achieved virtual-real color alignment; the compensation acquisition device is other acquisition devices except the reference acquisition device.

[0029] Specifically, during virtual shooting, considering the shooting requirements, multiple image acquisition devices are usually set up. The pictures are taken separately by the multiple image acquisition devices, and then post-processing such as editing is performed on the pictures taken by each image acquisition device to generate products that meet the requirements.

[0030] During the shooting process, due to the influence of internal factors of the image acquisition device and other aspects, there is a certain difference between the color values in the original rendered scene map rendered by the virtual rendering engine and the color values in the image finally acquired by the image acquisition device, that is, there is a certain color distortion. Therefore, usually, color calibration is performed on the color values in the original rendered scene map rendered by the virtual rendering engine and the color values in the image finally acquired by the image acquisition device to eliminate the above differences. In addition, in order to avoid color differences between the actual scene and the virtual scene in the picture captured by the image acquisition device, further parameter adjustments and other aspects are also made to the virtual rendering engine to achieve virtual-real color alignment and improve the shooting effect. That is to say, when the image acquisition device is determined, in order to make the virtual-real colors in the finally obtained image aligned, usually, color calibration processing is first performed on the virtual rendering engine, and then, on the basis of color calibration, parameter adjustment is performed on the virtual rendering engine to adjust the color values of the rendered image and achieve virtual-real color alignment.

[0031] When performing color calibration and virtual-real alignment, an image acquisition device can be selected from multiple image acquisition devices (for the convenience of description, the selected image acquisition device is called the reference acquisition device in the embodiments of the present application) to perform color calibration and virtual-real color alignment processing, so as to ensure that the images acquired through the above reference acquisition device are virtual-real color aligned, that is: for the reference acquisition device in the embodiments of the present application, color calibration has been pre-performed on the color values in the original rendered scene map rendered by the virtual rendering engine and the color values in the image finally acquired by the device, and moreover, color adjustment has been performed on the color values in the original rendered scene map rendered by the virtual rendering engine, achieving color alignment between the color values in the original rendered scene map rendered by the virtual rendering engine and the color values in the actual scene. In summary, the reference acquisition device is a device that has achieved virtual-real color alignment. The specific process of virtual-real color alignment of the reference acquisition device can be implemented based on related technologies.

[0032] However, due to different physical factors such as internal photosensitive elements of different image acquisition devices, different understandings of the spectrum are obtained. Therefore, after color calibration and alignment of the virtual rendering engine based on the reference acquisition device, there may still be a problem of color misalignment between the virtual and real pictures captured by the remaining acquisition devices (that is, the unselected image acquisition devices among the above multiple image acquisition devices).

[0033] Step 104, obtain the color mapping relationship between the reference acquisition device and the compensation acquisition device.

[0034] Specifically, the color mapping relationship obtained through step 104 can represent the mapping relationship between the reference color value obtained by the reference acquisition device after capturing the picture for the same display color and the compensation color value obtained by the compensation acquisition device after capturing the picture.

[0035] Step 106: Based on the color mapping relationship, perform color compensation on the captured image during the process of the compensation acquisition device capturing virtual and real pictures to obtain a compensated image.

[0036] Specifically, as described above, when performing color calibration and compensation, it can be based on the reference acquisition device. That is to say, it can be achieved by aligning virtual and real colors according to the understanding of the spectrum by the internal photosensitive elements of the reference acquisition device. The color mapping relationship between the reference acquisition device and the compensation acquisition device obtained through step 104 can represent the degree of difference in the understanding of the spectrum between the reference acquisition device and the compensation acquisition device. Therefore, after obtaining the above color mapping relationship, the color of the captured image can be compensated during the process of the compensation acquisition device capturing virtual and real pictures through this color mapping relationship, so as to obtain a compensated image with virtual and real alignment.

[0037] XR virtual shooting is to project the rendered scene map obtained by virtual engine rendering onto a screen for display, and then deploy physical objects (such as actors, props, etc.) in front of the screen. The image acquisition device simultaneously captures the physical objects and the rendered scene map on the screen. Therefore, it can be understood that the virtual and real picture acquisition in the embodiments of the present application can refer to the process of the image acquisition device simultaneously capturing the rendered scene map (virtual object) displayed on the screen and the physical objects deployed in the actual space.

[0038] According to the data processing method provided by the embodiments of the present application, a reference acquisition device that has achieved virtual and real color alignment and a compensation acquisition device that has not achieved virtual and real alignment are determined from multiple acquisition devices for virtual shooting. Then, the color mapping relationship between the reference acquisition device and the compensation acquisition device is obtained. Furthermore, color compensation is performed on the captured image obtained by the compensation acquisition device for virtual and real picture acquisition based on this color mapping relationship, so as to obtain a compensated image with virtual and real color alignment. In the embodiments of the present application, by means of the color mapping relationship between the reference acquisition device and the compensation acquisition device, the difference in the understanding of the spectrum between the reference acquisition device and the compensation acquisition device is applied to the compensation acquisition device, effectively reducing the color difference between the virtual content and the real scene content in the image captured by the compensation acquisition device, achieving virtual and real color alignment in the compensation acquisition device as well, and improving the shooting quality of virtual shooting.

[0039] In addition, compared with the method of separately performing the color calibration and color adjustment steps of the virtual rendering engine for each individual device among multiple image acquisition devices to achieve virtual-real color alignment, the virtual-real alignment solution provided by the embodiments of the present application only needs to perform the color calibration and adjustment steps of the virtual rendering engine for the reference acquisition device, without the need to repeatedly perform the color calibration and adjustment steps of the virtual rendering engine for the remaining devices. Therefore, through the embodiments of the present application, the shooting process can be made simpler and the shooting difficulty can be reduced.

[0040] Moreover, in many actual shooting scenarios, there is a need for multiple devices to shoot simultaneously. However, the above method of separately performing the color calibration and adjustment operations of the virtual rendering engine for each individual device can only achieve the virtual-real alignment effect of one device at a specific shooting moment, and thus cannot meet the above needs. On the contrary, the solution of the embodiments of the present application can first achieve the virtual-real alignment of the reference acquisition device, and then perform color compensation on the compensation acquisition device based on the color mapping relationship between the reference acquisition device and the compensation acquisition device, thereby achieving the virtual-real alignment of the compensation acquisition device and better meeting the above needs. The applicable scenarios of the embodiments of the present application are more extensive.

[0041] The data processing method of this embodiment can be executed by any suitable electronic device with data processing capabilities, including but not limited to: servers, PCs, etc.

[0042] Optionally, in some embodiments, the process of obtaining the color mapping relationship between the reference acquisition device and the compensation acquisition device may specifically include:

[0043] Obtain a reference acquisition image, which is an image obtained by the reference acquisition device capturing a picture of the target object; obtain a compensation acquisition image, which is an image obtained by the compensation acquisition device capturing a picture of the target object; respectively obtain the reference color value of the target object in the reference acquisition image and the compensation color value of the target object in the compensation acquisition image; and calculate the color mapping relationship between the reference acquisition device and the compensation acquisition device according to the reference color value and the compensation color value.

[0044] Specifically, since the color mapping relationship represents the mapping relationship between the reference color value obtained by the reference acquisition device after capturing a picture of the same display color and the compensation color value obtained by the compensation acquisition device after capturing a picture of the same display color. Therefore, a target object with a certain color value or multiple color values can be preset, and then the target object can be captured by the reference acquisition device and the compensation acquisition device respectively, and the color mapping relationship can be obtained through the color values of the target object in the reference acquisition image and the compensation acquisition image respectively.

[0045] Specifically, the color mapping relationship can be represented in the form of a color mapping relationship lookup table, which may include the correspondence between different reference color values and different compensation color values. After obtaining the color values of the target object in the reference acquisition image and the compensation acquisition image respectively, based on the correspondence between the color value of the target object in the reference color image and the color value of the target object in the compensation acquisition image, when the color values of the target object do not include all color values, other correspondence between color values outside the color value of the target object can also be calculated based on interpolation and other methods, a color mapping relationship lookup table is constructed, and this color mapping relationship lookup table is used as the color mapping relationship. In addition, the color mapping relationship can also be represented in the form of a matrix. Exemplarily, a reference color matrix can be formed by the color values of the target object in the reference acquisition image, and a compensation color matrix can be formed by the color values of the target object in the compensation color image, and then the mapping matrix between the reference color matrix and the compensation color matrix is obtained, and then this mapping matrix is used as the above-mentioned color mapping relationship.

[0046] In the above process of calculating the color mapping relationship, by introducing the same target object, a reference comparison condition for color value mapping is constructed for the calculation of the color mapping relationship between the reference acquisition device and the compensation acquisition device, so that the finally calculated color mapping relationship is a mapping relationship for the same display color, improving the accuracy of the calculation of the color mapping relationship.

[0047] In the embodiments of the present application, neither the color values of the target object nor the number of color values are limited, and can be custom-set according to actual situations. Exemplarily, multiple target objects can be set in advance. A single target object has a single color value, and different target objects have different color values. In this case, for each target object, the above process of obtaining the color mapping relationship between the reference acquisition device and the compensation acquisition device can be executed respectively, and a color mapping relationship applicable to multiple different color values can be obtained. In this method, since a single target object has a single color value, in the process of calculating a single mapping relationship, the color complexity of the image collected once is relatively low and the noise is small, which helps to improve the accuracy of the calculation of the color mapping relationship. In addition, a target object can also be set in advance, and this target object contains multiple different color values. In this case, by executing the above process of obtaining the color mapping relationship between the reference acquisition device and the compensation acquisition device once for this target object, a color mapping relationship applicable to multiple different color values can be obtained. In this method, since the target object contains multiple color values, a color mapping relationship for multiple different colors can be obtained by executing the above mapping relationship calculation process once. Therefore, it helps to improve the calculation efficiency of the color mapping relationship calculation.

[0048] In addition, in the scenario of XR virtual shooting, the target object in the above embodiments of the present application can be a virtual object rendered to the display screen by a virtual rendering engine, a real object set in the actual scenario, or can also include both virtual objects and real objects.

[0049] Optionally, in some embodiments, the above target object may include: a real object and a virtual object displayed on the display screen; correspondingly, the process of respectively obtaining the reference color value of the target object in the reference acquisition image and the compensation color value of the target object in the compensation acquisition image may specifically include:

[0050] Obtaining the reference real-scene color value of the real object in the reference acquisition image and the compensation real-scene color value of the real object in the compensation acquisition image; obtaining the reference virtual color value of the virtual object in the reference acquisition image and the compensation virtual color value of the virtual object in the compensation acquisition image;

[0051] Correspondingly, the process of calculating the color mapping relationship between the reference acquisition device and the compensation acquisition device according to the reference color value and the compensation color value may specifically include:

[0052] Calculating the real-scene color mapping relationship between the reference acquisition device and the compensation acquisition device according to the reference real-scene color value and the compensation real-scene color value; calculating the virtual color mapping relationship between the reference acquisition device and the compensation acquisition device according to the reference virtual color value and the compensation virtual color value;

[0053] Fusing the real-scene color mapping relationship and the virtual color mapping relationship to obtain the color mapping relationship between the reference acquisition device and the compensation acquisition device.

[0054] Specifically, for the color mapping relationship represented by the color mapping relationship lookup table, the specific process of fusing the real-scene color mapping relationship and the virtual color mapping relationship can be: determining the target reference real-scene color value and the target reference virtual color value with the same value as the target reference color value; determining the target compensation real-scene color value corresponding to the target reference real-scene color value from the real-scene color mapping relationship lookup table, and determining the target compensation virtual color value corresponding to the target reference virtual color value from the virtual color mapping relationship lookup table; then fusing the target compensation real-scene color value and the target compensation virtual color value to obtain a color value fusion result; based on the above target reference color value and the above color value fusion result, constructing a color mapping relationship lookup table as the color mapping relationship between the reference acquisition device and the compensation acquisition device. In the embodiments of the present application, no specific fusion method is limited for fusing the target compensation real-scene color value and the target compensation virtual color value. Exemplarily, it can be an average value method, or a weighted summation method according to a preset weight, etc.

[0055] Regarding the color mapping relationship represented in a matrix manner, the specific process of fusing the real-scene color mapping relationship and the virtual color mapping relationship can be as follows: perform element value fusion processing on two matrix element values with the same position coordinates in the real-scene color mapping matrix and the virtual color mapping matrix, and use the result of the fusion processing as the matrix element at that position coordinate in the finally obtained color mapping matrix. In the embodiments of the present application, the specific processing method for the above element value fusion processing is not limited. Exemplarily, it can be an average value calculation, or a weighted summation calculation according to a preset weight, and so on.

[0056] In the above embodiments of the present application, objects used as reference comparison conditions are respectively set in the real scene and the virtual scene. Furthermore, the process of obtaining the color mapping relationship is split into two branches: the real-scene branch and the virtual-scene branch, and the color mapping relationships corresponding to each branch are obtained respectively; then, fusion calculations are performed on the color mapping relationships corresponding to each branch to obtain the final color mapping relationship. The above process takes into account both the color differences between images after image acquisition in the real scene and the color differences between images after image acquisition in the virtual scene. Therefore, the finally obtained color mapping relationship can better adapt to the XR virtual shooting scene combining the real and the virtual, and improves the accuracy of the color mapping relationship calculation.

[0057] Optionally, in some embodiments, the process of respectively obtaining the reference color value of the target object in the reference acquisition image and the compensation color value of the target object in the compensation acquisition image may specifically include:

[0058] Respectively determine whether the reference acquisition image and the compensation acquisition image meet the preset image quality conditions;

[0059] If both the reference acquisition image and the compensation acquisition image meet the image quality conditions, then respectively obtain the reference color value of the target object in the reference acquisition image and the compensation color value of the target object in the compensation acquisition image.

[0060] Furthermore, after respectively determining whether the reference acquisition image and the compensation acquisition image meet the preset image quality conditions, the data processing method may further include:

[0061] If the reference acquisition image does not meet the image quality conditions, then return to the step of obtaining the reference acquisition image to perform the subsequent color mapping relationship calculation steps based on the newly obtained reference acquisition image;

[0062] If the compensation acquisition image does not meet the image quality conditions, then return to the step of obtaining the compensation acquisition image to perform the subsequent color mapping relationship calculation steps based on the newly obtained compensation acquisition image.

[0063] Specifically, when each image acquisition device performs screen acquisition, affected by factors such as the acquisition angle and ambient light, the quality of the acquired image may be poor. For example, the image brightness is too high (greater than the preset brightness threshold), the image brightness is too low (less than the preset brightness threshold). Further, for a target object containing multiple different color values, the brightness values of each pixel point in the acquired image are too concentrated (the difference in brightness values is small), or the number of color values contained in the acquired image is small, and so on. If subsequent calculation operations are performed based on the above-mentioned images with poor quality, it may lead to poor accuracy of the calculated color mapping relationship, or even the problem that the color mapping relationship cannot be obtained.

[0064] Therefore, in the above embodiments of the present application, after obtaining the reference acquisition image and the compensation acquisition image, the image quality can be determined first. When both the reference acquisition image and the compensation acquisition image meet the image quality conditions, subsequent calculation operations are performed. In this way, the problem of low accuracy of the calculation result caused by the too poor quality of the acquired image can be effectively avoided. That is, through the above process, the calculation accuracy of the color mapping relationship can be effectively improved.

[0065] Optionally, in some embodiments, based on the color mapping relationship, the process of performing color compensation on the acquisition image during the process of the compensation acquisition device performing virtual and real screen acquisition to obtain the compensated image may include:

[0066] Obtain the original compensated virtual and real image; the original compensated virtual and real image is the image obtained by the compensation acquisition device performing virtual and real screen acquisition after calibrating the color of the virtual rendering engine according to the color calibration mapping relationship corresponding to the compensation acquisition device;

[0067] According to the color mapping relationship, correct and compensate the color values of each pixel point in the original compensated virtual and real image to obtain the compensated image.

[0068] Specifically, for the image acquisition device, due to the influence of internal factors of the image acquisition device and other aspects, there is a certain difference between the color values in the original rendered scene map rendered by the virtual rendering engine and the color values in the image finally acquired by the image acquisition device. The color calibration mapping relationship corresponding to the image acquisition device represents the mapping relationship between the color values in the original rendered scene map rendered by the virtual rendering engine and the color values in the image finally acquired by the image acquisition device. Specifically, for the color calibration mapping relationship corresponding to the compensation acquisition device, it represents the mapping relationship between the color values in the original rendered scene map rendered by the virtual rendering engine and the color values in the finally acquired image.

[0069] Specifically, for compensating the color calibration mapping relationship corresponding to the acquisition device, the calculation process may include: inputting the original rendered color into the shooting system including the acquisition device to obtain the acquisition image, that is, rendering the original rendered color to the LED display screen by the broadcast control processor, and then obtaining the acquisition image by shooting the LED display screen through the acquisition device; then, based on the original rendered color value and the color value of the acquisition image, constructing a forward LUT (Look Up Table, color calibration relationship lookup table) to obtain the color calibration mapping relationship f(x) corresponding to the acquisition device.

[0070] In the above process, the virtual rendering engine is first color-calibrated using the color calibration mapping relationship corresponding to the acquisition device, that is, using the color calibration mapping relationship corresponding to the acquisition device (such as the above forward LUT) to find the calibrated rendered color corresponding to the original rendered color, so as to color-calibrate the original rendered color to obtain the calibrated rendered color, and then rendering the calibrated rendered color to the display screen for display. Then, the acquisition device is used to collect the virtual and real scene images of the display screen and the physical object outside the display screen to obtain the original compensated virtual and real image. After that, the color values of each pixel point in the above original compensated virtual and real image are adjusted according to the color mapping relationship between the reference acquisition device and the acquisition device (such as by looking up a table or multiplying by a transformation matrix, etc.) to obtain the compensated image with virtual and real alignment. In the above process, in the virtual image rendering stage, the color mapping relationship between the reference acquisition device and the acquisition device is not introduced, but the original color calibration scheme for the virtual rendering engine is maintained. Therefore, while achieving the virtual and real alignment of the acquisition device, the complexity of the shooting operation caused by parameter changes to the virtual rendering engine during the shooting process can be effectively reduced.

[0071] Optionally, in some other embodiments of the present application, the process of performing color compensation on the acquisition image during the process of the acquisition device collecting virtual and real scene images according to the color mapping relationship to obtain the compensated image may also include:

[0072] Obtaining the compensated virtual and real image collected by the acquisition device as the compensated image;

[0073] Wherein, the compensated virtual and real image is an image obtained by the acquisition device collecting virtual and real scene images after color-calibrating the virtual rendering engine according to the color calibration mapping relationship and the color mapping relationship corresponding to the acquisition device.

[0074] Specifically, in the process of obtaining the compensated image with virtual-real alignment, after obtaining the color mapping relationship between the reference acquisition device and the compensation acquisition device, when the virtual rendering engine performs virtual scene rendering, the color calibration mapping relationship corresponding to the compensation acquisition device is used to calibrate the original rendering color, and then the color mapping relationship is used to adjust the calibrated rendering color again to obtain the adjusted rendering color. Then, the adjusted rendering color is rendered to the display screen for display. Then, the compensation acquisition device performs virtual-real scene acquisition on the display screen and the physical object placed outside the display screen, and the compensated image can be obtained. In the above process, in the virtual image rendering stage, the color mapping relationship between the reference acquisition device and the compensation acquisition device is introduced. The advantage is that: the image obtained by the compensation acquisition device for virtual-real scene acquisition subsequently is the compensated image with virtual-real alignment. That is: in the above process, after the compensation acquisition device outputs the acquired image, there is no need to perform post-processing operations on the acquired image. Therefore, the data processing amount of the data processing solution provided in the embodiments of the present application can be reduced.

[0075] Optionally, in some embodiments, the process of correcting and compensating the color values of each pixel point in the original compensated virtual-real image according to the color mapping relationship to obtain the compensated image may further include:

[0076] Downsample the original compensated virtual-real image to obtain a sampled image;

[0077] According to the color mapping relationship, correct and compensate the color values of each pixel point in the sampled image to obtain a compensated sampled image;

[0078] Calculate the first pixel value difference between the compensated sampled image and the reference virtual-real image, and the second pixel value difference between the original compensated virtual-real image and the reference virtual-real image; the reference virtual-real image is the image obtained by the reference acquisition device for virtual-real scene acquisition;

[0079] If the first pixel value difference and the second pixel value difference meet the preset difference condition, correct and compensate the color values of each pixel point in the original compensated virtual-real image according to the color mapping relationship to obtain the compensated image.

[0080] Specifically, from a theoretical perspective, compared with the original compensated virtual-real image, using the calculated color mapping relationship to correct and compensate the color values of each pixel point in the original compensated virtual-real image usually results in a better virtual-real alignment effect of the obtained compensated image. However, due to factors such as calculation accuracy, it is usually impossible to quantitatively know the compensation effect before using the calculated color mapping relationship for image color compensation.

[0081] In view of the above situation, to avoid the problem that the virtual-real alignment effect in the finally compensated image is not significantly improved after calculating the color mapping relationship, in the above embodiments of the present application, after obtaining the original compensated virtual-real image, the original compensated virtual-real image is first downsampled to obtain a sampled image with a smaller data volume (i.e., lower resolution). Then, subsequent image color value modification and compensation operations are performed based on the sampled image, and the color difference data between the sampled image after color value correction and compensation and the reference virtual-real image collected by the reference acquisition device is quantitatively calculated. After that, whether to perform the color value modification and compensation operation on the complete original compensated virtual-real image is determined according to the magnitude of the color difference data. In the above embodiments of the present application, the processing volume of image data is reduced by downsampling, and then, under the condition of a small amount of calculation, the improvement of the image quality before and after performing the color value correction and compensation operation is objectively quantified.

[0082] Compared with the traditional method that relies on the subjective experience of the photographer to decide whether to perform the color value correction and compensation operation, the above embodiments of the present application realize the quantitative display of the image quality improvement effect, which helps the operator to decide whether to perform the color value correction and compensation operation according to the actual situation.

[0083] In addition, in the embodiments of the present application, the first pixel value difference is used to characterize the difference degree of the pixel value of each pixel point between the compensated sampled image and the reference virtual-real image, and the second pixel value difference is used to characterize the difference degree of the pixel value of each pixel point between the original compensated virtual-real image and the reference virtual-real image. The preset difference condition can be whether the difference between the first pixel value difference and the second pixel value difference is large enough (exemplarily, whether it is greater than a preset threshold). When the difference between the first pixel value difference and the second pixel value difference is large enough (exemplarily, greater than the preset threshold), it indicates that the compensation effect is good. At this time, the actual compensation operation can be performed, that is: according to the color mapping relationship, the color value of each pixel point in the original compensated virtual-real image is corrected and compensated to obtain the compensated image.

[0084] Furthermore, for the original compensated virtual-real image, the compensated sampled image, or the reference virtual-real image, there are usually many pixel points in the image, and the color values of each pixel point may not be the same. In view of the above situation, in the embodiments of the present application, when calculating the pixel value difference, a reference object (which can be a region or one or more pixel points) can be selected from the virtual-real picture in advance, and according to the position of the reference object in the picture, or by means of object detection, the corresponding reference pixel points of the reference object in the original compensated virtual-real image, the compensated sampled image, or the reference virtual-real image are determined respectively. Then, the first pixel value difference and the second pixel value difference are calculated based on the color values of the reference pixel points in the above three images.

[0085] For example, the original compensated virtual-real image can be collected synchronously by a compensation acquisition device, and the reference virtual-real image can be collected by a reference acquisition device. The original compensated virtual-real image is downsampled to obtain a sampled image with a lower resolution. The compensated sampled image is obtained by correction compensation through a color mapping relationship. At this time, the resolution of the compensated sampled image is lower than that of the reference virtual-real image. The corresponding relationship between each pixel point in the compensated sampled image and each region in the reference virtual-real image can be determined. For example, a pixel point in the compensated sampled image corresponds to a region within a preset range around the pixel point at the same position in the reference virtual-real image. The difference between the pixel value of this pixel point in the compensated sampled image and the average pixel value of the corresponding region in the reference virtual-real image is calculated. Finally, the above differences corresponding to all pixel points (or reference pixel points) in the compensated sampled image are statistically analyzed (for example, statistical values such as the average value and standard deviation of the differences are calculated) to obtain the first pixel value difference. Those skilled in the art should understand that the method for calculating the first pixel value difference is not limited to this. For example, the pixel value of a pixel point in the compensated sampled image can also be compared with the pixel values of pixel points at the same or similar positions in the reference virtual-real image to obtain the difference, and then the difference is statistically analyzed to obtain the first pixel value difference. The second pixel difference can be obtained in a similar manner.

[0086] In an application example, for the same shooting scene (for example, the rendered scene diagram displayed on the screen is the same), a small number of original compensated virtual-real images and reference virtual-real images can be collected first, and the above judgment can be made. When it is determined that the first pixel value difference and the second pixel value difference meet the preset difference conditions, the color values of each pixel point of the original compensated virtual-real image collected under this shooting scene are corrected and compensated. Otherwise, the correction and compensation of the original compensated virtual-real image collected under this shooting scene can be abandoned. After changing the shooting scene, the above process can be executed again.

[0087] See Figure 2 , Figure 2 which is the schematic diagram of the virtual shooting link corresponding to the embodiment of the present application. The following briefly describes the specific execution process of the data processing method provided by the embodiment of the present application in conjunction with Figure 2 :

[0088] In the first step, a virtual object containing a variety of different color values (that is, rich in colors) is rendered by a virtual rendering engine and displayed on the display screen. Exemplarily, Figure 2 the virtual objects in include: virtual color cards of different colors, and virtual objects such as tables and chairs.

[0089] In the second step, correspondingly to the first step, real objects containing different color values are placed in the actual scene. Exemplarily, Figure 2 the real objects in correspond to the virtual objects, and the real objects also include: physical color cards of different colors, and physical items such as tables and chairs.

[0090] Step 3: Adjust the device parameters of the reference acquisition device to perform screen acquisition, so as to obtain a reference acquisition image containing the above virtual object and real object. Then, perform image quality determination on the acquired reference acquisition image (such as whether the image is too bright or too dark, or whether the richness of color values in the image meets the preset requirements, etc.), and transmit the reference acquisition image with qualified image quality to the server. As Figure 2 shown, exemplarily, the reference acquisition image can be transmitted to the server in the form of an SDI (Serial Digital Interface) signal.

[0091] Step 4: Adjust the device parameters of the compensation acquisition device to perform screen acquisition, so as to obtain a compensation acquisition image containing the above virtual object and real object. Then, perform image quality determination on the acquired compensation acquisition image (such as whether the image is too bright or too dark, or whether the richness of color values in the image meets the preset requirements, etc.), and transmit the compensation acquisition image with qualified image quality to the server. As Figure 2 shown, exemplarily, the compensation acquisition image can be transmitted to the server in the form of an SDI signal.

[0092] Step 5: Configure each image data received by the server through a display corresponding to the server or a mobile device connected to the service, etc., to clarify which image is the reference acquisition image and which image is the compensation acquisition image, etc.; then, calculate the color mapping relationship between the reference acquisition device and the compensation acquisition device by the server based on the received reference acquisition image and compensation acquisition image. As Figure 2 shown, exemplarily, the color mapping relationship can be presented in the form of a LUT or a matrix.

[0093] Step 6: Based on the color mapping relationship calculated in Step 5, perform color compensation on the acquired image during the process of the compensation acquisition device performing virtual and real screen acquisition, and obtain a compensated image with better virtual and real alignment effect.

[0094] Step 7: In order to facilitate the operator to more objectively and quantitatively understand the image compensation effect, it is also possible to output the pixel value difference between the image acquired by the compensation acquisition device before color calibration compensation and the image acquired by the reference acquisition device, and the pixel value difference between the image acquired by the compensation acquisition device after color calibration compensation and the image acquired by the reference acquisition device.

[0095] See Figure 3 , Figure 3 which is a schematic flowchart of data processing according to an embodiment of the present application. The following will further combine Figure 3Briefly describe the specific step process of the data processing method provided in the embodiments of the present application:

[0096] First, build a physical link. For specific content, refer to Figure 2 The corresponding first and second steps; then, use a reference acquisition device to collect images to obtain a reference acquisition image. Further, to ensure image quality, the quality of the reference acquisition image can also be judged to determine whether the reference acquisition image meets the quality requirements. If not, return to collect images again through the reference acquisition device; if it meets the quality requirements, use a compensation acquisition device to collect images to obtain a compensation acquisition image. Further, to ensure image quality, the quality of the compensation acquisition image can also be judged to determine whether the compensation acquisition image meets the quality requirements. If not, return to collect images again through the compensation acquisition device; if the compensation acquisition image meets the quality requirements, the color mapping relationship between the reference acquisition device and the compensation acquisition device can be calculated based on the above reference acquisition image and compensation acquisition image; after obtaining the color mapping relationship, based on the color mapping relationship, color compensation of the collected images can be performed during the process of collecting virtual and real images by the compensation acquisition device to obtain a compensated image. Regarding the color compensation operation, two different correction and compensation methods are provided in the embodiments of the present application: First, perform color calibration compensation in the virtual rendering engine in advance, that is: mount the color mapping relationship in the virtual rendering engine, and after calibrating the color of the virtual rendering engine according to the color calibration mapping relationship corresponding to the compensation acquisition device and the color mapping relationship, then use the compensation acquisition device to collect virtual and real images to obtain the final compensated image; Second, mount the color mapping relationship in the compensation acquisition device. In the virtual image rendering stage, the above color mapping relationship is not introduced. Instead, after collecting the original acquisition image through the compensation acquisition device, the color values of each pixel point in the original acquisition image are corrected and compensated according to the color mapping relationship to obtain a compensated image.

[0097] See Figure 4 , Figure 4 It is a structural block diagram of a data processing device according to an embodiment of the present application. The device includes:

[0098] A device determination module 402, configured to determine a reference acquisition device and a compensation acquisition device among multiple acquisition devices. The reference acquisition device is an acquisition device that has achieved virtual and real color alignment; the compensation acquisition device is other acquisition devices except the reference acquisition device;

[0099] A mapping relationship acquisition module 404 is configured to acquire a color mapping relationship between a reference acquisition device and a compensation acquisition device; the color mapping relationship represents a mapping relationship between a reference color value obtained by the reference acquisition device through picture acquisition for the same display color and a compensation color value obtained by the compensation acquisition device through picture acquisition.

[0100] A compensation module 406 is configured to perform color compensation on the acquired image during the process of the compensation acquisition device acquiring virtual and real pictures based on the color mapping relationship, so as to obtain a compensated image.

[0101] Optionally, in some embodiments, the mapping relationship acquisition module 404 is specifically configured to:

[0102] Acquire a reference acquisition image, where the reference acquisition image is an image obtained by the reference acquisition device through picture acquisition of a target object;

[0103] Acquire a compensation acquisition image, where the compensation acquisition image is an image obtained by the compensation acquisition device through picture acquisition of the target object;

[0104] Respectively acquire the reference color value of the target object in the reference acquisition image and the compensation color value of the target object in the compensation acquisition image;

[0105] Calculate the color mapping relationship between the reference acquisition device and the compensation acquisition device according to the reference color value and the compensation color value.

[0106] Optionally, in some embodiments, the target object includes: a real object and a virtual object displayed on a display screen;

[0107] When the mapping relationship acquisition module 404 executes the steps of respectively acquiring the reference color value of the target object in the reference acquisition image and the compensation color value of the target object in the compensation acquisition image, it is specifically configured to:

[0108] Acquire the reference real color value of the real object in the reference acquisition image and the compensation real color value of the real object in the compensation acquisition image; acquire the reference virtual color value of the virtual object in the reference acquisition image and the compensation virtual color value of the virtual object in the compensation acquisition image;

[0109] When the mapping relationship acquisition module 404 executes the step of calculating the color mapping relationship between the reference acquisition device and the compensation acquisition device according to the reference color value and the compensation color value, it is specifically configured to:

[0110] Calculate the real - scene color mapping relationship between the reference acquisition device and the compensation acquisition device based on the reference real - scene color value and the compensation real - scene color value; calculate the virtual - scene color mapping relationship between the reference acquisition device and the compensation acquisition device based on the reference virtual color value and the compensation virtual color value;

[0111] Fuse the real - scene color mapping relationship and the virtual - scene color mapping relationship to obtain the color mapping relationship between the reference acquisition device and the compensation acquisition device.

[0112] Optionally, in some embodiments, when the mapping relationship acquisition module 404 executes the steps of respectively obtaining the reference color value of the target object in the reference acquisition image and the compensation color value of the target object in the compensation acquisition image, it specifically is used for:

[0113] Respectively determine whether the reference acquisition image and the compensation acquisition image meet the preset image quality conditions;

[0114] If both the reference acquisition image and the compensation acquisition image meet the image quality conditions, then respectively obtain the reference color value of the target object in the reference acquisition image and the compensation color value of the target object in the compensation acquisition image.

[0115] Optionally, in some embodiments, the compensation module 406 is specifically used for:

[0116] Obtain the original compensation virtual - real image; the original compensation virtual - real image is an image obtained by the compensation acquisition device for virtual - real scene acquisition after color - calibrating the virtual rendering engine according to the color calibration mapping relationship corresponding to the compensation acquisition device;

[0117] According to the color mapping relationship, correct and compensate the color values of each pixel point in the original compensation virtual - real image to obtain the compensated image.

[0118] Optionally, in some embodiments, the compensation module 406 is specifically used for:

[0119] Obtain the compensation virtual - real image collected by the compensation acquisition device as the compensated image;

[0120] Wherein, the compensation virtual - real image is an image obtained by the compensation acquisition device for virtual - real scene acquisition after color - calibrating the virtual rendering engine according to the color calibration mapping relationship and the color mapping relationship corresponding to the compensation acquisition device.

[0121] Optionally, in some embodiments, when the compensation module 406 executes the step of correcting and compensating the color values of each pixel point in the original compensation virtual - real image according to the color mapping relationship to obtain the compensated image, it specifically is used for:

[0122] Downsample the original compensated virtual-real image to obtain a sampled image;

[0123] According to the color mapping relationship, correct and compensate the color values of each pixel point in the sampled image to obtain a compensated sampled image;

[0124] Calculate the first pixel value difference between the compensated sampled image and the reference virtual-real image, and the second pixel value difference between the original compensated virtual-real image and the reference virtual-real image; the reference virtual-real image is an image obtained by the reference acquisition device for acquiring virtual-real images;

[0125] If the first pixel value difference and the second pixel value difference meet the preset difference condition, according to the color mapping relationship, correct and compensate the color values of each pixel point in the original compensated virtual-real image to obtain a compensated image.

[0126] The data processing device in this embodiment is used to implement the corresponding data processing method in the foregoing mapping relationship calculation method embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be elaborated here. In addition, the function implementation of each module in the data processing device in this embodiment can refer to the description of the corresponding part in the foregoing method embodiment, which will not be elaborated here either.

[0127] Refer to Figure 5 , which shows a schematic structural diagram of an electronic device according to an embodiment of the present application. The specific implementation of the electronic device is not limited in the specific embodiment of the present application.

[0128] As Figure 5 shown, the control terminal may include: a processor 502, a communication interface 504, a memory 506, and a communication bus 508.

[0129] Wherein:

[0130] The processor 502, the communication interface 504, and the memory 506 communicate with each other through the communication bus 508.

[0131] The communication interface 504 is used to communicate with other electronic devices or servers.

[0132] The processor 502 is used to execute the program 510, and specifically can execute the relevant steps in the foregoing data processing method embodiment.

[0133] Specifically, the program 510 may include program code, and the program code includes computer operation instructions.

[0134] The processor 502 may be a CPU, or a specific application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. One or more processors included in the intelligent device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0135] A memory 506 is used to store a program 510. The memory 506 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0136] The program 510 may include multiple computer instructions. Specifically, the program 510 may cause the processor 502 to perform the operations corresponding to the methods described in the foregoing multiple method embodiments through the multiple computer instructions.

[0137] For the specific implementation of each step in the program 510, reference may be made to the corresponding descriptions in the corresponding steps and units in the foregoing method embodiments, and the corresponding beneficial effects are obtained, which will not be elaborated herein. Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the devices and modules described above may refer to the corresponding process descriptions in the foregoing method embodiments, which will not be elaborated herein.

[0138] The embodiments of the present application further provide a computer storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the methods described in any one of the foregoing multiple method embodiments. The computer storage medium includes, but is not limited to: Compact Disc Read-Only Memory (CD-ROM), Random Access Memory (RAM), floppy disk, hard disk, or magneto-optical disk, etc.

[0139] The embodiments of the present application further provide a computer program product, including computer instructions, where the computer instructions instruct a computing device to perform the operations corresponding to any one of the foregoing multiple method embodiments.

[0140] In addition, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data for training the model, data for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data that have been authorized by the user or fully authorized by all parties. Moreover, the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0141] It should be pointed out that according to the needs of implementation, the various components / steps described in the embodiments of this application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the objectives of the embodiments of this application.

[0142] The methods according to the embodiments of this application can be implemented in hardware, firmware, or be implemented as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or be implemented as computer code that is originally stored in a remote recording medium or a non-transitory machine-readable medium and downloaded through a network and will be stored in a local recording medium. Thus, the methods described herein can be stored as such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA)). It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component (such as a Random Access Memory (RAM), a Read-Only Memory (ROM), a flash memory, etc.) that can store or receive software or computer code. When the software or computer code is accessed and executed by the computer, the processor, or the hardware, the methods described herein are implemented. In addition, when a general-purpose computer accesses the code for implementing the methods shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0143] Those of ordinary skill in the art can realize that the units and method steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the embodiments of this application.

[0144] The above embodiments are only used to illustrate the embodiments of this application, rather than to limit the embodiments of this application. Those of ordinary skill in the relevant technical field can also make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A data processing method, comprising: Determine a reference acquisition device and a compensation acquisition device among a plurality of acquisition devices, wherein the reference acquisition device is an acquisition device that has achieved virtual and real color alignment; The compensation acquisition device is another acquisition device other than the reference acquisition device; Acquire the color mapping relationship between the reference acquisition device and the compensation acquisition device; The color mapping relationship represents, for the same display color, a mapping relationship between a reference color value obtained after a reference acquisition device performs image acquisition and a compensation color value obtained after a compensation acquisition device performs image acquisition; Based on the color mapping relationship, color compensation of the captured image is performed during the process of the compensation capture device capturing the virtual and real images to obtain a compensated image.

2. The method according to claim 1, wherein: The step of acquiring the color mapping relationship between the reference acquisition device and the compensation acquisition device includes: Acquire a reference acquisition image, where the reference acquisition image is an image acquired by a reference acquisition device performing image acquisition on a target object; Acquire a compensated acquisition image, where the compensated acquisition image is an image obtained by performing image acquisition of the target object by a compensated acquisition device; Respectively acquiring a reference color value of the target object in the reference captured image and a compensated color value of the target object in the compensated captured image; A color mapping relationship between a reference acquisition device and a compensation acquisition device is calculated based on the reference color value and the compensation color value.

3. The method according to claim 2, wherein: The target objects include: real scene objects and virtual objects displayed on the display screen; The step of respectively acquiring a reference color value of the target object in the reference acquired image and a compensated color value of the target object in the compensated acquired image comprises: Acquire a reference real scene color value of the real scene object in the reference acquisition image, and a compensated real scene color value of the real scene object in the compensated acquisition image; acquire a reference virtual color value of the virtual object in the reference acquisition image, and a compensated virtual color value of the virtual object in the compensated acquisition image; The step of calculating the color mapping relationship between the reference acquisition device and the compensation acquisition device according to the reference color value and the compensation color value includes: According to the reference real scene color value and the compensated real scene color value, a real scene color mapping relationship between the reference acquisition device and the compensated acquisition device is calculated; according to the reference virtual color value and the compensated virtual color value, a virtual color mapping relationship between the reference acquisition device and the compensated acquisition device is calculated; The real scene color mapping relationship and the virtual color mapping relationship are integrated to obtain a color mapping relationship between a reference acquisition device and a compensation acquisition device.

4. The method according to claim 2, wherein: The step of respectively acquiring a reference color value of the target object in the reference acquired image and a compensated color value of the target object in the compensated acquired image comprises: Determining whether the reference acquired image and the compensated acquired image meet preset image quality conditions respectively; If both the reference acquired image and the compensated acquired image satisfy the image quality condition, a reference color value of the target object in the reference acquired image and a compensated color value of the target object in the compensated acquired image are respectively acquired.

5. The method according to any one of claims 1 to 4, wherein: Based on the color mapping relationship, performing color compensation on the captured image during the process of the compensation acquisition device acquiring the virtual and real images to obtain the compensated image includes: Acquire an original compensated virtual-real image; the original compensated virtual-real image is an image obtained by performing virtual-real picture acquisition by the compensation acquisition device after color calibration of the virtual rendering engine is performed according to the color calibration mapping relationship corresponding to the compensation acquisition device; According to the color mapping relationship, the color value of each pixel in the original compensated virtual-real image is corrected and compensated to obtain a compensated image.

6. The method according to any one of claims 1 to 4, wherein: Based on the color mapping relationship, performing color compensation on the captured image during the process of the compensation acquisition device acquiring the virtual and real images to obtain the compensated image includes: Acquire the compensated virtual and real image acquired by the compensation acquisition device as the compensated image; The compensated virtual-real image is an image obtained by performing virtual-real picture acquisition by the compensation acquisition device after color calibration of the virtual rendering engine is performed according to the color calibration mapping relationship corresponding to the compensation acquisition device and the color mapping relationship.

7. The method according to claim 5, wherein: The method of correcting and compensating the color value of each pixel in the original compensated virtual-real image according to the color mapping relationship to obtain a compensated image includes: Downsampling the original compensated virtual-real image to obtain a sampled image; According to the color mapping relationship, the color value of each pixel in the sampled image is corrected and compensated to obtain a compensated sampled image; Calculating a first pixel value difference between the compensated sampled image and a reference virtual-real image, and a second pixel value difference between the original compensated virtual-real image and the reference virtual-real image; the reference virtual-real image is an image obtained by collecting virtual-real images by a reference collection device; If the first pixel value difference and the second pixel value difference meet a preset difference condition, the color value of each pixel in the original compensated virtual-real image is corrected and compensated according to the color mapping relationship to obtain a compensated image.

8. A data processing device, comprising: A device determination module, used to determine a reference acquisition device and a compensation acquisition device among a plurality of acquisition devices, wherein the reference acquisition device is an acquisition device that has achieved virtual and real color alignment; The compensation acquisition device is another acquisition device other than the reference acquisition device; A mapping relationship acquisition module, used to acquire the color mapping relationship between the reference acquisition device and the compensation acquisition device; The color mapping relationship represents, for the same display color, a mapping relationship between a reference color value obtained after a reference acquisition device performs image acquisition and a compensation color value obtained after a compensation acquisition device performs image acquisition; The compensation module is used to perform color compensation on the captured image based on the color mapping relationship during the process of the compensation acquisition device acquiring the virtual and real images, so as to obtain a compensated image.

9. An electronic device, comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method according to any one of claims 1-7.

10. A computer storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product comprising computer instructions, wherein the computer instructions instruct a computing device to execute the method according to any one of claims 1 to 7.

Images