Color calibration method, device and equipment and computer storage medium

Abstract

The embodiment of the invention provides a color calibration method and device, equipment and a computer storage medium. The color calibration method comprises the following steps: acquiring a calibration reference image and an original acquisition image; determining a reference target area of the target object in the calibration reference image and an original target area of the target object in the original acquisition image; calculating reference color information and original color information; the reference color information represents color features of the reference target area; the original color information represents color features of the original target area; and according to the information difference between the reference color information and the original color information, performing color calibration on the original acquisition image to obtain a calibration image, and constructing a digital model of the target object based on the calibration image. According to the embodiment of the invention, the color of the digital model is more natural and real, and the quality of the digital model is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a color calibration method, device, electronic device, computer storage medium, and computer program product. Background Art

[0002] 3D scanning virtual assets refer to digitized objects or models of real objects obtained by taking all-round photos of real objects in the real world through image acquisition equipment, and then using these photos and 3D reconstruction tools or algorithms to reconstruct the real objects in 3D. The above-mentioned digitized objects or models usually contain a grid representing the spatial structure of the real object, and a layer of colored texture attached to the grid.

[0003] At present, the three-dimensional scanning virtual assets obtained using relevant technologies, that is, the constructed digital models, generally have the problem of unnatural or unrealistic model colors. Summary of the invention

[0004] In view of this, an embodiment of the present application provides a color calibration solution to at least partially solve the above problems.

[0005] According to a first aspect of an embodiment of the present application, a color calibration method is provided, comprising:

[0006] Acquire a calibration reference image and an original captured image; both the calibration reference image and the original captured image contain a target object;

[0007] Determine a reference target region of the target object in the calibration reference image and an original target region of the target object in the original acquired image;

[0008] Calculating reference color information and original color information; the reference color information represents the color characteristics of the reference target area; the original color information represents the color characteristics of the original target area;

[0009] According to the information difference between the reference color information and the original color information, the original captured image is color calibrated to obtain a calibrated image, so as to construct a digital model of the target object based on the calibrated image.

[0010] According to a second aspect of an embodiment of the present application, a color calibration device is provided, including:

[0011] An image acquisition module, used to acquire a calibration reference image and an original acquired image; both the calibration reference image and the original acquired image contain a target object;

[0012] A region determination module, used to determine a reference target region of the target object in the calibration reference image, and an original target region of the target object in the original acquired image;

[0013] A color information calculation module, used to calculate reference color information and original color information; the reference color information represents the color characteristics of the reference target area; the original color information represents the color characteristics of the original target area;

[0014] The color calibration module is used to perform color calibration on the original captured image according to the information difference between the reference color information and the original color information to obtain a calibrated image, so as to construct a digital model of the target object based on the calibrated image.

[0015] According to the third aspect of an embodiment of the present application, there is provided an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method described in the first aspect.

[0016] According to a fourth aspect of an embodiment of the present application, a computer storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described in the first aspect is implemented.

[0017] According to the color calibration scheme provided in the embodiment of the present application, an original captured image and a calibration reference image whose color information matches the actual scene are respectively acquired for the target object; then, a reference target area where the target object is located in the calibration reference image and an original target area where the target object is located in the original captured image are detected; then, based on the difference between the color features in the reference target area and the color features in the original target area, a color calibration adjustment is performed on the original captured image, thereby obtaining a calibrated image with more natural and realistic colors.

[0018] In the embodiment of the present application, the color information of the target object in the calibration reference image is used as a reference, and the information difference between the color information of the target object in the calibration reference image and the color information of the target object in the original acquisition image is used as a calibration basis. The color of the calibrated image can be made more natural and realistic. Furthermore, the model of the target object is reconstructed based on the calibrated image with more natural and realistic colors, so that the color of the digital model of the target object obtained in the end can be made more natural and realistic, thereby improving the quality of the digital model obtained in the end.

[0019] At the same time, as described above, in the embodiment of the present application, color calibration is performed based on the difference in color information between the calibration reference image and the original acquired image. That is to say, the focus in the example of the present application is on the difference in color information between the calibration reference image and the original acquired image, and whether the shooting conditions of the calibration reference image and the original acquired image (such as: ambient lighting conditions, equipment hardware parameters, etc.) are the same does not affect the specific implementation of the embodiment of the present application, that is: in the embodiment of the present application, the shooting conditions of the calibration reference image and the shooting conditions of the original acquired image can be different. Therefore, the embodiment of the present application reduces the difficulty of image acquisition operations and has high feasibility.

[0020] In addition, the embodiment of the present application performs color calibration on the two-dimensional original captured images captured for the target object as independent calibration objects. In the actual acquisition process, different original captured images can usually correspond to different shooting angles. Therefore, compared with the related solutions of first building a three-dimensional digital model and then performing color calibration on the three-dimensional digital model as a whole, the embodiment of the present application performs color calibration on each two-dimensional original captured image captured from different shooting angles, which can also solve the color difference problem between different areas of the three-dimensional digital model caused by multi-angle shooting for the same target object, and further improve the quality of the final digital model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0022] Figure 1 is a flowchart of a color calibration method according to an embodiment of the present application;

[0023] Figure 2 A schematic diagram of a shooting scene of an image to be color-adjusted;

[0024] Figure 3 A schematic diagram of an example scenario according to an embodiment of the present application;

[0025] Figure 4 is a three-dimensional reconstruction link flow chart according to an embodiment of the present application;

[0026] Figure 5 is a structural block diagram of a color calibration device according to an embodiment of the present application;

[0027] Figure 6 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0029] The specific implementation of the embodiment of the present application is further explained below in conjunction with the accompanying drawings of the embodiment of the present application.

[0030] Reference Figure 1 , Figure 1 1 is a flowchart of a color calibration method according to an embodiment of the present application. Specifically, the color calibration method provided in this embodiment includes the following steps:

[0031] Step 102: Acquire a calibration reference image and an original captured image.

[0032] The calibration reference image and the original captured image are images containing the target object, that is, the calibration reference image and the original captured image are both images captured for the target object.

[0033] Specifically, the target object in the embodiment of the present application can be any real object with a fixed shape and color to be digitally modeled, for example, a seat or sofa in an indoor scene, or a building or stone sculpture in an outdoor scene.

[0034] When constructing a digital model of a target object, images of the target object can be first collected at different angles by using an image acquisition device such as a camera, and then the collected images can be processed by a related 3D reconstruction tool or algorithm to obtain a digital model corresponding to the target object. To facilitate subsequent distinction, in the embodiments of the present application, the unprocessed original image obtained by collecting images of the target object at different angles by using an image acquisition device such as a camera can be referred to as an original collected image.

[0035] The calibration reference image in the embodiment of the present application can be an image captured for the target object to be digitally modeled. The difference from the original captured image is that the calibration reference image can be an image that has been color-adjusted to achieve mutual alignment between the image color and the real object color.

[0036] In the embodiments of the present application, there is no limitation on the method of obtaining the calibration reference image. For example, after acquiring the captured image by performing image acquisition on the target object, the color of the captured image can be manually adjusted according to experience or personal visual aesthetics to obtain the calibration reference image; or, with the help of a reference object whose actual color information is known, the actual color information of the reference object can be used as the basis and target for color adjustment to adjust the color of the captured image to obtain the calibration reference image.

[0037] Furthermore, as for the method of obtaining the calibration reference image by means of the reference object, the specific implementation process may include:

[0038] Acquire an image to be color-adjusted including a target object and a reference object, wherein actual color information of the reference object is known;

[0039] Performing reference object detection on the image to be toned, and determining a reference area of ​​the reference object in the image to be toned;

[0040] The image to be toned is subjected to color adjustment to obtain a calibration reference image; in the calibration reference image, the color information of the reference area is consistent with the actual color information of the reference object.

[0041] Specifically, with respect to a certain object, the color information of the object may refer to the color characteristics of the object, that is, the overall characteristics or overall rules of the object in terms of color values. The specific content of the color information is not limited here. Exemplarily, the color information may include: any one or more pieces of information with high visual perception, such as brightness information and hue information; further, the color information may also include other specific quantitative value information obtained by performing certain numerical calculations based on the color values ​​of the object in each color channel, and so on.

[0042] Since the color information of an object is the overall characteristics or overall rules of the object in terms of color value, the color information of the object can be obtained if the color value of the object is known. Based on the above analysis, in order to reduce the difficulty of implementation, a physical object with a known color value can be selected as a reference object, for example, a color card with a known color value (the color card can be a monochrome card or a multi-color card containing multiple color values ​​at the same time) can be selected as a reference object.

[0043] In the embodiment of the present application, the target object to be digitally modeled and the reference object with known color information can be placed in the same shooting picture, so that the image to be color-adjusted containing both the target object and the reference object can be acquired by the image acquisition device. Figure 2 , Figure 2 is a schematic diagram of the shooting scene of the image to be color-adjusted, Figure 2 The target object in the image is a stone tower, and the reference object is a color card containing multiple color values ​​placed on the stone tower. The stone tower and the color card can be photographed by an image acquisition device such as a camera to obtain an image to be adjusted.

[0044] After obtaining the image to be colored, the reference area where the reference object in the image to be colored is located can be determined by object detection, and then the color value of each pixel in the image to be colored can be adjusted so that the color information of the reference area in the adjusted image is consistent with the actual color information of the reference object. At this point, the adjusted image can be used as a calibration reference image in the embodiment of the present application.

[0045] The above method of obtaining a calibration reference image does not require human intervention, but adopts an algorithm-adaptive method to obtain the calibration reference image, so it is more efficient; more importantly, the above process, by introducing a reference object with known color information, specifically quantifies the goal of the color adjustment process of the image to be adjusted as follows: making the color information of the reference area in the color-adjusted image consistent with the actual color information of the reference object, that is, the basis for color adjustment is accurately quantified, so that the color of the reference object in the final calibration reference image is more realistic and natural, that is, the accuracy of the calibration reference image is improved.

[0046] Step 104 : determining a reference target region of the target object in the calibration reference image and an original target region of the target object in the original acquired image.

[0047] Specifically, through step 104, the area information of the target object in the calibration reference image and the area information of the target object in the original acquired image can be obtained respectively. In the example of the present application, any suitable object segmentation or object detection scheme can be used to determine the area information where the target object is located from each image. Exemplarily, the reference target area of ​​the target object in the calibration reference image and the original target area of ​​the target object in the original acquired image can be determined with the help of a machine learning model that can realize target object detection; non-machine learning algorithms, such as edge detection algorithms, morphological processing algorithms, etc., can also be used to determine the reference target area and the original target area.

[0048] Step 106, calculating reference color information and original color information.

[0049] Specifically, the reference color information can represent the color characteristics of the reference target area, that is, the overall characteristics or overall rules of the reference target area in the calibration reference image in terms of color values. Correspondingly, the original color information can represent the color characteristics of the original target area, that is, the overall characteristics or overall rules of the original target area in the original captured image in terms of color values.

[0050] The specific content of the color information is not limited here. For example, the color information may include: any one or more of the information with high visual perception, such as brightness information and hue information; further, the color information may also include other specific quantitative value information obtained by performing certain numerical calculations based on the color values ​​of the object in each color channel, and so on.

[0051] Step 108 , color calibration is performed on the original captured image according to the information difference between the reference color information and the original color information to obtain a calibrated image, so as to construct a digital model of the target object based on the calibrated image.

[0052] Specifically, as mentioned above, the color information represents the overall characteristics or overall rules of the region in terms of color values. Therefore, after obtaining the reference color information and the original color information, the original captured image can be color calibrated according to the information difference between the two.

[0053] The specific calibration method may be: adjusting the color values ​​of the pixels in the original captured image, that is, adjusting the overall color values ​​of the original captured image to obtain an adjusted captured image; then calculating the adjusted color information of the area where the target object is located in the adjusted captured image, and comparing the adjusted color information with the reference color information of the reference target area; if the information difference between the adjusted color information and the reference color information is small (exemplarily, such as less than a preset difference threshold), the adjusted captured image is used as the calibration image in the embodiment of the present application; conversely, if the information difference between the adjusted color information and the reference color information is large (exemplarily, such as greater than or equal to a preset difference threshold), the color adjustment of the adjusted captured image continues, and based on the new adjusted captured image, the steps of calculating the adjusted color information of the area where the target object is located in the adjusted captured image and comparing the adjusted color information with the reference color information are performed until the information difference between the adjusted color information and the reference color information is small, thereby obtaining a calibrated image.

[0054] Optionally, the specific calibration method may also be: quantifying a specific color value adjustment amount based on the information difference between the reference color information and the original color information, and then adjusting the overall color value of the original captured image based on the color value adjustment amount, thereby obtaining a calibrated image with a smaller difference between the color information and the reference color information.

[0055] See also Figure 3 , Figure 3 Schematic diagram of a scenario example according to an embodiment of the present application. Figure 3 The specific implementation process of the color calibration method provided in the embodiment of the present application is briefly described:

[0056] A calibration reference image including a target object and a plurality of original acquired images taken from different shooting angles for the target object are obtained; position information of a reference target area where the target object is located in the calibration reference image and position information of an original target area where the target object is located in the original acquired image are determined; the calibration reference image, the plurality of original acquired images, and the position information of the reference target area and the position information of the original target area are input as input conditions into a color calibration tool for executing the color calibration method provided in an embodiment of the present application, and the color calibration tool calculates reference color information corresponding to the reference target area and original color information corresponding to the original target area in each original acquired image; and color calibration is performed on each original acquired image according to the information difference between the reference color information and each original color information, thereby outputting a calibrated image corresponding to each original acquired image.

[0057] According to the color calibration scheme provided in the embodiment of the present application, an original captured image and a calibration reference image whose color information matches the actual scene are respectively acquired for the target object; then, a reference target area where the target object is located in the calibration reference image and an original target area where the target object is located in the original captured image are detected; then, based on the difference between the color features in the reference target area and the color features in the original target area, a color calibration adjustment is performed on the original captured image, thereby obtaining a calibrated image with more natural and realistic colors.

[0058] In the embodiment of the present application, the color information of the target object in the calibration reference image is used as a reference, and the information difference between the color information of the target object in the calibration reference image and the color information of the target object in the original acquisition image is used as a calibration basis. The color of the calibrated image can be made more natural and realistic. Furthermore, the model of the target object is reconstructed based on the calibrated image with more natural and realistic colors, so that the color of the digital model of the target object obtained in the end can be made more natural and realistic, thereby improving the quality of the digital model obtained in the end.

[0059] At the same time, as mentioned above, in the embodiment of the present application, color calibration is performed based on the difference in color information between the calibration reference image and the original acquired image. That is to say, the focus in the example of the present application is on the difference in color information between the calibration reference image and the original acquired image, and whether the shooting conditions of the calibration reference image and the original acquired image (such as: ambient lighting conditions, equipment hardware parameters, etc.) are the same does not affect the specific implementation of the embodiment of the present application, that is: in the embodiment of the present application, the shooting conditions of the calibration reference image and the shooting conditions of the original acquired image can be different. Therefore, the embodiment of the present application reduces the difficulty of image acquisition operation and has high feasibility.

[0060] In addition, the embodiment of the present application performs color calibration on the two-dimensional original captured images captured for the target object as independent calibration objects. In the actual acquisition process, different original captured images can usually correspond to different shooting angles. Therefore, compared with the related solutions of first building a three-dimensional digital model and then performing color calibration on the three-dimensional digital model as a whole, the embodiment of the present application performs color calibration on each two-dimensional original captured image captured from different shooting angles, which can also solve the color difference problem between different areas of the three-dimensional digital model caused by multi-angle shooting for the same target object, and further improve the quality of the final digital model.

[0061] In addition, for multiple target objects made of the same color material in the same scene, digital models are constructed separately, and there is usually a color difference problem between the digital models corresponding to the constructed target objects. In this case, the color calibration method provided in the embodiment of the present application can be executed for each target object separately, and the calibration reference images corresponding to different target objects are uniformly adjusted in color, so that the calibration reference images corresponding to different target objects maintain a high consistency in color information, ensuring that in the process of constructing the digital model of each target object, color calibration is performed based on the same benchmark and basis, thereby improving the color difference problem between the digital models corresponding to the above-mentioned multiple target objects.

[0062] The color calibration method of this embodiment can be executed by any appropriate electronic device with data processing capability, including but not limited to: a server, a PC, etc.

[0063] Optionally, in some of the embodiments, the process of calculating the reference color information and the original color information may specifically include:

[0064] Performing fusion calculation on the color value of each pixel in the reference target area under the preset color channel to obtain the target color value of the reference target area under the preset color channel;

[0065] Obtaining a reference color information value based on a target color value of a reference target area under a preset color channel;

[0066] Performing fusion calculation on the color value of each pixel in the original target area under the preset color channel to obtain the original color value of the original target area under the preset color channel;

[0067] Based on the original color value of the original target area under the preset color channel, the original color information value is obtained.

[0068] Specifically, since color information represents the overall characteristics or overall rules of a region in terms of color values. Therefore, when calculating the reference color information value, the color value of each pixel in the reference target area can be comprehensively considered. By fusing the color values ​​of each pixel in the reference target area, a more accurate reference color information value corresponding to the reference target area can be obtained, which improves the accuracy of the reference color information value. Correspondingly, when calculating the original color information value, the color value of each pixel in the original target area can be comprehensively considered. By fusing the color values ​​of each pixel in the original target area, a more accurate original color information value corresponding to the original reference area can be obtained, which improves the accuracy of the original color information value.

[0069] In the embodiment of the present application, the specific fusion strategy adopted for the fusion calculation of the color value of each pixel under the preset color channel is not limited. For example, the mean of the color value of each pixel under the preset color channel can be calculated as the target color value of the region under the preset color channel; the median of the color value of each pixel under the preset color channel can also be calculated as the target color value of the region under the preset color channel; the mode of the color value of each pixel under the preset color channel can also be calculated as the target color value of the region under the preset color channel, and so on.

[0070] In addition, when the color values ​​of each pixel are fused and calculated to obtain color information, the calculation is performed with the color channel as the calculation granularity, that is, the target color value of the reference target area under the preset color channel is used as the reference color information value, and the original color value of the original reference area under the preset color channel is used as the original color information value. Therefore, the information difference between the reference color information and the original color information obtained by subsequent calculation represents the coarser-grained color calibration mapping relationship at the color channel level between the calibration reference image and the original acquired image. Color calibration can be performed based on the above-mentioned coarser-grained mapping relationship, which can effectively reduce the workload of color calibration.

[0071] For example: the number of preset color channels can be set to 1. For example, the preset color channel can be the R color channel. The calculated reference color information value is 10, and the calculated original color information value is 8. Then the mean value of each pixel in the reference target area on the R color channel is 10, and the mean value of each pixel in the original target area on the R color channel is 8; the difference color value between the original target area and the reference target area on the R color channel is 2. At this time, the color calibration of the original captured image can be achieved by uniformly increasing the color value of each pixel in the original target area under the R color channel by 2.

[0072] For another example, the number of preset color channels may be set to be multiple. For example, the preset color channels may include: R color channel, G color channel and at least 2 color channels of B color channel. Afterwards, for each color channel among the above-mentioned multiple color channels, the target color value of the reference target area under the color channel and the original color value of the original target area under the color channel may be calculated respectively. The difference color value between the target area and the reference target area on each color channel may be calculated respectively, and a color value correction operation may be performed on each color channel respectively, that is, the difference color value on the corresponding color channel may be added to the color value of the pixel point in the original target area under each color channel respectively, thereby realizing color calibration of the original captured image.

[0073] Compared with a method of calculating corresponding compensation data for the color value of each pixel in the original target area, the embodiment of the present application can effectively reduce the workload of color calibration.

[0074] Optionally, in some of these embodiments, the number of preset color channels is multiple;

[0075] Based on the target color value of the reference target area under the preset color channel, a reference color information value is obtained, including:

[0076] The target color value of the reference target area in each preset color channel is used as an independent variable, and a preset function calculation formula is used to calculate the reference color information;

[0077] Based on the original color value of the original target area under the preset color channel, the original color information value is obtained, including:

[0078] The original color value of the original target area in each preset color channel is used as an independent variable, and a preset function calculation formula is used to calculate the original color information.

[0079] Specifically, when obtaining color information, the above embodiment takes multiple color channels into consideration at the same time, performs fusion calculation based on the color values ​​under the multiple color channels, and uses the calculation result as the color information. Compared with the method of using the color value under a single color channel as the color information value, the above method of fusing multiple color channels to obtain color information can improve the accuracy of the color information finally obtained.

[0080] In the embodiments of the present application, the specific form of the preset function calculation formula is not limited, and it can be customized according to actual conditions or experience. For example, the average of the color values ​​under multiple preset color channels can be calculated to obtain color information; the color values ​​under multiple preset color channels can also be calculated by performing difference or summation calculations on each of them, so as to sum or otherwise fuse the calculation results to obtain color information, etc.

[0081] Optionally, in some of the embodiments, the process of performing color calibration on the original captured image to obtain the calibrated image according to the information difference between the reference color information and the original color information may specifically include:

[0082] determining a numerical relationship between the reference color information and the original color information;

[0083] According to the numerical relationship, the color value of each pixel in the original target area of ​​the original acquired image under the preset color channel is adjusted to obtain a calibrated image.

[0084] Specifically, after obtaining the reference color information and the original color information, the numerical relationship between the two can be obtained, and then the color value of each pixel in the original target area under the preset color channel can be adjusted according to the above numerical relationship to obtain a calibrated image with color information consistent with the reference color information of the calibration reference image.

[0085] Exemplarily, the numerical relationship may be the difference between the reference color information and the original color information. After obtaining the difference, the color value of each pixel in the original target area under the preset color channel may be offset and compensated based on the difference, thereby obtaining a calibrated image. For example, if the difference between the reference color information and the original color information is 2, the color value of each pixel in the original target area under the preset color channel may be uniformly added with 2, thereby obtaining a compensated image.

[0086] The above process calculates the numerical relationship between the reference color information and the original color information, and then quantitatively adjusts the color of the pixels of the original captured image according to the numerical relationship, thereby more accurately achieving information alignment between the calibration image color information and the calibration reference image reference color information.

[0087] Optionally, in some of the embodiments, the process of determining a reference target area of ​​the target object in the calibration reference image and an original target area of ​​the target object in the original acquired image may specifically include:

[0088] The calibrated reference image and the original acquired image are used to perform three-dimensional reconstruction to obtain a three-dimensional reconstructed model of the target object;

[0089] Determine a reference target area of ​​the target object in the calibration reference image according to a correspondence between each position point in the three-dimensional reconstruction model and a pixel point in the calibration reference image;

[0090] According to the correspondence between each position point in the three-dimensional reconstructed model and the pixel points in the original acquired image, the original target area of ​​the target object in the original acquired image is determined.

[0091] Specifically, relevant 3D reconstruction software can be used to perform a 3D reconstruction algorithm based on the calibration reference image and the original acquired image, thereby obtaining a 3D reconstructed model of the target object, which can be a 3D point cloud model. The specific reconstruction process may include the following steps: pose alignment, feature point matching, point cloud generation, and mapping relationship establishment.

[0092] Among them, the posture alignment step is mainly used to obtain the camera posture information corresponding to each image. Specifically, feature point matching can be achieved through adaptive feature point detection algorithm, global optimization algorithm, etc., and the relative position between each feature point can be determined to obtain the camera posture information; as for the feature point matching process, the entity feature points can be marked in the target object in advance. After obtaining the calibration reference image and the original acquired image containing the target object, the virtual feature points corresponding to the entity feature points in each image can be detected, and the virtual feature point pairs corresponding to the same entity feature points can be found in each image; the point cloud generation step refers to the process of generating a point cloud in three-dimensional space according to the above-mentioned camera posture and feature point matching results. The above-mentioned point cloud will eventually constitute a three-dimensional reconstruction model. In the above-mentioned point cloud generation process, the correspondence between each point cloud in the three-dimensional reconstruction model and the pixel points in each image can also be obtained.

[0093] According to the above description of the three-dimensional reconstruction process, the three-dimensional reconstruction process itself can obtain the correspondence between each point cloud in the three-dimensional reconstruction model and the pixel points in each image. In the above embodiments of the present application, with the help of the above correspondence that can be obtained by the three-dimensional reconstruction process, without the need to perform other additional operations, the reference target area of ​​the target object in the calibration reference image and the original target area of ​​the target object in the original acquisition image can be quickly and efficiently determined. In other words, the above embodiments of the present application make full use of the intermediate information generated in the three-dimensional reconstruction process to determine the target area, which can effectively save computing resources and improve the efficiency of target area determination.

[0094] See also Figure 4 , Figure 4 A three-dimensional reconstruction link flow chart according to an embodiment of the present application.

[0095] First, combine Figure 4 The construction process of the digital model is explained as follows: when building a digital model of a target object, the target object can be scanned from different angles to obtain multiple captured images. The captured images at this time are usually raw data captured from the image sensor in the image acquisition device, such as data in RAW (raw material) format; after obtaining the image data in RAW format, the image format can be converted to obtain a more general and easy-to-store image format, such as JPG (Joint Photographic Experts Group) format; then, the converted image is 3D reconstructed to output the digital model of the target object. The 3D reconstruction process is as follows: based on the pose alignment of each converted image, the camera pose corresponding to each converted image is determined; according to each converted image and the camera pose information of each converted image, the relevant 3D reconstruction software is used to perform 3D reconstruction to obtain a 3D reconstructed model and texture coloring information corresponding to the 3D reconstructed model. At this point, the construction of the digital model is completed, and the digital model containing the 3D reconstructed model and texture coloring information can be output.

[0096] Secondly, combined with Figure 4 The three-dimensional reconstruction process of the color calibration method provided in the embodiment of the present application is briefly described. Compared with the construction process of the digital model in the first aspect, the three-dimensional reconstruction process of the color calibration method provided in the embodiment of the present application adds Figure 4 The two steps shown in the dotted box are: an image color adjustment step, and a color calibration step. Specifically, the image color adjustment step may refer to selecting one of the multiple acquired images obtained by scanning for color adjustment, thereby obtaining a calibration reference image; thereafter, three-dimensional reconstruction may be performed based on the calibration reference image and the original acquired image to obtain a three-dimensional reconstructed model of the target object; the color calibration step may refer to the color calibration solution provided in any of the above embodiments of the present application, through which a calibration image may be obtained, and then the above-constructed three-dimensional model may be texture colored according to the calibration image, thereby outputting a digital model with more realistic and natural colors.

[0097] Furthermore, after scanning the target object from different azimuth angles to obtain multiple captured images in raw data format captured by the image sensor in the image acquisition device, in addition to converting the format of the above-mentioned captured images, the color gamut of each image can also be adjusted to a unified color gamut space, illustratively, it can be a LAB color gamut space, sRGB (Standard Red Green Blue, standard red, green and blue) color gamut space, etc. After that, subsequent color calibration and three-dimensional reconstruction operations are performed on the images adjusted in format and color gamut space. Through the above-mentioned color gamut space unification operation, the difference between the image color information caused by the different color gamut spaces of each image can be effectively avoided, and the accuracy of the color calibration solution provided in the embodiment of the present application is further improved.

[0098] See also Figure 5 , Figure 5 : is a structural block diagram of a color calibration device according to an embodiment of the present application. The device includes:

[0099] The image acquisition module 502 is used to acquire a calibration reference image and an original captured image; both the calibration reference image and the original captured image contain the target object;

[0100] The region determination module 504 is used to determine the reference target region of the target object in the calibration reference image and the original target region of the target object in the original acquired image;

[0101] The color information calculation module 506 is used to calculate reference color information and original color information; the reference color information represents the color characteristics of the reference target area; the original color information represents the color characteristics of the original target area;

[0102] The color calibration module 508 is used to perform color calibration on the original captured image according to the information difference between the reference color information and the original color information to obtain a calibrated image, so as to construct a digital model of the target object based on the calibrated image.

[0103] Optionally, in some of the embodiments, the color information calculation module 506 is specifically configured to:

[0104] Performing fusion calculation on the color value of each pixel in the reference target area under the preset color channel to obtain the target color value of the reference target area under the preset color channel;

[0105] Obtaining a reference color information value based on a target color value of a reference target area under a preset color channel;

[0106] Performing fusion calculation on the color value of each pixel in the original target area under the preset color channel to obtain the original color value of the original target area under the preset color channel;

[0107] Based on the original color value of the original target area under the preset color channel, the original color information value is obtained.

[0108] Optionally, in some of these embodiments, the number of preset color channels is multiple;

[0109] The color information calculation module 506, when executing the step of obtaining the reference color information value based on the target color value of the reference target area under the preset color channel, is specifically used to:

[0110] The target color value of the reference target area in each preset color channel is used as an independent variable, and a preset function calculation formula is used to calculate the reference color information;

[0111] The color information calculation module 506, when executing the step of obtaining the original color information value based on the original color value of the original target area under the preset color channel, is specifically used to:

[0112] The original color value of the original target area in each preset color channel is used as an independent variable, and a preset function calculation formula is used to calculate the original color information.

[0113] Optionally, in some of the embodiments, the color calibration module 508, when executing the step of performing color calibration on the original captured image according to the information difference between the reference color information and the original color information to obtain the calibrated image, is specifically configured to:

[0114] determining a numerical relationship between the reference color information and the original color information;

[0115] According to the numerical relationship, the color value of each pixel in the original target area of ​​the original acquired image under the preset color channel is adjusted to obtain a calibrated image.

[0116] Optionally, in some embodiments, the region determination module 504 is specifically configured to:

[0117] The calibrated reference image and the original acquired image are used to perform three-dimensional reconstruction to obtain a three-dimensional reconstructed model of the target object;

[0118] Determine a reference target area of ​​the target object in the calibration reference image according to a correspondence between each position point in the three-dimensional reconstruction model and a pixel point in the calibration reference image;

[0119] According to the correspondence between each position point in the three-dimensional reconstructed model and the pixel points in the original acquired image, the original target area of ​​the target object in the original acquired image is determined.

[0120] Optionally, in some of the embodiments, the image acquisition module 502, when executing the step of acquiring a calibration reference image containing a target object, is specifically configured to:

[0121] Acquire an image to be color-adjusted including a target object and a reference object, wherein the actual color information of the reference object is known;

[0122] Performing reference object detection on the image to be toned, and determining a reference area of ​​the reference object in the image to be toned;

[0123] The image to be toned is subjected to color adjustment to obtain a calibration reference image; in the calibration reference image, the color information of the reference area is consistent with the actual color information of the reference object.

[0124] The color calibration device of this embodiment is used to implement the corresponding color calibration method in the above color calibration method embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here. In addition, the functional implementation of each module in the color calibration device of this embodiment can refer to the description of the corresponding part in the above method embodiment, which will not be described in detail here.

[0125] Reference Figure 6 , shows a schematic diagram of the structure of an electronic device according to an embodiment of the present application. The specific embodiment of the present application does not limit the specific implementation of the electronic device.

[0126] like Figure 6 As shown, the control terminal may include: a processor (processor) 602, a communication interface (Communications Interface) 604, a memory (memory) 606, and a communication bus 608.

[0127] in:

[0128] The processor 602 , the communication interface 604 , and the memory 606 communicate with each other via a communication bus 608 .

[0129] The communication interface 604 is used to communicate with other electronic devices or servers.

[0130] The processor 602 is used to execute the program 610, and specifically can execute the relevant steps in the above method embodiment.

[0131] Specifically, the program 610 may include program codes, which include computer operation instructions.

[0132] The processor 602 may be a CPU, or an application specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0133] The memory 606 is used to store the program 610. The memory 606 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0134] The program 610 may include multiple computer instructions. Specifically, the program 610 may enable the processor 602 to execute operations corresponding to the methods described in the aforementioned multiple method embodiments through the multiple computer instructions.

[0135] The specific implementation of each step in program 610 can refer to the corresponding description of the corresponding steps and units in the above method embodiment, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above method embodiment, which will not be repeated here.

[0136] The present application also provides a computer storage medium on which a computer program is stored, and when the program is executed by a processor, the method described in any of the above-mentioned multiple method embodiments is implemented. The computer storage medium includes but is not limited to: a compact disc read-only memory (CD-ROM), a random access memory (RAM), a floppy disk, a hard disk or a magneto-optical disk, etc.

[0137] An embodiment of the present application also provides a computer program product, including computer instructions, which instruct a computing device to execute operations corresponding to any one of the above-mentioned multiple method embodiments.

[0138] In addition, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used to train the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0139] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0140] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or implemented as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk or magneto-optical disk), or implemented as a computer code originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded through a network and stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor or programmable or dedicated hardware (such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA)). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., a random access memory (RAM), a read-only memory (ROM), a flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown here, the execution of the code converts the general-purpose computer into a dedicated computer for executing the method shown here.

[0141] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present application.

[0142] The above implementation methods are only used to illustrate the embodiments of the present application, and are not limitations on the embodiments of the present application. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present application. The scope of patent protection of the embodiments of the present application should be limited by the claims.

Claims

1. A color calibration method, comprising: Obtaining a calibration reference image and an original acquisition image; The calibration reference image and the original acquired image both contain a target object; Determine a reference target region of the target object in the calibration reference image and an original target region of the target object in the original acquired image; Calculate reference color information and original color information; The reference color information represents the color characteristics of the reference target area; the original color information represents the color characteristics of the original target area; According to the information difference between the reference color information and the original color information, the original captured image is color calibrated to obtain a calibrated image, so as to construct a digital model of the target object based on the calibrated image.

2. The method according to claim 1, wherein: The calculation of reference color information and original color information includes: Performing fusion calculation on the color value of each pixel in the reference target area under a preset color channel to obtain a target color value of the reference target area under the preset color channel; Obtaining a reference color information value based on a target color value of the reference target area in the preset color channel; Performing fusion calculation on the color value of each pixel in the original target area under the preset color channel to obtain the original color value of the original target area under the preset color channel; Based on the original color value of the original target area in the preset color channel, an original color information value is obtained.

3. The method according to claim 2, wherein: The number of the preset color channels is multiple; The obtaining of the reference color information value based on the target color value of the reference target area in the preset color channel includes: The target color value of the reference target area in each preset color channel is used as an independent variable, and a preset function calculation formula is used to calculate and obtain reference color information; The obtaining of the original color information value based on the original color value of the original target area in the preset color channel includes: The original color value of the original target area in each preset color channel is used as an independent variable, and the preset function calculation formula is used to calculate and obtain the original color information.

4. The method according to claim 2 or 3, wherein: The color calibration of the original captured image is performed according to the information difference between the reference color information and the original color information to obtain a calibrated image, including: Determining a numerical relationship between the reference color information and the original color information; According to the numerical relationship, the color value of each pixel in the original target area of ​​the original acquired image under the preset color channel is adjusted to obtain a calibrated image.

5. The method according to any one of claims 1 to 3, wherein: The step of determining a reference target area of ​​the target object in the calibration reference image and an original target area of ​​the target object in the original acquired image comprises: Performing three-dimensional reconstruction using the calibration reference image and the original acquired image to obtain a three-dimensional reconstructed model of the target object; Determining a reference target area of ​​a target object in the calibration reference image according to a correspondence between each position point in the three-dimensional reconstructed model and a pixel point in the calibration reference image; According to the correspondence between each position point in the three-dimensional reconstructed model and the pixel points in the original acquired image, an original target area of ​​the target object in the original acquired image is determined.

6. The method according to any one of claims 1 to 3, wherein: The obtaining of a calibration reference image containing the target object comprises: Acquire an image to be color-adjusted including a target object and a reference object, wherein actual color information of the reference object is known; Performing reference object detection on the image to be color-adjusted to determine a reference area of ​​the reference object in the image to be color-adjusted; The image to be color-adjusted is color-adjusted to obtain a calibration reference image; in the calibration reference image, the color information of the reference area is consistent with the actual color information of the reference object.

7. A color calibration device, comprising: An image acquisition module is used to acquire a calibration reference image and an original acquisition image; The calibration reference image and the original acquired image both contain a target object; A region determination module, used to determine a reference target region of the target object in the calibration reference image, and an original target region of the target object in the original acquired image; A color information calculation module, used to calculate reference color information and original color information; The reference color information represents the color characteristics of the reference target area; the original color information represents the color characteristics of the original target area; The color calibration module is used to perform color calibration on the original captured image according to the information difference between the reference color information and the original color information to obtain a calibrated image, so as to construct a digital model of the target object based on the calibrated image.

8. An electronic device, comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method according to any one of claims 1-6.

9. A computer storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method according to any one of claims 1 to 6.

10. A computer program product comprising computer instructions, wherein the computer instructions instruct a computing device to execute the method according to any one of claims 1 to 6.

Images