Color calibration method, device, apparatus and computer storage medium

Abstract

Embodiments of the present application provide a color calibration method, device, equipment and computer storage medium. The color calibration method comprises: obtaining a calibration reference image and an original acquisition image; determining a reference target region of a target object in the calibration reference image and an original target region of the target object in the original acquisition image; calculating reference color information and original color information; the reference color information represents color characteristics of the reference target region; the original color information represents color characteristics of the original target region; performing color calibration on the original acquisition image according to information differences between the reference color information and the original color information, to obtain a calibration image, and constructing a digital model of the target object based on the calibration image. Embodiments of the present application can make the color of the digital model more natural and real, and improve the quality of the digital model.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a color calibration method, apparatus, electronic device, computer storage medium, and computer program product. Background Technology

[0002] 3D scanned virtual assets refer to digital objects or models of real-world objects obtained by taking 3D photographs of real-world objects from all angles using image acquisition equipment, and then using these photographs, along with 3D reconstruction tools or algorithms, to reconstruct the real-world object in 3D. These digital objects or models typically contain a mesh representing the spatial structure of the real-world object, and a colored texture layer attached to the mesh.

[0003] At present, the 3D scanned virtual assets obtained using related technologies, that is, the constructed digital models, generally suffer from unnatural or unrealistic color representation. Summary of the Invention

[0004] In view of this, embodiments of this application provide a color calibration scheme to at least partially solve the above-mentioned problems.

[0005] According to a first aspect of the embodiments of this application, a color calibration method is provided, comprising:

[0006] Acquire a calibration reference image and a raw acquisition image; both the calibration reference image and the raw acquisition image contain the target object;

[0007] Determine the reference target region of the target object in the calibration reference image, and the original target region of the target object in the original acquired image;

[0008] Calculate reference color information and original color information; the reference color information represents the color characteristics of the reference target area; the original color information represents the color characteristics of the original target area.

[0009] Based on the information difference between the reference color information and the original color information, the original acquired image is color-calibrated to obtain a calibration image, and a digital model of the target object is constructed based on the calibration image.

[0010] According to a second aspect of the embodiments of this application, a color calibration apparatus is provided, comprising:

[0011] The image acquisition module is used to acquire a calibration reference image and a raw acquired image; both the calibration reference image and the raw acquired image contain the target object.

[0012] The region determination module is used to determine the reference target region of the target object in the calibration reference image, and the original target region of the target object in the original acquired image;

[0013] A color information calculation module is used to calculate reference color information and original color information; the reference color information represents the color characteristics of the reference target area; the original color information represents the color characteristics of the original target area.

[0014] The color calibration module is used to perform color calibration on the original acquired image based on the information difference between the reference color information and the original color information to obtain a calibration image, so as to construct a digital model of the target object based on the calibration image.

[0015] According to a third aspect of the present application, an electronic device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, wherein the executable instruction causes the processor to perform an operation corresponding to the method described in the first aspect.

[0016] According to a fourth aspect of the embodiments of this application, a computer storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0017] According to the color calibration scheme provided in the embodiments of this application, the original acquired image and the calibration reference image whose color information matches the actual scene are acquired for the target object respectively; then, the reference target area where the target object is located in the calibration reference image and the original target area where the target object is located in the original acquired image are detected; then, based on the difference between the color features in the reference target area and the color features in the original target area, the original acquired image is color-calibrated and adjusted, thereby obtaining a calibration image with more natural and realistic colors.

[0018] In this embodiment, the color information of the target object in the calibration reference image is used as a benchmark, and the information difference between the color information of the target object in the calibration reference image and the color information of the target object in the original acquired image is used as the calibration basis. Color calibration is performed on the original acquired image, making the colors of the calibrated image more natural and realistic. Furthermore, model reconstruction of the target object based on the calibrated image with more natural and realistic colors results in a more natural and realistic digital model of the target object, improving the quality of the final digital model.

[0019] Meanwhile, as mentioned above, in this embodiment, color calibration is performed based on the difference in color information between the calibration reference image and the original acquired image. In other words, this embodiment focuses on the difference in color information between the calibration reference image and the original acquired image. Whether the shooting conditions (such as ambient lighting conditions, device hardware parameters, etc.) of the calibration reference image and the original acquired image are the same does not affect the specific implementation of this embodiment. That is, in this embodiment, the shooting conditions of the calibration reference image and the original acquired image can be different. Therefore, this embodiment reduces the difficulty of image acquisition operation and has high practicality.

[0020] Furthermore, in this embodiment, color calibration is performed on the original two-dimensional images acquired from the target object as independent calibration objects. In actual acquisition, different original images typically correspond to different shooting angles. Therefore, compared to related schemes that first construct a three-dimensional digital model and then perform overall color calibration within the three-dimensional digital model, this embodiment's scheme of performing color calibration separately on each original two-dimensional image acquired from different shooting angles can also solve the color difference problem between different areas of the three-dimensional digital model caused by multi-angle shooting for the same target object, further improving the quality of the final digital model. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0022] Figure 1 This is a flowchart illustrating the steps of a color calibration method according to an embodiment of this application;

[0023] Figure 2 A schematic diagram of the shooting scene for the image to be color-corrected;

[0024] Figure 3 This is a schematic diagram illustrating a scenario example according to an embodiment of this application;

[0025] Figure 4 This is a flowchart illustrating a three-dimensional reconstruction process according to an embodiment of this application.

[0026] Figure 5 This is a structural block diagram of a color calibration device according to an embodiment of this application;

[0027] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0029] The specific implementation of the embodiments of this application will be further described below with reference to the accompanying drawings.

[0030] Reference Figure 1 , Figure 1 This is a flowchart illustrating the steps of a color calibration method according to an embodiment of this application. Specifically, the color calibration method provided in this embodiment includes the following steps:

[0031] Step 102: Obtain the calibration reference image and the original acquired image.

[0032] Among them, the calibration reference image and the original acquisition image are images containing the target object; that is, both the calibration reference image and the original acquisition image are images acquired for the target object.

[0033] Specifically, the target object in this application embodiment can be any real object with a fixed shape and color to be used for digital model construction. For example, it can be a chair or sofa in an indoor scene, or a building, stone sculpture, etc. in an outdoor scene.

[0034] When constructing a digital model of a target object, images of the target object from different angles can be acquired using image acquisition devices such as cameras. Then, the acquired images can be processed using relevant 3D reconstruction tools or algorithms to obtain the corresponding digital model of the target object. For ease of distinction later, in this embodiment, the unprocessed original images obtained by acquiring images of the target object from different angles using image acquisition devices such as cameras can be referred to as the original acquired images.

[0035] The calibration reference image in this embodiment can be an image acquired for the target object to be digitally modeled. Unlike the original acquired image, the calibration reference image can be a color-adjusted image that aligns the image colors with the colors of the real object.

[0036] In this embodiment, the method of obtaining the calibration reference image is not limited. For example, after acquiring an image of the target object, the color of the acquired image can be adjusted manually based on experience or personal visual aesthetics to obtain the calibration reference image; alternatively, a reference object with known actual color information can be used as the basis and target for color adjustment to adjust the acquired image and obtain the calibration reference image.

[0037] Furthermore, regarding the method of obtaining a calibration reference image using a reference object, the specific implementation process may include:

[0038] Obtain the color-corrected image containing the target object and the reference object, wherein the actual color information of the reference object is known;

[0039] Perform reference object detection on the image to be color-corrected to determine the reference region of the reference object in the image to be color-corrected;

[0040] The image to be color-corrected is then color-adjusted to obtain a calibration reference image; the color information of the reference area in the calibration reference image is consistent with the actual color information of the reference object.

[0041] Specifically, for a given object, the color information it possesses can refer to its color characteristics, that is, the overall characteristics or rules that the object exhibits in terms of color values. The specific content of the color information is not limited here. For example, color information may include one or more pieces of information with high visual perceptibility, such as brightness information and hue information; furthermore, color information may also include other quantified value information obtained through numerical calculations based on the color values ​​of the object in each color channel, and so on.

[0042] Since the color information of a given object refers to its overall characteristics or rules in terms of color values, the color information of that object can be obtained if its color values ​​are known. Based on the above analysis, to reduce the difficulty of implementation, an entity object with known color values ​​can be selected as the reference object. For example, a color chart with known color values ​​(which can be a single-color chart or a multi-color chart containing multiple color values) can be selected as the reference object.

[0043] In this embodiment, the target object to be digitally modeled and a reference object with known color information can be placed in the same shooting frame, thereby acquiring a color-corrected image that simultaneously contains both the target object and the reference object through an image acquisition device. See also Figure 2 , Figure 2 This is a schematic diagram of the shooting scene for the image to be color-corrected. Figure 2 The target object is a stone tower, while the reference object is a color chart containing multiple color values ​​placed on the stone tower. The stone tower and the color chart can be photographed using image acquisition devices such as cameras to obtain the image to be color-corrected.

[0044] After obtaining the image to be color-corrected, the reference region where the reference object is located in the image to be color-corrected can be determined by object detection. Then, by adjusting the color values ​​of each pixel in the image to be color-corrected, the color information of the reference region in the adjusted image is made consistent with the actual color information of the reference object. Thus, the adjusted image can be used as the calibration reference image in the embodiments of this application.

[0045] The method described above for obtaining calibration reference images requires no manual intervention; instead, it employs an adaptive algorithm to generate the images, thus increasing efficiency. More importantly, by introducing a reference object with known color information, the goal of color adjustment in the image to be color-corrected is specifically quantified: ensuring that the color information of the reference area in the adjusted image matches the actual color information of the reference object. In other words, the basis for color adjustment is precisely quantified, resulting in a more realistic and natural color for the reference object in the final calibration reference image, thereby improving the accuracy of the calibration reference image.

[0046] Step 104: Determine the reference target region of the target object in the calibration reference image, and the original target region of the target object in the original acquired image.

[0047] Specifically, step 104 allows for the acquisition of the target object's region information in the calibration reference image and the target object's region information in the original acquired image. In this application example, any suitable object segmentation or object detection scheme can be used to determine the region information of the target object from each image. For example, a machine learning model capable of target object detection can be used to determine the reference target region in the calibration reference image and the original target region in the original acquired image; alternatively, non-machine learning algorithms, such as edge detection algorithms and morphological processing algorithms, can be used to determine the reference target region and the original target region.

[0048] Step 106: Calculate the reference color information and the original color information.

[0049] Specifically, reference color information can characterize the color features of the reference target region, that is, the overall characteristics or rules of the reference target region in the calibration reference image in terms of color values. Correspondingly, original color information can characterize the color features of the original target region, that is, the overall characteristics or rules of the original target region in the original acquired image in terms of color values.

[0050] The specific content of color information is not limited here. For example, color information may include one or more pieces of information with high visual perceptibility, such as brightness information and hue information; furthermore, color information may also include other specific quantitative value information obtained by performing certain numerical calculations based on the color values ​​of the object in each color channel, and so on.

[0051] Step 108: Based on the information difference between the reference color information and the original color information, perform color calibration on the original acquired image to obtain a calibration image, and construct a digital model of the target object based on the calibration image.

[0052] Specifically, as mentioned above, color information represents the overall characteristics or rules of a region in terms of color values. Therefore, after obtaining the reference color information and the original color information, the original acquired image can be color-calibrated based on the information differences between the two.

[0053] The specific calibration method can be as follows: adjust the color values ​​of pixels in the original acquired image, that is, adjust the overall color values ​​of the original acquired image to obtain an adjusted acquired image; then calculate the adjusted color information of the target object area in the adjusted acquired image, and compare the adjusted color information with the reference color information of the target area; if the information difference between the adjusted color information and the reference color information is small (for example, less than a preset difference threshold), then the adjusted acquired image is used as the calibration image in this application embodiment; otherwise, if the information difference between the adjusted color information and the reference color information is large (for example, greater than or equal to a preset difference threshold), then continue to adjust the color of the adjusted acquired image, and based on the new adjusted acquired image, perform the steps of calculating the adjusted color information of the target object area in the adjusted acquired image and comparing the adjusted color information with the reference color information, until the information difference between the adjusted color information and the reference color information is small, and obtain the calibration image.

[0054] Alternatively, the specific calibration method can be as follows: based on the information difference between the reference color information and the original color information, a specific color value adjustment amount is quantified, and then the overall color value of the original acquired image is adjusted according to the color value adjustment amount, so as to obtain a calibration image with a smaller difference between the color information and the reference color information.

[0055] See Figure 3 , Figure 3 This is a schematic diagram illustrating a scenario example according to an embodiment of this application. The following is in conjunction with... Figure 3 A brief description of the specific execution process of the color calibration method provided in the embodiments of this application is as follows:

[0056] The process involves acquiring a calibration reference image containing the target object, and multiple original acquisition images of the target object taken from different shooting angles. The location information of the reference target region in the calibration reference image and the location information of the original target region in the original acquisition images are determined. The calibration reference image, the multiple original acquisition images, and the location information of the reference and original target regions are input into a color calibration tool that executes the color calibration method provided in this application embodiment. The color calibration tool calculates the reference color information corresponding to the reference target region and the original color information corresponding to the original target region in each original acquisition image. Based on the information differences between the reference color information and each original color information, color calibration is performed on each original acquisition image, thereby outputting a calibration image corresponding to each original acquisition image.

[0057] According to the color calibration scheme provided in the embodiments of this application, the original acquired image and the calibration reference image whose color information matches the actual scene are acquired for the target object respectively; then, the reference target area where the target object is located in the calibration reference image and the original target area where the target object is located in the original acquired image are detected; then, based on the difference between the color features in the reference target area and the color features in the original target area, the original acquired image is color-calibrated and adjusted, thereby obtaining a calibration image with more natural and realistic colors.

[0058] In this embodiment, the color information of the target object in the calibration reference image is used as a benchmark, and the information difference between the color information of the target object in the calibration reference image and the color information of the target object in the original acquired image is used as the calibration basis. Color calibration is performed on the original acquired image, making the colors of the calibrated image more natural and realistic. Furthermore, model reconstruction of the target object based on the calibrated image with more natural and realistic colors results in a more natural and realistic digital model of the target object, improving the quality of the final digital model.

[0059] Meanwhile, as mentioned above, the color calibration in this application embodiment is based on the difference in color information between the calibration reference image and the original acquired image. In other words, this application example focuses on the difference in color information between the calibration reference image and the original acquired image. Whether the shooting conditions (such as ambient lighting conditions, device hardware parameters, etc.) of the calibration reference image and the original acquired image are the same does not affect the specific implementation of this application embodiment. That is, in this application embodiment, the shooting conditions of the calibration reference image and the shooting conditions of the original acquired image can be different. Therefore, this application embodiment reduces the difficulty of image acquisition operation and has high practicality.

[0060] Furthermore, in this embodiment, color calibration is performed on the original two-dimensional images acquired from the target object as independent calibration objects. In actual acquisition, different original images typically correspond to different shooting angles. Therefore, compared to related schemes that first construct a three-dimensional digital model and then perform overall color calibration within the three-dimensional digital model, this embodiment's scheme of performing color calibration separately on each original two-dimensional image acquired from different shooting angles can also solve the color difference problem between different areas of the three-dimensional digital model caused by multi-angle shooting for the same target object, further improving the quality of the final digital model.

[0061] Furthermore, when multiple target objects made of the same color and material in the same scene are digitally modeled separately, color differences often exist between the resulting digital models of each target object. In this case, the color calibration method provided in this application can be applied to each target object separately, and the calibration reference images corresponding to different target objects can be uniformly adjusted to maintain high consistency in color information. This ensures that color calibration is performed based on the same benchmark and standard during the construction of the digital models of each target object, thereby improving the color difference problem between the digital models of multiple target objects.

[0062] The color calibration method of this embodiment can be executed by any suitable electronic device with data processing capabilities, including but not limited to: servers, PCs, etc.

[0063] Optionally, in some embodiments, the process of calculating the reference color information and the original color information described above may specifically include:

[0064] The color values ​​of each pixel in the reference target area under the preset color channel are fused and calculated to obtain the target color value of the reference target area under the preset color channel.

[0065] Based on the target color value of the reference target area in the preset color channel, the reference color information value is obtained;

[0066] The color values ​​of each pixel in the original target area under the preset color channel are fused and calculated to obtain the original color value of the original target area under the preset color channel.

[0067] The original color information value is obtained based on the original color value of the original target area under the preset color channel.

[0068] Specifically, since color information represents the overall characteristics or rules of a region in terms of color values, when calculating reference color information values, the color values ​​of each pixel in the reference target region can be comprehensively considered. By fusing the color values ​​of each pixel in the reference target region, a more accurate reference color information value corresponding to the reference target region can be obtained, thus improving the accuracy of the reference color information value. Correspondingly, when calculating original color information values, the color values ​​of each pixel in the original target region can be comprehensively considered. By fusing the color values ​​of each pixel in the original target region, a more accurate original color information value corresponding to the original reference region can be obtained, thus improving the accuracy of the original color information value.

[0069] In this embodiment, the specific fusion strategy adopted for fusion calculation of the color values ​​of each pixel in the preset color channel is not limited. For example, the mean of the color values ​​of each pixel in the preset color channel can be calculated as the target color value of the region in the preset color channel; the median of the color values ​​of each pixel in the preset color channel can be calculated as the target color value of the region in the preset color channel; the mode of the color values ​​of each pixel in the preset color channel can be calculated as the target color value of the region in the preset color channel, and so on.

[0070] Furthermore, when fusing the color values ​​of each pixel to obtain color information, the calculation is performed at the color channel level. That is, the target color value of the reference target area in the preset color channel is used as the reference color information value, and the original color value of the original reference area in the preset color channel is used as the original color information value. Therefore, the information difference between the subsequently calculated reference color information and the original color information represents a coarse-grained color calibration mapping relationship at the color channel level between the calibration reference image and the original acquired image. Based on the above coarse-grained mapping relationship, color calibration can effectively reduce the workload of color calibration.

[0071] For example, the number of preset color channels can be set to 1. For instance, the preset color channel can be the R color channel. The calculated reference color information value is 10, and the calculated original color information value is 8. This means that the average value of each pixel in the reference target area in the R color channel is 10, and the average value of each pixel in the original target area in the R color channel is 8. The difference in color value between the original target area and the reference target area in the R color channel is 2. In this case, the color calibration of the original acquired image can be achieved by uniformly increasing the color value of each pixel in the original target area in the R color channel by 2.

[0072] For example, the number of preset color channels can be set to multiple. For instance, the preset color channels may include at least two of the following: R color channel, G color channel, and B color channel. Then, for each of the above multiple color channels, the target color value of the reference target area in that color channel and the original color value of the original target area in that color channel can be calculated respectively. The difference color value between the target area and the reference target area in each color channel can be calculated respectively, and a color value correction operation can be performed for each color channel. That is, the difference color value in the corresponding color channel is added to the color value of the pixel in the original target area in each color channel, thereby realizing the color calibration of the original acquired image.

[0073] Compared with the method of calculating the corresponding compensation data for the color value of each pixel in the original target area, the embodiments of this application can effectively reduce the workload of color calibration.

[0074] Optionally, in some embodiments, the number of preset color channels is multiple;

[0075] Based on the target color value of the reference target area in the preset color channel, reference color information values ​​are obtained, including:

[0076] Using the target color value of the reference target area in each preset color channel as the independent variable, the reference color information is calculated using a preset function formula.

[0077] Based on the original color values ​​of the original target area under the preset color channels, the original color information values ​​are obtained, including:

[0078] Using the original color values ​​of the original target area in each preset color channel as independent variables, the original color information is calculated using a preset function formula.

[0079] Specifically, the above embodiments consider multiple color channels simultaneously when obtaining color information, and perform fusion calculations based on the color values ​​of multiple color channels, thereby using the calculation results as color information. Compared with the method of using the color value of a single color channel as the color information value, the above method of fusing multiple color channels to obtain color information can improve the accuracy of the final obtained color information.

[0080] In this embodiment, the specific form of the preset function calculation formula is not limited, and can be customized according to actual conditions or experience. For example, the average value of color values ​​under multiple preset color channels can be calculated to obtain color information; alternatively, the color values ​​under multiple preset color channels can be calculated by performing differences or summations on each pair, and then the calculation results can be summed or otherwise fused to obtain color information, and so on.

[0081] Optionally, in some embodiments, the process of color calibrating the original acquired image based on the information difference between the reference color information and the original color information to obtain a calibrated image may specifically include:

[0082] Determine the numerical relationship between the reference color information and the original color information;

[0083] Based on numerical relationships, the color values ​​of each pixel in the original target area of ​​the original acquired image are adjusted in the preset color channel to obtain a calibrated image.

[0084] Specifically, after obtaining the reference color information and the original color information, the numerical relationship between the two can be obtained. Then, based on the above numerical relationship, the color values ​​of each pixel in the original target area under the preset color channel are adjusted to obtain a calibration image whose color information is consistent with the reference color information of the calibration reference image.

[0085] For example, the aforementioned numerical relationship can be the difference between reference color information and original color information. After obtaining the difference, the color values ​​of each pixel in the original target area under the preset color channel can be offset compensation based on the difference to obtain a calibrated image. For example, if the difference between the reference color information and the original color information is 2, then the color values ​​of each pixel in the original target area under the preset color channel can be uniformly increased by 2 to obtain a compensated image.

[0086] The above process calculates the numerical relationship between the reference color information and the original color information, and then uses this numerical relationship to quantitatively adjust the color of the pixels in the original acquired image, thereby more accurately aligning the color information of the calibration image with the reference color information of the calibration reference image.

[0087] Optionally, in some embodiments, the process of determining the reference target region of the target object in the calibration reference image, and the original target region of the target object in the original acquired image, may specifically include:

[0088] A 3D reconstruction model of the target object is obtained by using a calibration reference image and the original acquired image.

[0089] Based on the correspondence between each location point in the 3D reconstruction model and the pixel points in the calibration reference image, the reference target area of ​​the target object in the calibration reference image is determined;

[0090] Based on the correspondence between each location point in the 3D reconstruction model and the pixel points in the original acquired image, the original target area of ​​the target object in the original acquired image is determined.

[0091] Specifically, relevant 3D reconstruction software can be used to execute 3D reconstruction algorithms based on calibration reference images and original acquired images, thereby obtaining a 3D reconstructed model of the target object. For example, this can be a 3D point cloud model. The specific reconstruction process can include the following steps: pose alignment, feature point matching, point cloud generation, and establishment of mapping relationships.

[0092] The pose alignment step is mainly used to obtain the camera pose information corresponding to each image. Specifically, feature point matching can be achieved through adaptive feature point detection algorithms, global optimization algorithms, etc., and the relative positions between each feature point can be determined to obtain the camera pose information. In terms of the feature point matching process, entity feature points can be marked in the target object in advance. After obtaining the calibration reference image and the original acquired image containing the target object, virtual feature points corresponding to entity feature points in each image can be detected, and virtual feature point pairs corresponding to the same entity feature point can be found in each image. The point cloud generation step refers to the process of generating a point cloud in three-dimensional space based on the above camera pose and feature point matching results. The above point cloud will eventually constitute a three-dimensional reconstruction model. In the above point cloud generation process, the correspondence between each point cloud in the three-dimensional reconstruction model and the pixel points in each image can also be obtained.

[0093] Based on the above description of the 3D reconstruction process, the 3D reconstruction process itself can obtain the correspondence between each point cloud in the 3D reconstruction model and the pixels in each image. In the above embodiments of this application, by utilizing the above correspondence that can be obtained from the 3D reconstruction process itself, without performing any additional operations, the reference target region of the target object in the calibration reference image and the original target region of the target object in the original acquired image can be quickly and efficiently determined. In other words, the above embodiments of this application make full use of the intermediate information generated during the 3D reconstruction process to determine the target region, which can effectively save computing resources and improve the efficiency of target region determination.

[0094] See Figure 4 , Figure 4 This is a flowchart of a three-dimensional reconstruction process according to an embodiment of this application.

[0095] Firstly, let's combine... Figure 4 The process of building a digital model is explained as follows: When building a digital model of a target object, the object is first scanned from different angles to obtain multiple images. These images are typically raw data captured by the image sensor in the image acquisition device, such as RAW format data. After obtaining the RAW image data, the images can be converted to a more universal and easily stored image format, such as JPG (Joint Photographic Experts Group) format. Then, the converted images are reconstructed in 3D to output the digital model of the target object. The 3D reconstruction process involves: aligning the poses of each converted image to determine the corresponding camera pose; using relevant 3D reconstruction software based on the converted images and their camera pose information to perform 3D reconstruction, resulting in a 3D reconstructed model and its corresponding texture and shading information. At this point, the digital model is complete, and a digital model containing both the 3D reconstructed model and texture and shading information can be output.

[0096] Secondly, in combination Figure 4 A brief description of the three-dimensional reconstruction process based on the color calibration method provided in this application embodiment is given. Compared with the digital model construction process of the first aspect described above, the three-dimensional reconstruction process based on the color calibration method provided in this application embodiment adds... Figure 4 The two steps indicated by the dashed box are the image color adjustment step and the color calibration step. Specifically, the image color adjustment step can refer to selecting one of multiple scanned images for color adjustment to obtain a calibration reference image; then, 3D reconstruction can be performed based on the calibration reference image and the original captured image to obtain a 3D reconstructed model of the target object; the color calibration step can refer to the color calibration scheme provided in any of the above embodiments of this application. Through the color calibration step, a calibration image can be obtained, and then the texture coloring of the constructed 3D model can be performed based on the calibration image, thereby outputting a digital model with more realistic and natural colors.

[0097] Furthermore, after scanning the target object from different angles to obtain multiple raw data format images captured by the image sensor in the image acquisition device, in addition to format conversion of the acquired images, the color gamut of each image can be adjusted to a unified color gamut space. For example, this could be the LAB color gamut space, sRGB (Standard Red Green Blue) color gamut space, etc. Then, subsequent color calibration and 3D reconstruction operations are performed on the images after format and color gamut space adjustment. Through this unified color gamut space operation, differences in image color information caused by different color gamut spaces can be effectively avoided, further improving the accuracy of the color calibration scheme provided in this application embodiment.

[0098] See Figure 5 , Figure 5 This is a structural block diagram of a color calibration device according to an embodiment of the present application. The device includes:

[0099] The image acquisition module 502 is used to acquire a calibration reference image and a raw acquisition image; both the calibration reference image and the raw acquisition image contain the target object.

[0100] The region determination module 504 is used to determine the reference target region of the target object in the calibration reference image, and the original target region of the target object in the original acquired image.

[0101] The color information calculation module 506 is used to calculate reference color information and original color information; the reference color information represents the color characteristics of the reference target area; the original color information represents the color characteristics of the original target area.

[0102] The color calibration module 508 is used to perform color calibration on the original acquired image based on the information difference between the reference color information and the original color information to obtain a calibration image, and to build a digital model of the target object based on the calibration image.

[0103] Optionally, in some embodiments, the color information calculation module 506 is specifically used for:

[0104] The color values ​​of each pixel in the reference target area under the preset color channel are fused and calculated to obtain the target color value of the reference target area under the preset color channel.

[0105] Based on the target color value of the reference target area in the preset color channel, the reference color information value is obtained;

[0106] The color values ​​of each pixel in the original target area under the preset color channel are fused and calculated to obtain the original color value of the original target area under the preset color channel.

[0107] The original color information value is obtained based on the original color value of the original target area under the preset color channel.

[0108] Optionally, in some embodiments, the number of preset color channels is multiple;

[0109] The color information calculation module 506, when performing the step of obtaining reference color information values ​​based on the target color values ​​of the reference target area in a preset color channel, is specifically used for:

[0110] Using the target color value of the reference target area in each preset color channel as the independent variable, the reference color information is calculated using a preset function formula.

[0111] The color information calculation module 506, when performing the step of obtaining the original color information value based on the original color value of the original target area under the preset color channel, is specifically used for:

[0112] Using the original color values ​​of the original target area in each preset color channel as independent variables, the original color information is calculated using a preset function formula.

[0113] Optionally, in some embodiments, the color calibration module 508, when performing the step of color calibration of the original acquired image based on the information difference between the reference color information and the original color information to obtain a calibrated image, is specifically used for:

[0114] Determine the numerical relationship between the reference color information and the original color information;

[0115] Based on numerical relationships, the color values ​​of each pixel in the original target area of ​​the original acquired image are adjusted in the preset color channel to obtain a calibrated image.

[0116] Optionally, in some embodiments, the region determination module 504 is specifically used for:

[0117] A 3D reconstruction model of the target object is obtained by using a calibration reference image and the original acquired image.

[0118] Based on the correspondence between each location point in the 3D reconstruction model and the pixel points in the calibration reference image, the reference target area of ​​the target object in the calibration reference image is determined;

[0119] Based on the correspondence between each location point in the 3D reconstruction model and the pixel points in the original acquired image, the original target area of ​​the target object in the original acquired image is determined.

[0120] Optionally, in some embodiments, the image acquisition module 502, when performing the step of acquiring a calibration reference image containing the target object, is specifically used for:

[0121] Obtain the color-corrected image containing the target object and the reference object, where the actual color information of the reference object is known;

[0122] Perform reference object detection on the image to be color-corrected to determine the reference region of the reference object in the image to be color-corrected;

[0123] The image to be color-corrected is then color-adjusted to obtain a calibration reference image; the color information of the reference area in the calibration reference image is consistent with the actual color information of the reference object.

[0124] The color calibration device of this embodiment is used to implement the corresponding color calibration method in the aforementioned color calibration method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. Furthermore, the functional implementation of each module in the color calibration device of this embodiment can be referred to the description of the corresponding part in the aforementioned method embodiments, which will also not be repeated here.

[0125] Reference Figure 6 This document illustrates a schematic diagram of an electronic device according to an embodiment of this application. The specific embodiments of this application do not limit the specific implementation of the electronic device.

[0126] like Figure 6 As shown, the control terminal may include: a processor 602, a communications interface 604, a memory 606, and a communications bus 608.

[0127] in:

[0128] The processor 602, communication interface 604, and memory 606 communicate with each other via communication bus 608.

[0129] Communication interface 604 is used for communication with other electronic devices or servers.

[0130] The processor 602 is used to execute program 610, specifically to perform the relevant steps in the above method embodiments.

[0131] Specifically, program 610 may include program code that includes computer operation instructions.

[0132] The processor 602 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The smart device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0133] Memory 606 is used to store program 610. Memory 606 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0134] Program 610 may include multiple computer instructions, and specifically, program 610 may use multiple computer instructions to cause processor 602 to execute the operations corresponding to the methods described in the foregoing multiple method embodiments.

[0135] The specific implementation of each step in program 610 can be found in the corresponding descriptions of the steps and units in the above method embodiments, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0136] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in any of the foregoing method embodiments. The computer storage medium includes, but is not limited to, compact disc read-only memory (CD-ROM), random access memory (RAM), floppy disk, hard disk, or magneto-optical disk.

[0137] This application also provides a computer program product, including computer instructions that instruct a computing device to perform an operation corresponding to any of the methods in the above-described multiple method embodiments.

[0138] Furthermore, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used for training the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0139] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0140] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA)). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., Random Access Memory (RAM), Read-Only Memory (ROM), Flash Memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0141] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0142] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A color calibration method, comprising: Acquire calibration reference image and raw acquired image; Both the calibration reference image and the original acquired image contain the target object. The calibration reference image is an image obtained by color adjustment processing of the color-adjusted image based on the actual color information of the reference object, which realizes the alignment of the image color with the color of the real object. The color-adjusted image is captured simultaneously for the target object and the reference object. The original acquired image is captured for the target object. Different original acquired images correspond to different shooting angles. The method involves determining a reference target region for the target object in the calibration reference image and an original target region for the target object in the original acquired image. This is achieved by performing 3D reconstruction using the calibration reference image and the original acquired image, and determining the reference target region and the original target region based on the correspondence between pixels in the calibration reference image and pixels in the original acquired image and their corresponding locations in the reconstructed 3D model. Reference color information and original color information are then calculated. The reference color information characterizes the color features of the reference target region, and the original color information characterizes the color features of the original target region. Based on the information difference between the reference color information and the original color information, the original acquired image is color-calibrated to obtain a calibration image. The 3D reconstruction model is then texture-colored based on the calibration image to obtain a digital model of the target object.

2. The method according to claim 1, wherein, The calculation of reference color information and original color information includes: The color values ​​of each pixel in the reference target area under the preset color channel are fused and calculated to obtain the target color value of the reference target area under the preset color channel. Based on the target color value of the reference target area in the preset color channel, a reference color information value is obtained; The color values ​​of each pixel in the original target area under the preset color channel are fused and calculated to obtain the original color value of the original target area under the preset color channel. Based on the original color values ​​of the original target area under the preset color channel, the original color information value is obtained.

3. The method according to claim 2, wherein, The number of preset color channels is multiple; The step of obtaining reference color information values ​​based on the target color values ​​of the reference target region in the preset color channel includes: Using the target color value of the reference target area in each preset color channel as the independent variable, the reference color information is calculated using a preset function formula. The process of obtaining the original color information value based on the original color value of the original target region under the preset color channel includes: Using the original color values ​​of the original target area in each preset color channel as independent variables, the original color information is calculated using the preset function formula.

4. The method according to claim 2 or 3, wherein, The step of color calibration of the original acquired image based on the information difference between the reference color information and the original color information to obtain a calibrated image includes: Determine the numerical relationship between the reference color information and the original color information; Based on the numerical relationship, the color values ​​of each pixel in the original target area of ​​the original acquired image under the preset color channel are adjusted to obtain a calibration image.

5. The method according to any one of claims 1-3, wherein, The step of obtaining the calibration reference image containing the target object includes: Obtain a color-corrected image containing a target object and a reference object, wherein the actual color information of the reference object is known; Perform reference object detection on the image to be color-corrected to determine the reference region of the reference object in the image to be color-corrected. The color of the image to be color-corrected is adjusted to obtain a calibration reference image; the color information of the reference region in the calibration reference image is consistent with the actual color information of the reference object.

6. A color calibration device, comprising: The image acquisition module is used to acquire the packet calibration reference image and the original acquired image; Both the calibration reference image and the original acquired image contain the target object. The calibration reference image is an image obtained by color adjustment processing of the color-adjusted image based on the actual color information of the reference object, which realizes the alignment of the image color with the color of the real object. The color-adjusted image is captured simultaneously for the target object and the reference object. The original acquired image is captured for the target object. Different original acquired images correspond to different shooting angles. The region determination module is used to determine the reference target region of the target object in the calibration reference image, and the original target region of the target object in the original acquired image; wherein, three-dimensional reconstruction is performed using the calibration reference image and the original acquired image, and the reference target region and the original target region are determined according to the correspondence between the pixels in the calibration reference image and the pixels in the original acquired image and the corresponding position points in the reconstructed three-dimensional reconstruction model; A color information calculation module is used to calculate reference color information and original color information; the reference color information represents the color characteristics of the reference target area; the original color information represents the color characteristics of the original target area. The color calibration module is used to perform color calibration on the original acquired image based on the information difference between the reference color information and the original color information to obtain a calibration image, and then perform texture coloring on the three-dimensional reconstruction model based on the calibration image to obtain a digital model of the target object.

7. An electronic device, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation corresponding to the method as described in any one of claims 1-5.

8. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1-5.

9. A computer program product comprising computer instructions that instruct a computing device to perform the method as described in any one of claims 1-5.

Images