Image color consistency processing method and device for large-scale scene three-dimensional reconstruction
By constructing the adjacency relationship and color optimization model of the image data set and adjusting the image parameters, the problem of inconsistent image color in three-dimensional reconstruction is solved, and the accuracy and visual effect of the model are improved.
Patent Information
- Application Number
- CN202411105331.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-12-10
AI Technical Summary
The prior art is difficult to ensure the color consistency of tilted photographic images in three-dimensional reconstruction, resulting in color differences between images affecting the accuracy of point cloud reconstruction and model visual effects.
By constructing the adjacency relationship of the image data set, selecting multiple adjacency groups, calculating the mean and standard deviation of pixel points, building a color optimization model, and adjusting the image parameters using the Levenberg-Marquardt method to ensure the consistency of the brightness and contrast of the image.
It effectively solves the problem of inconsistent image color, improves the accuracy and visual effect of the three-dimensional model, and improves the reconstruction efficiency of large-scale scenes.
Smart Images
Figure CN119107398A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a method and device for processing image color consistency for three-dimensional reconstruction of large-scale scenes. Background Art
[0002] 3D reconstruction is the creation of a 3D model from 2D images or other data sources. Its goal is to create a virtual representation of an object or scene. 3D reconstruction is an important technology in fields such as computer vision, robotics, and virtual reality, and can be used for a variety of purposes, such as visualization, animation, simulation, and medical imaging.
[0003] The 3D reconstruction technology based on oblique photogrammetry combines a variety of complex processes, including oblique photography image acquisition, camera pose estimation, 3D point cloud, mesh and texture reconstruction. In the process of oblique photography image acquisition, it is difficult to ensure the consistency of image color due to factors such as shooting lighting conditions and camera parameters, that is, the pixel intensity observed at the same scene point is different. This color difference between images will first affect the results of dense matching, and then affect the accuracy of point cloud reconstruction. Texture reconstruction is the process of projecting and mapping the best viewing image onto each face of the mesh to generate a real scene model. When selecting photos from different perspectives for texture reconstruction of the same scene, it will also cause inconsistent colors of adjacent faces, thus affecting the visual effect of the model.
[0004] Existing technologies consider view continuity in texture reconstruction or use local correction, gradient domain fusion and other methods to eliminate color differences. However, these local methods are not helpful for the global consistency of texture in a broader sense. And only a small number of pixel value constraints on the edges of adjacent patches are used, which can easily produce erroneous correction results or smearing artifacts when applied to complex scenes with very large changes. In addition, the efficiency of reconstructing large-scale scenes at one time is very low. Generally, the huge scene is divided into smaller tiles, and the image of each tile is reconstructed into a grid and texture respectively. This process will cause boundary continuity problems between tiles. Summary of the invention
[0005] The embodiments of the present application solve the problem of color inconsistency in oblique photography images in the prior art by providing a method and device for image color consistency processing for large-scale scene three-dimensional reconstruction.
[0006] In the first aspect, an embodiment of the present application provides a method for processing image color consistency for three-dimensional reconstruction of large-scale scenes, including: constructing an adjacency relationship of an image data set to obtain adjacent images and same-name points of each image; selecting multiple adjacent images to form an adjacent image group according to the number of same-name points in each image; respectively determining the adjacent mean and adjacent standard deviation of the pixel points of the adjacent images in the adjacent image group; constructing a color optimization model, and using the adjacent mean and the adjacent standard deviation to fill a parameter matrix; solving the parameter matrix to obtain the adjustment amount of the image to be adjusted; and adjusting the image to be adjusted to meet the target mean and target standard deviation according to the color optimization model and the adjustment amount.
[0007] In combination with the first aspect, in a possible implementation method, the adjacency relationship of the image data set is constructed to obtain adjacent images and same-name points of each image, including: extracting key feature points of the images in the image data set to generate feature descriptors, and matching feature points according to the feature descriptors to obtain same-name points; determining the relative position and posture between images according to the feature point matching relationship of the same-name points, and thereby establishing the adjacency relationship between the images to obtain adjacent images.
[0008] In combination with the first aspect, in a possible implementation method, respectively determining the adjacent mean and adjacent standard deviation of the pixel points in the adjacent images in the adjacent image group includes: grouping the same-name points in the adjacent images according to a preset size; and respectively calculating the adjacent mean and adjacent standard deviation of the pixel points in each group.
[0009] In combination with the first aspect, in a possible implementation, after the color optimization model is constructed, the method further includes: initializing parameters and variables of the color optimization model; and optimizing the formula as follows:
[0010] In the formula, f i and f j The statistical parameters of the same-name points in two related images are represented by g. i and g j represents the parameter adjustment of the corresponding image to be solved, r represents the residual, that is, the deviation between the model's prediction result and the actual measurement result, and w ij represents the weight of the residual r, which is determined by the number of points with the same name in the two associated images, h represents the Huber norm, and λ represents the smoothing parameter used to smooth g i and g j .
[0011] In combination with the first aspect, in a possible implementation manner, the color optimization model is as follows:
[0012] In the formula, Q calrepresents the corrected image, Δσ represents the adjustment amount of the adjacent standard deviation, σ represents the adjacent standard deviation, Q represents the image to be adjusted, μ represents the adjacent mean, and Δμ represents the adjustment amount of the adjacent mean.
[0013] In combination with the first aspect, in a possible implementation method, after constructing the adjacency relationship of the image data set to obtain the adjacent images and same-name points of each image, it also includes: estimating the camera initial parameters and the image position and posture between the images based on the same-name points; optimizing the camera initial parameters, the image position and posture, and the sparse point cloud.
[0014] In combination with the first aspect, in a possible implementation manner, performing feature point matching according to the feature descriptor to obtain homonymous points further includes: eliminating mismatched homonymous points through a consistency algorithm.
[0015] In the second aspect, an embodiment of the present application provides an image color consistency processing device for three-dimensional reconstruction of large-scale scenes, including: a construction module, used to construct an adjacency relationship of an image data set to obtain adjacent images and same-name points of each image; a selection module, used to select multiple adjacent images according to the number of same-name points in each image to form an adjacent image group; a determination module, used to respectively determine the adjacent mean and adjacent standard deviation of the pixel points of the adjacent images in the adjacent image group; a filling module, used to construct a color optimization model and use the adjacent mean and the adjacent standard deviation to fill a parameter matrix; a solution module, used to solve the parameter matrix to obtain the adjustment amount of the image to be adjusted; an adjustment module, used to adjust the image to be adjusted to meet the target mean and target standard deviation according to the color optimization model and the adjustment amount.
[0016] In a third aspect, an embodiment of the present application provides a device, comprising: a processor; a memory for storing processor-executable instructions; when the processor executes the executable instructions, it implements the method described in the first aspect or any possible implementation method of the first aspect.
[0017] In a fourth aspect, an embodiment of the present application provides a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium includes a medium for storing a computer program or instructions, which, when executed, enables the method described in the first aspect or any possible implementation method of the first aspect to be implemented.
[0018] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:
[0019] The embodiment of the present application avoids the complexity of large-scale image calculation and storage by selecting multiple adjacent images to form an adjacent image group, improves the consistency of image brightness and contrast through a color optimization model, and effectively solves the problem of color inconsistency in oblique photography images in the prior art, thereby achieving consistency correction of image brightness and color, which can improve the accuracy of the three-dimensional model and enhance the visual effect of the three-dimensional model. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0021] Figure 1 A flowchart of a method for processing image color consistency for large-scale scene 3D reconstruction provided in an embodiment of the present application;
[0022] Figure 2 A schematic diagram of the structure of an image color consistency processing device for large-scale scene 3D reconstruction provided in an embodiment of the present application;
[0023] Figure 3 The same-name points of adjacent images provided in the embodiment of the present application;
[0024] Figure 4 The image to be adjusted provided in the embodiment of the present application;
[0025] Figure 5 The adjusted image provided in the embodiment of the present application. DETAILED DESCRIPTION
[0026] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0027] The following describes some of the techniques involved in the embodiments of the present application to facilitate understanding, and they should be considered as merely exemplary. Therefore, it should be appreciated by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, some descriptions of well-known functions and structures are omitted in the following description.
[0028] Figure 11 is a flowchart of a method for processing image color consistency for large-scale scene 3D reconstruction provided by an embodiment of the present application, comprising steps 101 to 106. Figure 1 This is only an execution order shown in the embodiment of the present application, and does not represent the only execution order of the image color consistency processing method for large-scale scene 3D reconstruction. In the case that the final result can be achieved, Figure 1 The steps shown may be performed in parallel or reversed.
[0029] Step 101: Construct the adjacency relationship of the image data set to obtain the adjacent images and the same-name points of each image. In the embodiment of the present application, the key feature points of the images in the image data set are extracted to generate feature descriptors, and the feature points are matched according to the feature descriptors to obtain the same-name points. Specifically, the SIFT (English: Scale-Invariant Feature Transform, Chinese full name: Scale-Invariant Feature Transform) algorithm is used to extract the key feature points of all images in the image data set to generate feature descriptors. The ANN (English: Approximate Nearest Neighbor, Chinese full name: Approximate Nearest Neighbor) algorithm is used to complete the matching between feature points according to the feature descriptors to obtain the same-name points. Figure 3 Shown is a schematic diagram of the same-name points of adjacent images provided by this application.
[0030] In addition, mismatched points of the same name may be excluded by a consistency algorithm. Exemplarily, a RANSAC algorithm may be used here.
[0031] The relative position and posture between images are determined based on the feature point matching relationship of the same-name points, and the adjacency relationship between images is established to obtain adjacent images. Specifically, the relative position and posture between images are determined based on the feature point matching relationship between the same-name points. Not only that, the initial camera parameters and the image position and posture between images can also be estimated based on the same-name points. The initial camera parameters, image position and posture, and sparse point cloud are optimized. Here, the sparse point cloud generated by the SfM pipeline, the same-name points in the SfM pipeline results are common 3D points in multiple images. Once the same-name points are matched, the adjacency relationship between images can be established. These adjacency relationships can be represented as an undirected graph. Among them, each node represents an image, and the edges between nodes represent the adjacency relationship between images.
[0032] Those skilled in the art should understand that adjacent images and points with the same name exist in pairs.
[0033] Step 102: Select multiple adjacent images according to the number of points of the same name in each image to form an adjacent image group. In the embodiment of the present application, the first multiple groups of images with the most points of the same name are selected from the adjacent images to form an adjacent image group. The complexity of large-scale image calculation and storage can be avoided, and the adjacent images are simplified through this step. The number of adjacent images selected here can be determined according to actual needs.
[0034] In addition, the pixel values of the same-name points may be affected by factors such as image compression artifacts, pixel alignment errors, and angle changes. The image can be preprocessed by low-pass filtering to suppress the influence of noise or angle changes to a certain extent and improve the accuracy of the estimation.
[0035] Step 103: respectively determine the adjacent mean and adjacent standard deviation of the pixel points of the adjacent images in the adjacent image group. In the embodiment of the present application, the points with the same name in the adjacent images are grouped according to a preset size. The preset size here can be set according to actual experience, and can also be set by those skilled in the art according to experience. By grouping adjacent images, the stability of the model solution can be optimized.
[0036] Calculate the adjacent mean and adjacent standard deviation of the pixels in each group. The adjacent mean and adjacent standard deviation can provide information on the color distribution of the image, which is used for subsequent color adjustment and optimization. The adjacent mean and adjacent standard deviation are easy to calculate and can effectively characterize the color characteristics and distribution of the image. The adjacent mean is used to describe the central trend of the pixel value of the image, and the adjacent standard deviation is used to describe the degree of dispersion of the pixel value of the image.
[0037] Step 104: construct a color optimization model, and fill the parameter matrix with the adjacent mean and the adjacent standard deviation. In the embodiment of the present application, the parameters and variables of the color optimization model are initialized.
[0038] The optimization formula is as follows:
[0039] In the formula, f i and f j The statistical parameters of the same-name points in two related images are represented by g. i and g k represents the parameter adjustment of the corresponding image to be solved, r represents the residual, that is, the deviation between the model's prediction result and the actual measurement result, and w ij represents the weight of the residual r, which is determined by the number of points with the same name in the two associated images, h represents the Huber norm, and λ represents the smoothing parameter used to smooth g i and g j .
[0040] The smoothing parameter is used to ensure that g i and g jThe adjustment amount tends to be minimum and the overall brightness after adjustment tends to be average.
[0041] The Huber norm is designed to make robust estimates and reduce the impact of outliers. The details are as follows:
[0042] Where h(x) represents the Huber norm, δ represents the threshold, and x represents the residual.
[0043] In actual scenarios, due to mismatching caused by noise, measurement error, occlusion and other reasons, outliers are generated in the sparse point cloud results of motion recovery structure that are significantly different from most other data points. These outliers cannot usually be completely removed by automatic methods and may have an adverse effect on model solution through error transmission. Outliers usually have large residuals. The Huber norm introduced in this application introduces a threshold δ to the absolute value of the residual, uses a quadratic function within the threshold, and uses a linear function when it exceeds the threshold, so as to reduce the weight of outliers during the iteration process. The threshold δ is determined according to the actual situation of the data.
[0044] The color optimization model is as follows:
[0045] In the formula, Q cal represents the corrected image, Δσ represents the adjustment amount of the neighboring standard deviation, σ represents the neighboring standard deviation, Q represents the image to be adjusted, μ represents the neighboring mean, and Δμ represents the adjustment amount of the neighboring mean.
[0046] Step 105: Solve the parameter matrix to obtain the adjustment amount of the image to be adjusted. In an embodiment of the present application, the Levenberg-Marquardt method is used to solve the adjustment amount of the image. Since the image data set is multi-view, there are a large number of redundant observations when collecting data. The present application uses the Levenberg-Marquardt method to solve. The Levenberg-Marquardt method is a method for solving nonlinear least squares problems. Its basic idea is to continuously update the value of the parameters through iterative optimization so that the error of the objective function gradually decreases. Each iteration calculates the gradient of the objective function based on the current parameter value, and updates the parameters by solving an incremental equation. It combines the advantages of the steepest descent method and the Gauss-Newton method, is insensitive to the initial parameters of the model, and aims to converge to the optimal solution quickly and stably.
[0047] For large matrix solving problems, the main computational cost is the linear equation solving part in each iteration. Matrix decomposition methods can be used to reduce the amount of computation. Cholesky decomposition is a method to decompose a symmetric positive definite matrix into a lower triangular matrix and its transpose. Sparse Cholesky decomposition is performed on the normal equations. It can save a lot of time and memory costs for large sparse problems.
[0048] Levenberg-Marquardt should converge quickly when the problem conditions are good, and the variable g obtained by solution is i and g k Used for image color correction.
[0049] Step 106: Adjust the image to be adjusted to meet the target mean and target standard deviation according to the color optimization model and the adjustment amount. In the embodiment of the present application, the color optimization model can be used to simultaneously correct the mean and standard deviation of the image to adjust the brightness and contrast of the image, and finally obtain an image after color consistency processing, such as Figure 4 and Figure 5 Shown are the image to be adjusted and the image after adjustment.
[0050] Although the present application provides method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in this embodiment is only one way of executing the order of many steps and does not represent the only execution order. When the actual device or client product is executed, it can be executed in sequence or in parallel according to the method shown in this embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment).
[0051] like Figure 3 As shown, the embodiment of the present application also provides an image color consistency processing device 300 for large-scale scene 3D reconstruction. The device includes: a construction module 301, a selection module 302, a determination module 303, a filling module 304, a solution module 305 and an adjustment module 306, as follows.
[0052] The construction module 301 is used to construct the adjacency relationship of the image data set to obtain the adjacent images and the same-name points of each image. The construction module 301 is specifically used to extract the key feature points of the image in the image data set to generate a feature descriptor, and to match the feature points according to the feature descriptor to obtain the same-name points. Specifically, the SIFT (English: Scale-Invariant Feature Transform, Chinese full name: Scale-Invariant Feature Transform) algorithm is used to extract the key feature points of all images in the image data set to generate feature descriptors. The ANN (English: Approximate Nearest Neighbor, Chinese full name: Approximate Nearest Neighbor) algorithm is used to complete the matching between feature points according to the feature descriptor to obtain the same-name points. Figure 3 Shown is a schematic diagram of the same-name points of adjacent images provided by this application.
[0053] In addition, mismatched points of the same name may be excluded by a consistency algorithm. Exemplarily, a RANSAC algorithm may be used here.
[0054] The relative position and posture between images are determined based on the feature point matching relationship of the same-name points, and the adjacency relationship between images is established to obtain adjacent images. Specifically, the relative position and posture between images are determined based on the feature point matching relationship between the same-name points. Not only that, the initial camera parameters and the image position and posture between images can also be estimated based on the same-name points. The initial camera parameters, image position and posture, and sparse point cloud are optimized. Here, the sparse point cloud generated by the SfM pipeline, the same-name points in the SfM pipeline results are common 3D points in multiple images. Once the same-name points are matched, the adjacency relationship between images can be established. These adjacency relationships can be represented as an undirected graph. Among them, each node represents an image, and the edges between nodes represent the adjacency relationship between images.
[0055] Those skilled in the art should understand that adjacent images and points with the same name exist in pairs.
[0056] The selection module 302 is used to select multiple adjacent images to form an adjacent image group according to the number of points of the same name in each image. The selection module 302 is specifically used to select the first multiple groups of images with the most points of the same name in the adjacent images to form an adjacent image group. The complexity of large-scale image calculation and storage can be avoided, and the adjacent images are simplified through this step. The number of adjacent images selected here can be determined according to actual needs.
[0057] In addition, the pixel values of the same-name points may be affected by factors such as image compression artifacts, pixel alignment errors, and angle changes. The image can be preprocessed by low-pass filtering to suppress the influence of noise or angle changes to a certain extent and improve the accuracy of the estimation.
[0058] The determination module 303 is used to respectively determine the adjacent mean and adjacent standard deviation of the pixel points of the adjacent images in the adjacent image group. The determination module 303 is specifically used to group the points of the same name in the adjacent images according to a preset size. The preset size here can be set according to actual experience, and can also be set by those skilled in the art according to experience. By grouping adjacent images, the stability of the model solution can be optimized.
[0059] Calculate the adjacent mean and adjacent standard deviation of the pixels in each group. The adjacent mean and adjacent standard deviation can provide information on the color distribution of the image, which is used for subsequent color adjustment and optimization. The adjacent mean and adjacent standard deviation are easy to calculate and can effectively characterize the color characteristics and distribution of the image. The adjacent mean is used to describe the central trend of the pixel value of the image, and the adjacent standard deviation is used to describe the degree of dispersion of the pixel value of the image.
[0060] The filling module 304 is used to construct a color optimization model and fill the parameter matrix with adjacent means and adjacent standard deviations. The filling module 304 is specifically used to initialize the parameters and variables of the color optimization model.
[0061] The optimization formula is as follows:
[0062] In the formula, f i and f j The statistical parameters of the same-name points in two related images are represented by g. i and g j represents the parameter adjustment of the corresponding image to be solved, r represents the residual, that is, the deviation between the model's prediction result and the actual measurement result, and w ij represents the weight of the residual r, which is determined by the number of points with the same name in the two associated images, h represents the Huber norm, and λ represents the smoothing parameter used to smooth g i and g j .
[0063] The smoothing parameter is used to ensure that g i and g j The adjustment amount tends to be minimum and the overall brightness after adjustment tends to be average.
[0064] The Huber norm is designed to make robust estimates and reduce the impact of outliers. The details are as follows:
[0065] Where h(x) represents the Huber norm, δ represents the threshold, and x represents the residual.
[0066] In actual scenarios, due to mismatching caused by noise, measurement error, occlusion and other reasons, outliers are generated in the sparse point cloud results of motion recovery structure that are significantly different from most other data points. These outliers cannot usually be completely removed by automatic methods and may have an adverse effect on model solution through error transmission. Outliers usually have large residuals. The Huber norm introduced in this application introduces a threshold δ to the absolute value of the residual, uses a quadratic function within the threshold, and uses a linear function when it exceeds the threshold, so as to reduce the weight of outliers during the iteration process. The threshold δ is determined according to the actual situation of the data.
[0067] The color optimization model is as follows:
[0068] In the formula, Q cal represents the corrected image, Δσ represents the adjustment amount of the neighboring standard deviation, σ represents the neighboring standard deviation, Q represents the image to be adjusted, μ represents the neighboring mean, and Δμ represents the adjustment amount of the neighboring mean.
[0069] The solution module 305 is used to solve the parameter matrix to obtain the adjustment amount of the image to be adjusted. The solution module 305 is specifically used to use the Levenberg-Marquardt method to solve the adjustment amount of the image. Since the image data set is multi-view, there are a large number of redundant observations when collecting data. This application uses the Levenberg-Marquardt method to solve. The Levenberg-Marquardt method is a method for solving nonlinear least squares problems. Its basic idea is to continuously update the value of the parameters through iterative optimization so that the error of the objective function gradually decreases. Each iteration calculates the gradient of the objective function based on the current parameter value, and updates the parameters by solving an incremental equation. It combines the advantages of the steepest descent method and the Gauss-Newton method, is insensitive to the initial parameters of the model, and aims to converge to the optimal solution quickly and stably.
[0070] For large matrix solving problems, the main computational cost is the linear equation solving part in each iteration. Matrix decomposition methods can be used to reduce the amount of computation. Cholesky decomposition is a method to decompose a symmetric positive definite matrix into a lower triangular matrix and its transpose. Sparse Cholesky decomposition is performed on the normal equations. It can save a lot of time and memory costs for large sparse problems.
[0071] Levenberg-Marquardt should converge quickly when the problem conditions are good, and the variable g obtained by solution is i and g j Used for image color correction.
[0072] The adjustment module 306 is used to adjust the image to be adjusted to meet the target mean and target standard deviation according to the color optimization model and the adjustment amount. The adjustment module 306 is specifically used to use the color optimization model to simultaneously correct the mean and standard deviation of the image so as to adjust the brightness and contrast of the image, and finally obtain an image after color consistency processing, such as Figure 4 and Figure 5 Shown are the image to be adjusted and the image after adjustment.
[0073] Some modules in the apparatus described in the present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.
[0074] The devices or modules described in the above application embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above devices are described in various modules according to their functions. When implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware. Of course, the module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.
[0075] The methods, devices or modules described in this application can be implemented in the form of computer-readable program codes. The controller can be implemented in any appropriate manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program codes (such as software or firmware) that can be executed by the (micro)processor, logic gates, switches, application-specific integrated circuits (English: Application Specific Integrated Circuit; Abbreviation: ASIC), programmable logic controllers and embedded microcontrollers. Examples of controllers include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in the form of pure computer-readable program codes, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included in it for implementing various functions can also be regarded as structures within the hardware component. Or even, the means for realizing various functions may be regarded as both a software module for realizing the method and a structure within a hardware component.
[0076] An embodiment of the present application further provides a device, comprising: a processor; a memory for storing processor executable instructions; when the processor executes the executable instructions, the method described in the embodiment of the present application is implemented.
[0077] The embodiments of the present application also provide a non-volatile computer-readable storage medium on which a computer program or instruction is stored. When the computer program or instruction is executed, the method described in the embodiments of the present application is implemented.
[0078] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist independently, or two or more modules may be integrated into one module.
[0079] The above storage media include but are not limited to random access memory (RAM), read-only memory (ROM), cache, hard disk (HDD) or memory card. The memory can be used to store computer program instructions.
[0080] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary hardware. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, or it can be reflected in the implementation process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.
[0081] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. All or part of this application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, mobile communication terminals, multi-processor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.
[0082] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the present application.
Claims
1. A method for processing image color consistency for large-scale scene three-dimensional reconstruction, characterized in that: include: Construct the adjacency relationship of the image data set to obtain the adjacent images and points of the same name of each image; Selecting a plurality of adjacent images to form an adjacent image group according to the number of the same-name points in each image; Respectively determining the adjacent mean and adjacent standard deviation of the pixel points of the adjacent images in the adjacent image group; Constructing a color optimization model, and filling a parameter matrix with the adjacent mean and the adjacent standard deviation; Solving the parameter matrix to obtain the adjustment amount of the image to be adjusted; The image to be adjusted is adjusted to meet a target mean value and a target standard deviation according to the color optimization model and the adjustment amount.
2. The method according to claim 1, characterized in that The construction of the adjacency relationship of the image data set to obtain the adjacent images and the same-name points of each image includes: Extracting key feature points of the image in the image data set to generate feature descriptors, and performing feature point matching according to the feature descriptors to obtain points with the same name; The relative position and posture between the images are determined according to the feature point matching relationship of the same-name points, and the adjacency relationship between the images is established to obtain adjacent images.
3. The method according to claim 1, characterized in that The respectively determining the adjacent means and adjacent standard deviations of the pixels of the adjacent images in the adjacent image group comprises: Grouping the points with the same name in the adjacent images according to a preset size; Calculate the adjacent mean and adjacent standard deviation of the pixels in each group respectively.
4. The method according to claim 1, characterized in that: After the color optimization model is constructed, the method further includes: Initializing parameters and variables of the color optimization model; The optimization formula is as follows: In the formula, f i and f j The statistical parameters of the same-name points in two related images are represented by g. i and g j represents the parameter adjustment of the corresponding image to be solved, r represents the residual, that is, the deviation between the model's prediction result and the actual measurement result, and w ij represents the weight of the residual r, which is determined by the number of points with the same name in the two associated images, h represents the Huber norm, and λ represents the smoothing parameter used to smooth g i and g j .
5. The method according to claim 4, characterized in that The color optimization model is as follows: In the formula, Q cal represents the corrected image, Δσ represents the adjustment amount of the adjacent standard deviation, σ represents the adjacent standard deviation, Q represents the image to be adjusted, μ represents the adjacent mean, and Δμ represents the adjustment amount of the adjacent mean.
6. The method according to claim 1, characterized in that After the adjacent images and the same-name points of each image are obtained by constructing the adjacency relationship of the image data set, the method further includes: Estimate the initial camera parameters and the image position and posture between the images according to the same-name points; The initial parameters of the camera, the image position and posture, and the sparse point cloud are optimized.
7. The method according to claim 1, characterized in that The step of matching feature points according to the feature descriptors to obtain points with the same name also includes: The mismatched homonymous points are eliminated through a consistency algorithm.
8. An image color consistency processing device for large-scale scene three-dimensional reconstruction, characterized in that: include: A construction module is used to construct the adjacency relationship of the image data set to obtain the adjacent images and the same-name points of each image; A selection module, used for selecting a plurality of adjacent images to form an adjacent image group according to the number of the same-name points in each image; A determination module, used to respectively determine the adjacent mean and adjacent standard deviation of the pixel points of the adjacent images in the adjacent image group; A filling module, used for constructing a color optimization model and filling a parameter matrix with the adjacent mean and the adjacent standard deviation; A solution module, used for solving the parameter matrix to obtain the adjustment amount of the image to be adjusted; The adjustment module is used to adjust the image to be adjusted to meet a target mean value and a target standard deviation according to the color optimization model and the adjustment amount.
9. A device for performing an image color consistency processing method for large-scale scene three-dimensional reconstruction, characterized in that: include: processor; a memory for storing processor-executable instructions; When the processor executes the executable instructions, the method according to any one of claims 1 to 7 is implemented.
10. A non-volatile computer-readable storage medium, characterized in that: The device comprises a computer program or an instruction for storing the computer program or the instruction, which, when executed, enables the method according to any one of claims 1 to 7 to be implemented.