Camera response nonlinear correction method, related device and storage medium

Abstract

The invention discloses a camera response nonlinear correction method, a related device and a storage medium, which are used for reducing the calculation complexity and improving the correction quality while ensuring a high-precision fitting effect. The method comprises the steps that a to-be-fitted camera is used for shooting a standard light source, a series of original images under different exposure times are obtained, a data point set of the exposure times and response values is obtained according to the original images, and the data point set is used for generating a camera response curve; calculating a segmentation threshold according to the data point set and determining segmentation points, and dividing the camera response curve into a plurality of segmentation intervals based on the segmentation points; fitting each segmented interval by adopting a least square method to obtain a linear function corresponding to each segmented interval; integrating the linear functions of all the segment intervals, and performing smoothing processing near segment points by using a quadratic function to obtain a complete segment linear fitting function; and correcting an image shot by the camera by using a piecewise linear fitting function.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular to a method, a related device and a storage medium for correcting nonlinearity of camera response. Background Art

[0002] In digital cameras and other image capture devices, the response of the sensor usually exhibits nonlinear characteristics due to its own characteristics and circuit design. This nonlinear response can cause color distortion and reduced contrast in different brightness areas of the image, thus affecting the overall quality and perception of the image.

[0003] The camera response curve often exhibits complex nonlinear characteristics, and some simple fitting methods are difficult to accurately fit it. For example, in a high dynamic range scene, the camera's response to strong light and weak light may have different nonlinear laws. Existing fitting technology may not be able to accurately fit the curves of different light intensity areas at the same time, resulting in distortion of image details in highlights or dark areas. Some high-precision fitting algorithms, such as fitting methods based on complex mathematical models, can improve fitting accuracy, but they have high computational complexity, require a lot of computing resources and time, and cannot meet real-time requirements.

[0004] In the prior art, a common correction algorithm is to fit the camera response curve by establishing a polynomial model. However, when the camera response curve is obviously nonlinear and has a complex variation pattern, the polynomial fitting method requires a higher number of times to achieve a certain fitting accuracy, and overfitting is prone to occur in the process, making it impossible to achieve an ideal correction effect in practical applications. Summary of the invention

[0005] The present application provides a method, a related device and a storage medium for correcting nonlinear camera response, which are used to reduce the computational complexity and improve the correction quality while ensuring a high-precision fitting effect.

[0006] The first aspect of the present application provides a method for correcting nonlinearity of camera response, comprising: Using the camera to be fitted to shoot a standard light source, obtaining a series of original images at different exposure times, and obtaining a data point set of exposure time and response value according to the original images, wherein the data point set is used to generate a camera response curve; Calculating a segmentation threshold and determining a segmentation point according to the data point set, and dividing the camera response curve into a plurality of segmentation intervals based on the segmentation point; Fitting each segmented interval using the least square method to obtain a linear function corresponding to each segmented interval; Integrate the linear functions of all segmented intervals, and use a quadratic function to perform smoothing near the segmented points to obtain a complete piecewise linear fitting function; The piecewise linear fitting function is applied to correct the image taken by the camera.

[0007] Optionally, calculating a segmentation threshold and determining a segmentation point according to the data point set includes: Calculating the slopes between adjacent data points in the data point set, and calculating the difference between adjacent slopes; Calculate the segmentation threshold according to the mean and standard deviation of the differences between the adjacent slopes; When the difference is greater than the segmentation threshold, the data point corresponding to the difference is determined as a segmentation point.

[0008] Optionally, the least square method is used to fit each segmented interval to obtain a linear function corresponding to each segmented interval, including: Substituting the data points in each segmented interval into a linear function, constructing the sum of squares of errors between the linear function and the data points according to the least squares principle, wherein the linear function contains coefficients to be fitted; Taking partial derivatives of the coefficients to be fitted based on minimization of the sum of squared errors to obtain a system of linear equations in two variables; The optimal fitting coefficient is obtained by solving the set of two-variable linear equations, and the linear function corresponding to each segmented interval is determined according to the optimal fitting coefficient.

[0009] Optionally, the linear functions of all segmented intervals are integrated, and a quadratic function is used to perform smoothing near the segmented points to obtain a complete piecewise linear fitting function, including: Integrate the linear functions of all segmented intervals to obtain a piecewise linear function; Determine a transition region near each segmentation point, and define a smooth transition function in the transition region, wherein the smooth transition function is a quadratic function; The smooth transition function is introduced on the basis of the piecewise linear function to generate a complete piecewise linear fitting function.

[0010] Optionally, determining a transition area near each segmentation point includes: The transition area near each segmentation point is determined according to the domain of the linear function corresponding to each segmentation point and a preset ratio value, where the preset ratio value is a positive integer.

[0011] Optionally, the method further includes: The smooth transition function is solved by the conditions that the boundary function values ​​are equal and the derivatives at the segmentation points match.

[0012] Optionally, applying the piecewise linear fitting function to correct the image taken by the camera includes: For each pixel in the image captured by the camera, searching for a corresponding target segmentation interval according to the exposure time; The linear function or the smooth transition function corresponding to the target segmentation interval is used to calculate the corrected grayscale value, and the corrected grayscale value is applied to the image captured by the camera to complete the nonlinear correction.

[0013] A second aspect of the present application provides a system for nonlinear correction of camera response, comprising: An acquisition unit, used to use the camera to be fitted to shoot the standard light source to obtain a series of original images under different exposure times, and obtain a data point set of exposure time and response value according to the original images, wherein the data point set is used to generate a camera response curve; A determination unit, configured to calculate a segmentation threshold and determine a segmentation point according to the data point set, and divide the camera response curve into a plurality of segmentation intervals based on the segmentation point; A fitting unit, used for fitting each segmented interval by using the least square method to obtain a linear function corresponding to each segmented interval; A smoothing unit, used for integrating the linear functions of all segmented intervals and performing smoothing near the segmented points using a quadratic function to obtain a complete piecewise linear fitting function; A correction unit is used to apply the piecewise linear fitting function to correct the image taken by the camera.

[0014] Optionally, the determining unit is specifically configured to: Calculating the slopes between adjacent data points in the data point set, and calculating the difference between adjacent slopes; Calculate the segmentation threshold according to the mean and standard deviation of the differences between the adjacent slopes; When the difference is greater than the segmentation threshold, the data point corresponding to the difference is determined as a segmentation point.

[0015] Optionally, the fitting unit is specifically used for: Substituting the data points in each segmented interval into a linear function, constructing the sum of squares of errors between the linear function and the data points according to the least squares principle, wherein the linear function contains coefficients to be fitted; Taking partial derivatives of the coefficients to be fitted based on minimization of the sum of squared errors to obtain a system of linear equations in two variables; The optimal fitting coefficient is obtained by solving the set of two-variable linear equations, and the linear function corresponding to each segmented interval is determined according to the optimal fitting coefficient.

[0016] Optionally, the smoothing unit is specifically used for: Integrate the linear functions of all segmented intervals to obtain a piecewise linear function; Determine a transition region near each segmentation point, and define a smooth transition function in the transition region, wherein the smooth transition function is a quadratic function; The smooth transition function is introduced on the basis of the piecewise linear function to generate a complete piecewise linear fitting function.

[0017] Optionally, the smoothing unit is further configured to: The transition area near each segmentation point is determined according to the domain of the linear function corresponding to each segmentation point and a preset ratio value, where the preset ratio value is a positive integer.

[0018] Optionally, the smoothing unit is further configured to: The smooth transition function is solved by the conditions that the boundary function values ​​are equal and the derivatives at the segmentation points match.

[0019] Optionally, the correction unit is specifically used for: For each pixel in the image captured by the camera, searching for a corresponding target segmentation interval according to the exposure time; The linear function or the smooth transition function corresponding to the target segmentation interval is used to calculate the corrected grayscale value, and the corrected grayscale value is applied to the image captured by the camera to complete the nonlinear correction.

[0020] A third aspect of the present application provides a device for nonlinear correction of camera response, the device comprising: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the first aspect and any optional method for nonlinear correction of camera response in the first aspect.

[0021] A fourth aspect of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, the method for nonlinear correction of camera response according to the first aspect and any optional method in the first aspect is executed.

[0022] It can be seen from the above technical solutions that this application has the following advantages: When the camera response curve has a complex variation pattern, it is divided into multiple segmented intervals according to its actual variation, and different linear functions are used for fitting in each segmented interval, so that local optimization can be performed for the characteristics of each interval. At the same time, the function near the segmentation point is also smoothed using a quadratic function, so that the final integrated piecewise linear fitting function can more accurately approximate the real response curve. Compared with the high-order polynomial function that may be involved in the polynomial fitting method, the calculation form of the linear function is simpler, avoiding complex high-order power operations and solving a large number of coefficients, which not only ensures high-precision fitting effects, but also greatly reduces the calculation complexity, and has a promising prospect for promotion and application. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solution in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 A schematic flow chart of an embodiment of a method for nonlinear correction of camera response provided in the present application; Figure 2 A schematic flow chart of another embodiment of the method for nonlinear correction of camera response provided by the present application; Figure 3 A schematic diagram of the structure of an embodiment of a system for nonlinear correction of camera response provided by the present application; Figure 4 A schematic diagram of the structure of an embodiment of a device for nonlinear correction of camera response provided in the present application. DETAILED DESCRIPTION

[0025] The present application provides a method, a related device and a storage medium for correcting nonlinear camera response, which are used to reduce the computational complexity and improve the correction quality while ensuring a high-precision fitting effect.

[0026] It should be noted that the method for nonlinear correction of camera response provided in this application can be applied to a terminal or a server. For example, the terminal can be a smart phone or a computer, a tablet computer, a portable computer terminal, or a fixed terminal such as a desktop computer. For the convenience of explanation, this application uses the terminal as the execution subject for example.

[0027] See also Figure 1 , Figure 1 An embodiment of a method for correcting nonlinearity of camera response provided by the present application includes: 101. Use the camera to be fitted to shoot a standard light source to obtain a series of original images at different exposure times, and obtain a data point set of exposure time and response value according to the original images, and the data point set is used to generate a camera response curve; Different cameras have different sensor and circuit designs, so their response curves are also different. By shooting a specific camera, you can obtain the unique response data of that camera, so that targeted correction can be achieved.

[0028] First, use the camera to be fitted to shoot a standard light source, which refers to a light source with a known and stable light intensity output. During the shooting process, the exposure time is controlled and a series of different exposure time values ​​are set to obtain a series of original images under different exposure times. In the captured image, a fixed area is selected. The fixed area should have uniform brightness, no obstruction and representative characteristics. Therefore, an area near the center of the image can be selected as the fixed area. For the original image at each exposure time, the average grayscale value of all pixels in the selected fixed area is calculated, and the average value can be used as the response value of the camera.

[0029] Each exposure time value is paired with its corresponding average grayscale value (response value) to form a series of data point sets. These data point sets reflect the response characteristics of the camera at different exposure times. Therefore, these data point sets can constitute the data required for fitting and generating the camera response curve.

[0030] 102. Calculate a segmentation threshold and determine a segmentation point according to the data point set, and divide the camera response curve into a plurality of segmentation intervals based on the segmentation point; Due to the nonlinear characteristics of the camera sensor and circuit design, this camera response curve is usually not a straight line, but presents complex nonlinear characteristics, and the degree of nonlinearity in different areas may be different. Therefore, this embodiment calculates the segmentation threshold based on the data point set, and uses the segmentation threshold to filter the segmentation points. It should be noted that the segmentation threshold is calculated based on the data point set, rather than a fixed value set manually, which means that the size of the segmentation threshold will be adjusted according to the actual camera response data and can adapt to the characteristics of different cameras. In some specific embodiments, the segmentation threshold can be calculated based on the slope of the curve, and the segmentation point can specifically be a point located at a position where the slope changes greatly in the camera response curve.

[0031] Afterwards, the complex camera response curve is divided into multiple segmented intervals based on the segmentation points. The segmented interval is a curve segment defined by two adjacent segmentation points, and each segmented interval contains a continuous camera response curve. Subsequently, the camera response curve in each interval can be individually linearly fitted. This method is more flexible than a single global polynomial fit and can better adapt to local changes in the curve.

[0032] 103. Use the least square method to fit each segmented interval to obtain the linear function corresponding to each segmented interval; The complex nonlinear camera response curve is decomposed into multiple approximately linear segmented intervals, and the least squares method is used to fit each segmented interval to obtain the linear function corresponding to each segmented interval, thereby converting the complex nonlinear problem into multiple simple linear problems, reducing the difficulty of fitting. Since each segmented interval is relatively small, the camera response curve is approximately linear within the segmented interval, so the linear function can well approximate the real curve.

[0033] 104. Integrate the linear functions of all segmented intervals and use a quadratic function to perform smoothing near the segmentation points to obtain a complete piecewise linear fitting function; Integrate the linear functions fitted in each segmented interval, that is, combine the linear functions in all segmented intervals according to their corresponding interval ranges to form a preliminary piecewise linear function. However, since each segmented interval is linearly fitted independently, there may be discontinuities between the linear functions at the segmentation points, that is, the function values ​​or slopes are not equal, resulting in obvious segmentation traces. These segmentation traces will affect the final correction effect, so smoothing is required near the segmentation points. Specifically, in order to eliminate the discontinuities that may occur in different linear functions at the segmentation points, a quadratic function can be used to smooth them so that the overall fitting curve is closer to the actual camera response curve. In addition to the quadratic function, other types of functions can also be selected for smoothing, such as cubic functions, spline functions, etc., but different types of functions have different smoothing effects and computational complexities, so it is preferred to use a quadratic function.

[0034] On the basis of the preliminary piecewise linear function, a smooth function is used to replace the linear function near the segmentation point, so that the curve is smoother and more continuous at the segmentation point. Finally, a complete piecewise linear fitting function is formed, which is used to define the camera response curve in the entire exposure time range.

[0035] 105. Apply piecewise linear fitting function to correct the image taken by the camera.

[0036] Using the complete piecewise linear fitting function obtained in step 104, for each pixel in the original image captured by the camera, find the segmented interval corresponding to the pixel in the piecewise linear fitting function according to the exposure time of the pixel, and then use the linear function corresponding to the segmented interval to calculate the corrected gray value and replace it. After performing the above operation on all pixels in the original image, an image after nonlinear correction is obtained, thereby eliminating the color distortion and contrast reduction caused by the nonlinear response of the camera in the original image, and improving the image quality. The final corrected image will be more realistic and consistent with the visual perception of the human eye.

[0037] In this embodiment, when the camera response curve has a complex variation pattern, it is divided into multiple segmented intervals according to the actual variation of the camera response curve, and different linear functions are used for fitting in each segmented interval, so that local optimization can be performed for the characteristics of each interval. At the same time, the function near the segmentation point is also smoothed using a quadratic function, so that the piecewise linear fitting function finally integrated can more accurately approximate the real response curve. Compared with the high-order polynomial function that may be involved in the polynomial fitting method, the calculation form of the linear function is simpler, avoiding complex high-order power operations and solving a large number of coefficients, which not only ensures a high-precision fitting effect, but also greatly reduces the calculation complexity, and has a prospect for promotion and application.

[0038] The following is a detailed description of the camera response nonlinear correction method provided by this application. Figure 2 , Figure 2 An embodiment of a method for correcting nonlinearity of camera response provided by the present application includes: 201. Use the camera to be fitted to shoot a standard light source to obtain a series of original images at different exposure times, and obtain a data point set of exposure time and response value according to the original images, and the data point set is used to generate a camera response curve; In this embodiment, step 201 is similar to step 101 in the aforementioned embodiment and will not be described again here.

[0039] 202. Calculate the slopes between adjacent data points in the data point set, and calculate the difference between adjacent slopes; Since the camera response curve usually has nonlinear characteristics, the change pattern of the curve can be identified by analyzing the rate of change (slope) between adjacent data points. Specifically, for each pair of adjacent data points, calculate its slope , the calculation formula is , , where n is the total number of data points. After that, the difference between adjacent slopes is calculated, which can reflect the change in the curvature of the camera response curve.

[0040] 203. Calculate the segmentation threshold according to the mean and standard deviation of the differences between adjacent slopes; By analyzing the statistical characteristics of the slope difference, a suitable threshold can be automatically determined to determine which points have a large enough curvature and should be used as segmentation points. Specifically, the segmentation threshold can be calculated using the mean and standard deviation. The segmentation threshold calculated in this way can be adaptively adjusted according to the characteristics of different camera response curves, avoiding the error caused by artificially setting a fixed threshold. For example, the segmentation threshold can be set to the mean plus 3 times the standard deviation, which can effectively identify significant slope differences.

[0041] 204. When the difference is greater than the segmentation threshold, a data point corresponding to the difference is determined as a segmentation point, and the camera response curve is divided into a plurality of segmentation intervals based on the segmentation point; The difference between the slopes of adjacent data points reflects the drastic degree of local changes in the curve. The area with too large a difference may correspond to the inflection point or nonlinear mutation of the curve. Therefore, the differences between the adjacent slopes calculated in step 202 are traversed, and each difference is compared with the segmentation threshold calculated in step 203. When a certain difference is greater than the segmentation threshold, the point is identified as a segmentation point, and the corresponding original data point is recorded. By comparing the slope difference with the segmentation threshold, the position of the segmentation point can be automatically determined, avoiding the error caused by artificially setting the segmentation point. After the segmentation point is determined, the camera response curve is divided into several segmentation intervals according to these segmentation points, each interval is defined by two adjacent segmentation points, the starting point of the first segmentation interval is the first data point, and the end point of the last segmentation interval is the last data point. Furthermore, the segmentation points can be further optimized, for example, segmentation points that are too close to each other are removed, or adjacent segmentation points are merged, and the optimized segmentation points are used as the final segmentation points.

[0042] The purpose of segmenting the response curve is to accurately fit the local characteristics. Through steps 202, 203 and 204, the curve can be automatically divided into several segment intervals according to the actual changes in the camera response curve. Dividing the response curve into multiple segments can better capture the changing characteristics in the region. Therefore, this adaptive segmentation method can effectively improve the fitting accuracy and adapt to the response characteristics of different cameras.

[0043] 205. Substitute the data points in each segmented interval into the form of a linear function, and construct the sum of squares of errors between the linear function and the data points according to the principle of least squares method, wherein the linear function contains the coefficients to be fitted; After dividing the segmented intervals, substitute the data points in each segmented interval into the linear function. Assume that in the i-th segmented interval , the linear function is in the form of ,in and is the coefficient to be determined for the i-th segment. After that, the data points in this interval are , j satisfies Substitute the linear function, calculate the error between the predicted value and the actual value, and construct the error sum of squares according to the principle of least squares method By quantifying the sum of squared errors , we can evaluate the degree of fit of the linear function to the data. The smaller the sum of squared errors, the better the fit. Since the linear function is simple in form and the construction and subsequent calculation complexity of the sum of squared errors are low, it is particularly suitable for large-scale data processing and real-time applications.

[0044] 206. Based on the minimization of the sum of squared errors, partial derivatives of the coefficients to be fitted are obtained to obtain a system of linear equations of two variables; In order to make the error sum of squares To achieve the minimum, we need to find the parameters that make it reach the minimum value and , whose minimum point corresponds to the solution with the smallest error. and By calculating the partial derivatives respectively and setting them equal to zero, we can find the location of the extreme point. and After sorting out the two partial derivative equations, we can get a system of linear equations of two variables. That is, the complex optimization problem is transformed into the problem of solving a system of linear equations, thereby simplifying the solution process and greatly reducing the computational complexity.

[0045] 207. Obtain the optimal fitting coefficient by solving the set of two-variable linear equations, and determine the linear function corresponding to each segment interval according to the optimal fitting coefficient; By solving the above two-variable linear equations, the optimal fitting coefficient of each segment interval can be calculated: and , thereby determining the corresponding linear function Since the error sum of squares of a linear function is a convex function, the solution is and It is the global optimal solution, and there is no local optimal trap. The linear function of each segmented interval is optimized independently. The linear equation group of each segmented interval is small in scale and simple to solve. There is no need for complex global joint optimization. It can accurately fit the local characteristics of the segmented interval, thereby improving the fitting quality of the overall curve. In this way, the local characteristics of each segmented interval are accurately fitted, while maintaining the global trend of the camera response curve, it can better capture local detail changes.

[0046] 208. Integrate the linear functions of all segmented intervals to obtain a piecewise linear function; Since each piecewise function is only responsible for correction within a specific exposure time range, it can cover the entire dynamic range after integration. Therefore, after obtaining the linear function that fits independently in each segmented interval, all linear functions are integrated into a global piecewise linear function, thereby obtaining a piecewise linear function that can describe the entire camera response curve. Specifically, for the entire definition domain, the function uses the corresponding linear function in each segmented interval to form a complete fitting model.

[0047] The piecewise linear function obtained by fitting is:

[0048] 209. Determine a transition region near each segmentation point, and define a smooth transition function in the transition region, where the smooth transition function is a quadratic function; Since piecewise linear fitting will produce discontinuities at the segmentation points, it is necessary to define a smooth transition function near the segmentation points. The role of the smooth transition function is to eliminate these discontinuities and make the overall function smoother. The specific approach is to define a transition area near each segmentation point, such as ], and then define a new smooth transition function in this transition area The quadratic function is chosen because its first-order derivative is continuous and can achieve a smooth transition effect. It should be noted that smoothing is only performed in a small area near the segmentation point, which not only retains the overall characteristics of the piecewise linear function, but also improves the smoothness of the key position. The size of the transition area It can be adjusted according to actual needs to strike a balance between smoothing effect and fitting accuracy.

[0049] In order to ensure a smooth transition, the smooth transition function can be solved by the conditions of equal boundary function values ​​and matching derivatives at segmentation points. Specifically, the smooth transition function needs to meet the following three conditions: Condition 1: On the left boundary is equal to the previous linear function; Condition 2: On the right boundary is equal to the next linear function; Condition 3: At the segmentation point The first-order derivatives match at , that is, take the average of the previous and next linear function derivatives to ensure the continuity of the first-order derivative.

[0050] By solving the linear equation established by the above three conditions, the values ​​of A, B, and C in the smooth transition function can be uniquely determined.

[0051] In some specific embodiments, the size of the transition region It can be determined according to the domain range of the linear function. That is, the transition area near each segmentation point is determined according to the domain of the linear function corresponding to each segmentation point and the preset ratio value, and the preset ratio value is a positive integer. If the function domain is ], then you can take ( is an appropriate positive integer, which is selected according to the actual curve) This definition method can better adapt to the definition domains of different scales, so that the range near the segmentation point has a certain proportional relationship with the entire definition domain.

[0052] 210. On the basis of piecewise linear function, a smooth transition function is introduced to generate a complete piecewise linear fitting function; By embedding the smooth transition function defined in step 209 into the piecewise linear function obtained in step 208, a complete and smooth piecewise linear fitting function is generated. The specific implementation method is to ] Use smooth transition function , while the original piecewise linear function is continued to be used in other areas. By embedding a smooth transition function, the new function is not only continuous in function value at the segmentation point, but also in the first-order derivative, avoiding mutations. While maintaining the accuracy of the piecewise linear fitting, the smoothing process makes the curve closer to the actual camera response characteristics, especially in areas with drastic changes. The fitting function generated by this method is suitable for complex camera response curves, such as high dynamic range scenes, and can effectively handle areas with large brightness changes.

[0053] 211. For each pixel in the image captured by the camera, find the corresponding target segment interval according to the exposure time; In the actual application of piecewise linear fitting function, the target segment interval to which each pixel belongs is determined according to the exposure time of each pixel in the image captured by the camera. The camera response curve is usually nonlinear. In order to correct it, the curve is divided into multiple segment intervals, each of which corresponds to a specific correction function. By finding the interval where the exposure time is located, the appropriate function can be selected for the subsequent grayscale value correction.

[0054] 212. Use a linear function or a smooth transition function corresponding to the target segmentation interval to calculate the corrected grayscale value, apply the corrected grayscale value to the image captured by the camera, and complete the nonlinear correction.

[0055] After determining the target segmented interval, the corresponding function is applied to calculate the corrected grayscale value of the pixel. Specifically: if the exposure time is inside a segmented interval (non-transition area), the linear function of the segmented interval is used; if the exposure time is in the transition area of ​​an adjacent segmented interval, the corresponding smooth transition function is used. According to the segmented interval where the exposure time is located and the corresponding function, the grayscale value that is more in line with the actual response of the camera is calculated to achieve nonlinear correction. In addition, a smooth transition function is used at the junction of the segmented intervals to ensure that the corrected grayscale value changes smoothly near the segmentation point to avoid mutations or discontinuities.

[0056] Finally, the calculated corrected grayscale value is applied to each pixel of the image to generate a final image after nonlinear correction. Through local and precise grayscale value correction, the true brightness information of the image is restored, the color distortion or contrast problem caused by the nonlinear response of the camera is reduced, and an image with better visual effects is generated. The corrected image can more realistically reflect the brightness information of the scene, especially in high dynamic range scenes, the details of highlights and shadows can be restored. This piecewise linear fitting method effectively compensates for the nonlinear characteristics of the camera response curve through local correction and transition processing. It is not only suitable for complex response curves, but also ensures the naturalness and authenticity of the image while maintaining computational efficiency.

[0057] The following is a detailed description of the camera response nonlinear correction system provided by this application. Figure 3 , Figure 3 Another embodiment of the system for nonlinear correction of camera response provided by the present application includes: An acquisition unit 301 is used to shoot a standard light source using a camera to be fitted to obtain a series of original images at different exposure times, and obtain a data point set of exposure time and response value according to the original images, and the data point set is used to generate a camera response curve; A determination unit 302, configured to calculate a segmentation threshold and determine a segmentation point according to a set of data points, and divide the camera response curve into a plurality of segmentation intervals based on the segmentation point; A fitting unit 303 is used to fit each segmented interval using the least square method to obtain a linear function corresponding to each segmented interval; A smoothing unit 304 is used to integrate the linear functions of all segmented intervals and use a quadratic function to perform smoothing near the segmentation points to obtain a complete piecewise linear fitting function; The correction unit 305 is used to correct the image taken by the camera by applying a piecewise linear fitting function.

[0058] Optionally, the determining unit 302 is specifically configured to: Calculate the slopes between adjacent data points in a set of data points, and calculate the difference between adjacent slopes; The segmentation threshold is calculated based on the mean and standard deviation of the differences between adjacent slopes; When the difference is greater than the segmentation threshold, the data point corresponding to the difference is determined as the segmentation point.

[0059] Optionally, the fitting unit 303 is specifically configured to: Substitute the data points in each segmented interval into the form of a linear function, and construct the sum of squares of the errors between the linear function and the data points according to the principle of least squares. The linear function contains the coefficients to be fitted. Based on minimization of the sum of squared errors, partial derivatives of the fitting coefficients are obtained to obtain a system of linear equations in two variables; The optimal fitting coefficient is obtained by solving a set of two-variable linear equations, and the linear function corresponding to each segment interval is determined based on the optimal fitting coefficient.

[0060] Optionally, the smoothing unit 304 is specifically configured to: Integrate the linear functions of all the segmented intervals to obtain a piecewise linear function; Determine the transition area near each segmentation point, and define a smooth transition function in the transition area, where the smooth transition function is a quadratic function; A smooth transition function is introduced on the basis of the piecewise linear function to generate a complete piecewise linear fitting function.

[0061] Optionally, the smoothing unit 304 is further configured to: The transition area near each segmentation point is determined according to the domain of the linear function corresponding to each segmentation point and a preset ratio value, and the preset ratio value is a positive integer.

[0062] Optionally, the smoothing unit 304 is further configured to: The smooth transition function is solved by the conditions of equal boundary function values ​​and matching derivatives at segment points.

[0063] Optionally, the correction unit 305 is specifically used for: For each pixel in the image captured by the camera, find the corresponding target segment interval according to the exposure time; The corrected grayscale value is calculated using a linear function or a smooth transition function corresponding to the target segmentation interval, and the corrected grayscale value is applied to the image captured by the camera to complete the nonlinear correction.

[0064] In this embodiment, when the camera response curve changes in a complex pattern, the determination unit 302 divides the camera response curve into multiple segmented intervals according to the actual changes in the camera response curve, and the fitting unit 303 uses different linear functions for fitting in each segmented interval, so that local optimization can be performed for the characteristics of each interval. At the same time, the smoothing unit 304 also uses a quadratic function to perform smooth transition processing on the function near the segmentation point, so that the final integrated piecewise linear fitting function can more accurately approximate the real response curve. Compared with the high-order polynomial function that may be involved in the polynomial fitting method, the calculation form of the linear function is simpler, avoiding complex high-order power operations and solving a large number of coefficients, which not only ensures a high-precision fitting effect, but also greatly reduces the calculation complexity, and has a prospect for promotion and application. In the system of this embodiment, the specific functions of each unit are the same as those mentioned above. Figure 2 The steps in the method embodiment shown correspond to each other and will not be repeated here.

[0065] This application also provides a device for nonlinear correction of camera response, see Figure 4 , Figure 4 An embodiment of a device for nonlinear correction of camera response provided by the present application includes: Processor 401, memory 402, input and output unit 403, bus 404; The processor 401 is connected to the memory 402, the input and output unit 403 and the bus 404; The memory 402 stores a program, and the processor 401 calls the program to execute any of the above methods for correcting the nonlinearity of camera response.

[0066] The present application also relates to a computer-readable storage medium, on which a program is stored. When the program is run on a computer, the computer executes any of the above-mentioned methods for nonlinear correction of camera response.

[0067] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0068] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0069] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0070] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0071] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A method for correcting nonlinearity of camera response, characterized in that: The method comprises: Using the camera to be fitted to shoot a standard light source, obtaining a series of original images at different exposure times, and obtaining a data point set of exposure time and response value according to the original images, wherein the data point set is used to generate a camera response curve; Calculating a segmentation threshold and determining a segmentation point according to the data point set, and dividing the camera response curve into a plurality of segmentation intervals based on the segmentation point; Fitting each segmented interval using the least square method to obtain a linear function corresponding to each segmented interval; Integrate the linear functions of all segmented intervals, and use a quadratic function to perform smoothing near the segmented points to obtain a complete piecewise linear fitting function; The piecewise linear fitting function is applied to correct the image taken by the camera.

2. The method according to claim 1, characterized in that The calculating the segmentation threshold and determining the segmentation point according to the data point set comprises: Calculating the slopes between adjacent data points in the data point set, and calculating the difference between adjacent slopes; Calculate the segmentation threshold according to the mean and standard deviation of the differences between the adjacent slopes; When the difference is greater than the segmentation threshold, the data point corresponding to the difference is determined as a segmentation point.

3. The method according to claim 1, characterized in that The least square method is used to fit each segmented interval to obtain a linear function corresponding to each segmented interval, including: Substituting the data points in each segmented interval into a linear function, constructing the sum of squares of errors between the linear function and the data points according to the least squares principle, wherein the linear function contains coefficients to be fitted; Taking partial derivatives of the coefficients to be fitted based on minimization of the sum of squared errors to obtain a system of linear equations in two variables; The optimal fitting coefficient is obtained by solving the set of two-variable linear equations, and the linear function corresponding to each segmented interval is determined according to the optimal fitting coefficient.

4. The method according to any one of claims 1 to 3, characterized in that The linear functions of all segmented intervals are integrated, and a quadratic function is used to perform smoothing near the segmented points to obtain a complete piecewise linear fitting function, including: Integrate the linear functions of all segmented intervals to obtain a piecewise linear function; Determine a transition region near each segmentation point, and define a smooth transition function in the transition region, wherein the smooth transition function is a quadratic function; The smooth transition function is introduced on the basis of the piecewise linear function to generate a complete piecewise linear fitting function.

5. The method according to claim 4, characterized in that Determine a transition area near each segmentation point, including: The transition area near each segmentation point is determined according to the domain of the linear function corresponding to each segmentation point and a preset ratio value, where the preset ratio value is a positive integer.

6. The method according to claim 4, characterized in that The method further comprises: The smooth transition function is solved by the conditions that the boundary function values ​​are equal and the derivatives at the segmentation points match.

7. The method according to claim 4, characterized in that The applying the piecewise linear fitting function to correct the image taken by the camera includes: For each pixel in the image captured by the camera, searching for a corresponding target segment interval according to the exposure time; The linear function or the smooth transition function corresponding to the target segmentation interval is used to calculate the corrected grayscale value, and the corrected grayscale value is applied to the image captured by the camera to complete the nonlinear correction.

8. A system for correcting nonlinearity of camera response, characterized in that: The system comprises: An acquisition unit, used to use the camera to be fitted to shoot the standard light source to obtain a series of original images under different exposure times, and obtain a data point set of exposure time and response value according to the original images, wherein the data point set is used to generate a camera response curve; A determination unit, configured to calculate a segmentation threshold and determine a segmentation point according to the data point set, and divide the camera response curve into a plurality of segmentation intervals based on the segmentation point; A fitting unit, used for fitting each segmented interval by using the least square method to obtain a linear function corresponding to each segmented interval; A smoothing unit, used for integrating the linear functions of all segmented intervals and performing smoothing near the segmented points using a quadratic function to obtain a complete piecewise linear fitting function; A correction unit is used to apply the piecewise linear fitting function to correct the image taken by the camera.

9. A device for correcting nonlinear camera response, characterized in that: The device comprises: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, and when the program is executed on a computer, the method according to any one of claims 1 to 7 is performed.

Images