A method, related device and storage medium for correcting camera response non-linearity

Abstract

The present application discloses a method for non-linear correction of camera response, related devices and storage media, which are used to reduce the computational complexity while ensuring a high-precision fitting effect and improving the correction quality. The method of the present application includes: using a camera to be fitted to photograph a standard light source to obtain a series of original images under different exposure times, and obtaining a data point set of exposure times and response values according to the original images, where the data point set is used to generate a camera response curve; calculating segment thresholds and determining segment points according to the data point set, and dividing the camera response curve into several segment intervals based on the segment points; using the least squares method to fit each segment interval to obtain a linear function corresponding to each segment interval; integrating the linear functions of all segment intervals, and performing smoothing processing near the segment points using a quadratic function to obtain a complete piecewise linear fitting function; applying the piecewise linear fitting function to correct the images captured by the camera.

Description

Technical Field

[0001] This application relates to the technical field of image processing, and in particular, to a method for correcting camera response non-linearity, related devices, and storage media. Background Art

[0002] In digital cameras and other image capture devices, due to the characteristics of the sensor itself and circuit design, their responses usually exhibit non-linear characteristics. This non-linear response can cause problems such as color distortion and contrast reduction in different brightness regions of the image, thereby affecting the overall quality and perception of the image.

[0003] The camera response curve often exhibits complex non-linear characteristics, and some simple fitting methods are difficult to accurately fit it. For example, in high dynamic range scenarios, the responses of the camera to strong light and weak light may have different non-linear laws. Existing fitting techniques may not be able to accurately fit the curves in different light intensity regions simultaneously, resulting in distortion of the details in the highlights or dark parts of the image. Some high-precision fitting algorithms, such as fitting methods based on complex mathematical models, although they can improve the fitting accuracy, have high computational complexity, require a large amount of computing resources and time, and cannot meet the real-time requirements.

[0004] In the prior art, the common correction algorithm is to fit the camera response curve by establishing a polynomial model. However, in the case where the camera response curve has obvious non-linearity and complex variation laws, the polynomial fitting method requires a high degree to achieve a certain fitting accuracy, and overfitting is likely to occur during the process, and an ideal correction effect cannot be obtained in practical applications. Summary of the Invention

[0005] This application provides a method for correcting camera response non-linearity, related devices, and storage media, which are used to reduce the computational complexity while ensuring a high-precision fitting effect and improve the correction quality.

[0006] The first aspect of this application provides a method for correcting camera response non-linearity, including:

[0007] Using the camera to be fitted to photograph a standard light source, obtaining a series of original images under different exposure times, and acquiring a data point set of exposure times and response values according to the original images, where the data point set is used to generate a camera response curve;

[0008] Calculating segment thresholds and determining segment points according to the data point set, and dividing the camera response curve into several segment intervals based on the segment points;

[0009] Using the least squares method to fit each segment interval to obtain a linear function corresponding to each segment interval;

[0010] Integrate the linear functions of all segmented intervals, and use a quadratic function to smooth near the segmentation points to obtain a complete piecewise linear fitting function;

[0011] Apply the piecewise linear fitting function to correct the image captured by the camera.

[0012] Optionally, calculating the segmentation threshold according to the set of data points and determining the segmentation points includes:

[0013] Calculate the slope between adjacent data points in the set of data points, and calculate the difference between adjacent slopes;

[0014] Calculate the segmentation threshold according to the mean and standard deviation of the differences between adjacent slopes;

[0015] When the difference is greater than the segmentation threshold, determine the data point corresponding to the difference as the segmentation point.

[0016] Optionally, fitting each of the segmented intervals using the least squares method to obtain the linear function corresponding to each segmented interval, includes:

[0017] Substitute the data points in each segmented interval into the form of a linear function, and construct the sum of squared errors between the linear function and the data points according to the least squares principle. The linear function contains coefficients to be fitted;

[0018] Take the partial derivative of the coefficients to be fitted based on the minimization of the sum of squared errors to obtain a system of linear equations in two variables;

[0019] Solve the system of linear equations in two variables to obtain the optimal fitting coefficients, and determine the linear function corresponding to each segmented interval according to the optimal fitting coefficients.

[0020] Optionally, integrating the linear functions of all segmented intervals and using a quadratic function to smooth near the segmentation points to obtain a complete piecewise linear fitting function, includes:

[0021] Integrate the linear functions of all segmented intervals to obtain a piecewise linear function;

[0022] Determine the transition region near each segmentation point, and define a smooth transition function within the transition region. The smooth transition function is a quadratic function;

[0023] Introduce the smooth transition function on the basis of the piecewise linear function to generate a complete piecewise linear fitting function.

[0024] Optionally, determining the transition region near each segmentation point includes:

[0025] Determine the transition region near each of the segmentation points according to the domain of the linear function corresponding to each of the segmentation points and a preset proportional value, where the preset proportional value is a positive integer.

[0026] Optionally, the method further includes:

[0027] Solve the smooth transition function by the conditions that the boundary function values are equal and the derivatives at the segmentation points match.

[0028] Optionally, applying the piecewise linear fitting function to correct the image captured by the camera includes:

[0029] For each pixel in the image captured by the camera, find the corresponding target segmentation interval according to the exposure time;

[0030] Calculate the corrected gray value using the linear function or the smooth transition function corresponding to the target segmentation interval, and apply the corrected gray value to the image captured by the camera to complete the non-linear correction.

[0031] A second aspect of the present application provides a system for non-linear correction of camera response, including:

[0032] An acquisition unit, configured to use a camera to be fitted to photograph a standard light source to obtain a series of original images at different exposure times, and obtain a data point set of exposure time and response value according to the original images, where the data point set is used to generate a camera response curve;

[0033] A determination unit, configured to calculate segmentation thresholds and determine segmentation points according to the data point set, and divide the camera response curve into several segmentation intervals based on the segmentation points;

[0034] A fitting unit, configured to perform least squares fitting on each of the segmentation intervals to obtain a linear function corresponding to each of the segmentation intervals;

[0035] A smoothing unit, configured to integrate the linear functions of all segmentation intervals and perform smoothing processing near the segmentation points using a quadratic function to obtain a complete piecewise linear fitting function;

[0036] A correction unit, configured to apply the piecewise linear fitting function to correct the image captured by the camera.

[0037] Optionally, the determination unit is specifically configured to:

[0038] Calculate the slope between adjacent data points in the data point set, and calculate the difference between adjacent slopes;

[0039] Calculate the segmentation threshold according to the mean and standard deviation of the differences between adjacent slopes;

[0040] When the difference is greater than the segmentation threshold, determine the data point corresponding to the difference as a segmentation point.

[0041] Optionally, the fitting unit is specifically configured to:

[0042] Substitute the data points in each segmentation interval into the form of a linear function, and construct the sum of squared errors between the linear function and the data points according to the least squares principle. The linear function contains coefficients to be fitted;

[0043] Take the partial derivatives of the coefficients to be fitted based on the minimization of the sum of squared errors to obtain a system of linear equations with two variables;

[0044] Solve the system of linear equations with two variables to obtain the optimal fitting coefficients, and determine the linear function corresponding to each segmentation interval according to the optimal fitting coefficients.

[0045] Optionally, the smoothing unit is specifically configured to:

[0046] Integrate the linear functions of all segmentation intervals to obtain a piecewise linear function;

[0047] Determine the transition region near each segmentation point, and define a smooth transition function within the transition region. The smooth transition function is a quadratic function;

[0048] Introduce the smooth transition function on the basis of the piecewise linear function to generate a complete piecewise linear fitting function.

[0049] Optionally, the smoothing unit is specifically further configured to:

[0050] Determine the transition region near each segmentation point according to the domain of the linear function corresponding to each segmentation point and a preset proportional value. The preset proportional value is a positive integer.

[0051] Optionally, the smoothing unit is specifically further configured to:

[0052] Solve the smooth transition function through the conditions of equal boundary function values and matching derivatives at the segmentation points.

[0053] Optionally, the correction unit is specifically configured to:

[0054] For each pixel in the image captured by the camera, find the corresponding target segmentation interval according to the exposure time;

[0055] Calculate the corrected grayscale value using the linear function or the smooth transition function corresponding to the target segmented interval, and apply the corrected grayscale value to the image captured by the camera to complete the non-linear correction.

[0056] The third aspect of the present application provides a device for non-linear correction of camera response, and the device includes:

[0057] A processor, a memory, an input / output unit, and a bus;

[0058] The processor is connected to the memory, the input / output unit, and the bus;

[0059] The memory stores a program, and the processor calls the program to execute the method for non-linear correction of camera response in the first aspect and any optional one in the first aspect.

[0060] The fourth aspect of the present application provides a computer-readable storage medium, and a program is stored on the computer-readable storage medium, and when the program is executed on a computer, it executes the method for non-linear correction of camera response in the first aspect and any optional one in the first aspect.

[0061] It can be seen from the above technical solutions that the present application has the following advantages:

[0062] In the case where the change rule of the camera response curve is complex, the camera response curve is divided into multiple segmented intervals according to the actual change situation of the camera response curve, and different linear functions are used for fitting in each segmented interval, so as to be able to perform local optimization for the characteristics of each interval. At the same time, a quadratic function is used for smooth transition processing of the function near the segmentation point, so that the finally integrated piecewise linear fitting function can more accurately approximate the true response curve. Compared with the high-degree polynomial functions that may be involved in the polynomial fitting method, the calculation form of the linear function is simpler, avoiding complex high-power operations and the solution of a large number of coefficients, ensuring both a high-precision fitting effect and greatly reducing the calculation complexity, and having a promotion and application prospect. Description of the Drawings

[0063] In order to more clearly illustrate the technical solutions in the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0064] Figure 1 It is a schematic flowchart of an embodiment of the method for non-linear correction of camera response provided by the present application;

[0065] Figure 2Another embodiment flowchart of the method for correcting the non-linearity of camera response provided by this application;

[0066] Figure 3 A schematic structural diagram of an embodiment of the system for correcting the non-linearity of camera response provided by this application;

[0067] Figure 4 A schematic structural diagram of an embodiment of the device for correcting the non-linearity of camera response provided by this application. Detailed implementation manners

[0068] This application provides a method, related device and storage medium for correcting the non-linearity of camera response, which are used to reduce the computational complexity while ensuring a high-precision fitting effect and improve the correction quality.

[0069] It should be noted that the method for correcting the non-linearity of camera response provided by this application can be applied to a terminal or a server. For example, the terminal can be a smart phone, a computer, a tablet computer, a portable computer terminal, or a fixed terminal such as a desktop computer. For the convenience of description, this application takes the terminal as the execution subject for example.

[0070] Please refer to Figure 1 , Figure 1 which is an embodiment of the method for correcting the non-linearity of camera response provided by this application. The method includes:

[0071] 101. Use the camera to be fitted to photograph a standard light source, obtain a series of original images under different exposure times, and obtain a set of data points of exposure time and response value according to the original images. The set of data points is used to generate a camera response curve;

[0072] Different cameras have different sensor and circuit designs, so their response curves are also different. By photographing a specific camera, the unique response data of this camera can be obtained, so as to achieve targeted correction.

[0073] First, use the camera to be fitted to photograph a standard light source, which refers to a light source with a known and stable light intensity output. Control the exposure time during the photographing process, set a series of different exposure time values, so as to obtain a series of original images under different exposure times. In the photographed images, select a fixed area, which should have the characteristics of uniform brightness, no occlusion and representativeness. Therefore, an area near the center of the image can be selected as the fixed area. For each original image under a different exposure time, calculate the average value of the gray values of all pixels in the selected fixed area, and this average value can be used as the response value of the camera.

[0074] Pair each exposure time value with its corresponding average grayscale value (response value) to form a series of data point sets. These data point sets reflect the response characteristics of the camera at different exposure times. Therefore, these data point sets can constitute the data required for fitting and generating the camera response curve.

[0075] 102. Calculate the piecewise thresholds based on the data point sets and determine the breakpoints. Divide the camera response curve into several piecewise intervals based on the breakpoints.

[0076] Due to the non-linear characteristics of the camera sensor and circuit design, this camera response curve is usually not a straight line but exhibits complex non-linear characteristics, and the degree of non-linearity may vary in different regions. Therefore, in this embodiment, the piecewise thresholds are calculated based on the data point sets, and the breakpoints are selected using these piecewise thresholds. It should be noted that the piecewise thresholds are calculated based on the data point sets rather than being fixed values set artificially. This means that the size of the piecewise thresholds will be adjusted according to the actual camera response data and can adapt to the characteristics of different cameras. In some specific embodiments, the piecewise thresholds can be calculated based on the curve slope, and the breakpoint can specifically be the point located at a position with a relatively large change in slope in the camera response curve.

[0077] After that, based on the breakpoints, the complex camera response curve is divided into multiple piecewise intervals. Each piecewise interval is a curve segment defined by two adjacent breakpoints, and each piecewise interval contains a continuous segment of the camera response curve. Subsequently, linear fitting can be performed separately on the camera response curve within each interval. This method is more flexible than a single global polynomial fitting and can better adapt to the local changes of the curve.

[0078] 103. Use the least squares method to fit each piecewise interval to obtain the linear function corresponding to each piecewise interval.

[0079] Decompose the complex non-linear camera response curve into multiple approximately linear piecewise intervals, and use the least squares method to fit each piecewise interval to obtain the linear function corresponding to each piecewise interval. Thus, the complex non-linear problem is transformed into multiple simple linear problems, reducing the fitting difficulty. Since each piecewise interval is relatively small and the camera response curve is approximately linear within this piecewise interval, the linear function can well approximate the real curve.

[0080] 104. Integrate the linear functions of all piecewise intervals and use a quadratic function to perform smoothing near the breakpoints to obtain the complete piecewise linear fitting function.

[0081] Integrate the linear functions fitted within each segmented interval, that is, combine the linear functions within all segmented intervals according to their corresponding interval ranges to form a preliminary piecewise linear function. However, since each segmented interval is linearly fitted independently, there may be discontinuities between the linear functions at the segmentation points, that is, the function values or slopes are not equal, resulting in obvious segmentation traces. These segmentation traces will affect the final correction effect, so smoothing processing needs to be performed near the segmentation points. Specifically, in order to eliminate the possible discontinuities of different linear functions at the segmentation points, a quadratic function can be used for smoothing processing to make the overall fitted curve closer to the true camera response curve. In addition to the quadratic function, other types of functions can also be selected for smoothing processing, such as cubic functions, spline functions, etc. However, different types of functions have different smoothing effects and computational complexities, so the quadratic function is preferably used.

[0082] Based on the preliminary piecewise linear function, use a smoothing function to replace the linear function near the segmentation point, so that the curve is smoother and more continuous at the segmentation point, and finally form a complete piecewise linear fitting function, which is used to define the camera response curve within the entire exposure time range.

[0083] 105. Apply the piecewise linear fitting function to correct the image captured by the camera.

[0084] Using the complete piecewise linear fitting function obtained in step 104, for each pixel in the original image captured by the camera, find the corresponding segmented interval of the pixel in the piecewise linear fitting function according to the exposure time of the pixel, and then use the linear function corresponding to the segmented interval to calculate the corrected gray value and replace it. After performing the above operations on all pixels in the original image, an image after non-linear correction is obtained, thereby eliminating the color distortion and contrast reduction caused by the non-linearity of the camera response in the original image, and achieving an improvement in image quality. The finally corrected image will be more realistic and conform to the visual perception of the human eye.

[0085] In this embodiment, in the case where the variation law of the camera response curve is complex, the camera response curve is divided into multiple segmented intervals according to the actual variation of the camera response curve, and different linear functions are used for fitting within each segmented interval, so as to be able to perform local optimization for the characteristics of each interval. At the same time, a quadratic function is also used for smooth transition processing of the function near the segmentation point, so that the finally integrated piecewise linear fitting function can more accurately approximate the true response curve. Compared with the high-degree polynomial functions that may be involved in the polynomial fitting method, the calculation form of the linear function is simpler, avoiding complex high-power operations and the solution of a large number of coefficients, ensuring both a high-precision fitting effect and greatly reducing the computational complexity, and having the prospect of popularization and application.

[0086] The method for correcting the non - linearity of camera response provided by the present application will be described in detail below. Please refer to Figure 2 , Figure 2 This is an embodiment of the method for correcting the non - linearity of camera response provided by the present application. The method includes:

[0087] 201. Use the camera to be fitted to photograph a standard light source, obtain a series of original images under different exposure times, and acquire a set of data points of exposure time and response value according to the original images. The set of data points is used to generate a camera response curve;

[0088] In this embodiment, step 201 is similar to step 101 of the foregoing embodiment, and will not be elaborated here.

[0089] 202. Calculate the slope between adjacent data points in the set of data points, and calculate the difference between adjacent slopes;

[0090] Since the camera response curve usually has non - linear characteristics, the change pattern of the curve can be identified by analyzing the change rate (slope) between adjacent data points at this time. Specifically, for each pair of adjacent data points, calculate its slope , and the calculation formula is , , where n is the total number of data points. Thereafter, calculate the difference between adjacent slopes, and this difference can reflect the change in the bending degree of the camera response curve.

[0091] 203. Calculate the segmentation threshold according to the mean and standard deviation of the difference between adjacent slopes;

[0092] By analyzing the statistical characteristics of the slope differences, a suitable threshold can be automatically determined to judge which points have a large enough bending degree and should be used as segmentation points. Specifically, the segmentation threshold can be calculated using the mean and standard deviation. The calculated segmentation threshold can be adaptively adjusted according to the characteristics of different camera response curves, avoiding the error caused by artificially setting a fixed threshold. For example, the segmentation threshold can be set to the mean plus 3 times the standard deviation, so as to effectively identify significant slope differences.

[0093] 204. When the difference is greater than the segmentation threshold, determine the data point corresponding to the difference as a segmentation point, and divide the camera response curve into several segmentation intervals based on the segmentation point;

[0094] The difference in the slopes of adjacent data points reflects the degree of local variation of the curve. Regions with excessive differences may correspond to inflection points or non-linear mutations of the curve. Therefore, traverse the differences between adjacent slopes calculated in step 202, and compare each difference with the segment threshold calculated in step 203. When a certain difference is greater than the segment threshold, mark this point as a segment point and record the corresponding original data point. By comparing the slope difference with the segment threshold, the position of the segment point can be automatically determined, avoiding the errors caused by manually setting the segment point. After determining the segment points, divide the camera response curve into several segment intervals according to these segment points. Each interval is defined by two adjacent segment points. The starting point of the first segment interval is the first data point, and the ending point of the last segment interval is the last data point. Further, the segment points can be further optimized. For example, remove segment points that are too close, or merge adjacent segment points, and use the optimized segment points as the final segment points.

[0095] The purpose of segmenting in the response curve is to accurately fit the local characteristics. Through steps 202, 203, and 204, the curve can be automatically divided into several segment intervals according to the actual changes of the camera response curve. Dividing the response curve into multiple segments can better capture the change characteristics within the region. Therefore, this adaptive segmentation method can effectively improve the fitting accuracy and adapt to the response characteristics of different cameras.

[0096] 205. Substitute the data points within each segment interval into the form of a linear function, and construct the sum of squared errors between the linear function and the data points according to the least squares principle. The linear function contains the coefficients to be fitted.

[0097] After dividing the segment intervals, substitute the data points within each segment interval into the linear function. Assume that within the i-th segment interval , the form of the linear function is , where and are the coefficients to be determined for the i-th segment to be fitted. After that, substitute the data points within this interval, where j satisfies into the linear function, calculate the error between the predicted value and the actual value, and construct the sum of squared errors according to the least squares principle. By quantifying this sum of squared errors , the fitting degree of the linear function to the data can be evaluated. The smaller the sum of squared errors, the better the fitting. Since the form of the linear function is simple, and the construction and subsequent calculation complexity of the sum of squared errors are low, it is especially suitable for large-scale data processing and real-time applications.

[0098] 206. Take the partial derivatives of the coefficients to be fitted based on the minimization of the sum of squared errors to obtain a system of linear equations with two variables.

[0099] To minimize the sum of squared errors it is necessary to find the parameters that minimize it and , and the minimum point corresponds to the solution with the minimum error. By taking the partial derivatives of and respectively and setting the partial derivatives equal to zero, the position of this extreme point can be found. After organizing the two partial derivative equations of and , a system of linear equations with two variables can be obtained. That is, the complex optimization problem is transformed into the problem of solving a system of linear equations, thereby simplifying the solution process and significantly reducing the computational complexity.

[0100] 207. Obtain the optimal fitting coefficients by solving the system of linear equations with two variables, and determine the linear function corresponding to each segmented interval according to the optimal fitting coefficients;

[0101] By solving the above system of linear equations with two variables, the optimal fitting coefficients and for each segmented interval can be calculated, so as to determine the corresponding linear function . Since the sum of squared errors of the linear function is a convex function, the obtained and are the global optimal solutions, and there is no local optimal trap. Moreover, the linear functions of each segmented interval are independently optimized, the scale of the system of linear equations for each segmented interval is small, the solution is simple, and there is no need for complex global joint optimization. It can accurately fit according to the local characteristics of the segmented interval, thereby improving the fitting quality of the overall curve. By accurately fitting the local characteristics of each segmented interval in this way, while maintaining the global trend of the camera response curve, it can better capture the local detail changes.

[0102] 208. Integrate the linear functions of all segmented intervals to obtain a piecewise linear function;

[0103] Since each piecewise function is only responsible for the correction within a specific exposure time range, it can only cover the entire dynamic range after integration. Therefore, after obtaining the independently fitted linear functions within each segmented interval, all the linear functions are integrated into a global piecewise linear function, so as to obtain a piecewise linear function that can describe the entire camera response curve. Specifically, for the entire domain, the function uses the corresponding linear function within each segmented interval, thereby forming a complete fitting model.

[0104] The piecewise linear function obtained by fitting is:

[0105]

[0106] 209. Determine the transition regions near each segmentation point, and define a smooth transition function within the transition regions. The smooth transition function is a quadratic function;

[0107] Since piecewise linear fitting will produce discontinuities at the segmentation points, it is necessary to define a smooth transition function near the segmentation points. The role of the smooth transition function is to eliminate these discontinuities and make the overall function smoother. The specific approach is to define a transition region near each segmentation point, such as , and then define a new smooth transition function within this transition region . The quadratic function is chosen because its first derivative is continuous, enabling a smooth transition effect. It should be noted that the smoothing is only performed within a small region near the segmentation point, which not only preserves the overall characteristics of the piecewise linear function but also improves the smoothness at key positions. The size of the transition region can be adjusted according to actual requirements to achieve a balance between the smoothing effect and the fitting accuracy.

[0108] To ensure a smooth transition, the smooth transition function can be solved by the conditions of equal boundary function values and matching derivatives at the segmentation points. Specifically, the smooth transition function needs to satisfy the following three conditions:

[0109] Condition 1: It is equal to the previous linear function at the left boundary ;

[0110] Condition 2: It is equal to the next linear function at the right boundary ;

[0111] Condition 3: The first derivatives match at the segmentation point , that is, take the average value of the derivatives of the previous and next linear functions to ensure the continuity of the first derivative.

[0112] By solving the linear equations established by the above three conditions, the values of A, B, and C in the smooth transition function can be uniquely determined.

[0113] In some specific embodiments, the size of the transition region can be determined according to the domain range of the linear function. That is, the transition regions near each segmentation point are determined according to the domain of the linear function corresponding to each segmentation point and a preset proportional value, and the preset proportional value is a positive integer. If the function domain is , then ( is an appropriate positive integer selected according to the actual curve) can be taken. Such a definition method can better adapt to domains of different scales, making the range near the segmentation point have a certain proportional relationship with the entire domain.

[0114] 210. Introduce a smooth transition function based on the piecewise linear function to generate a complete piecewise linear fitting function;

[0115] By embedding the smooth transition function defined in step 209 into the piecewise linear function obtained in step 208, a complete and smooth piecewise linear fitting function is generated. The specific implementation method is to use the smooth transition function within the transition region , and continue to use the original piecewise linear function in other regions. By embedding the smooth transition function, the new function is not only continuous in function value but also continuous in the first derivative at the piecewise points, avoiding mutations. While maintaining the piecewise linear fitting accuracy, the smoothing process makes the curve closer to the true camera response characteristics, especially in regions with drastic changes. The fitting function generated by this method is applicable to complex camera response curves, such as high-dynamic-range scenes, and can effectively handle regions with large brightness changes.

[0116] 211. For each pixel in the image captured by the camera, find the corresponding target piecewise interval according to the exposure time;

[0117] In the practical application of the piecewise linear fitting function, according to the exposure time of each pixel in the image captured by the camera, determine the target piecewise interval to which it belongs. The camera response curve is usually non-linear. To correct it, the curve is divided into multiple piecewise intervals, and each interval corresponds to a specific correction function. By finding the interval where the exposure time is located, an appropriate function can be selected for the subsequent gray value correction.

[0118] 212. Calculate the corrected gray value using the linear function or smooth transition function corresponding to the target piecewise interval, and apply the corrected gray value to the image captured by the camera to complete the non-linear correction.

[0119] After determining the target piecewise interval, the corresponding function is applied to calculate the corrected gray value of the pixel. Specifically: if the exposure time is within the interior (non-transition region) of a certain piecewise interval, the linear function of that piecewise interval is used; if the exposure time is in the transition region between adjacent piecewise intervals, the corresponding smooth transition function is used. According to the piecewise interval where the exposure time is located and the corresponding function, a gray value that is more in line with the true response of the camera is calculated, thereby achieving non-linear correction. And the smooth transition function is used at the junction of the piecewise intervals to ensure that the corrected gray value changes smoothly near the piecewise points, avoiding mutations or discontinuities.

[0120] Finally, apply the calculated corrected gray value to each pixel of the image to generate the final image after non - linear correction. Through local and precise gray value correction, restore the true brightness information of the image, reduce color distortion or contrast problems caused by the non - linearity of the camera response, and generate an image with better visual effects. The corrected image can more truly reflect the brightness information of the scene. Especially in high - dynamic - range scenes, it can restore the details of highlights and dark parts. This method of piece - wise linear fitting effectively compensates for the non - linear characteristics of the camera response curve through local correction and transition processing. It is not only applicable to the case of complex response curves, but also can ensure the naturalness and authenticity of the image while maintaining the calculation efficiency.

[0121] The following will provide a detailed description of the system for non - linear correction of camera response provided by this application. Please refer to Figure 3 , Figure 3 Another embodiment of the system for non - linear correction of camera response provided by this application. The system includes:

[0122] An acquisition unit 301, configured to use a camera to be fitted to photograph a standard light source, obtain a series of original images under different exposure times, and obtain a set of data points of exposure time and response value according to the original images. The set of data points is used to generate a camera response curve;

[0123] A determination unit 302, configured to calculate piece - wise thresholds and determine break points according to the set of data points, and divide the camera response curve into several piece - wise intervals based on the break points;

[0124] A fitting unit 303, configured to perform least - squares fitting on each piece - wise interval to obtain a linear function corresponding to each piece - wise interval;

[0125] A smoothing unit 304, configured to integrate the linear functions of all piece - wise intervals and perform smoothing processing near the break points using a quadratic function to obtain a complete piece - wise linear fitting function;

[0126] A correction unit 305, configured to correct the image captured by the camera by applying the piece - wise linear fitting function.

[0127] Optionally, the determination unit 302 is specifically configured to:

[0128] Calculate the slope between adjacent data points in the set of data points, and calculate the difference between adjacent slopes;

[0129] Calculate the piece - wise threshold according to the mean and standard deviation of the differences between adjacent slopes;

[0130] When the difference is greater than the piece - wise threshold, determine the data point corresponding to the difference as the break point.

[0131] Optionally, the fitting unit 303 is specifically configured to:

[0132] Substitute the data points in each segmented interval into the form of a linear function, construct the sum of squared errors between the linear function and the data points according to the principle of least squares, and the linear function contains the coefficients to be fitted.

[0133] Take the partial derivatives of the coefficients to be fitted based on the minimization of the sum of squared errors to obtain a system of linear equations with two variables.

[0134] Solve the system of linear equations with two variables to obtain the optimal fitting coefficients, and determine the linear function corresponding to each segmented interval according to the optimal fitting coefficients.

[0135] Optionally, the smoothing unit 304 is specifically configured to:

[0136] Integrate the linear functions of all segmented intervals to obtain a piecewise linear function.

[0137] Determine the transition region near each segmentation point, and define a smooth transition function within the transition region. The smooth transition function is a quadratic function.

[0138] Introduce the smooth transition function on the basis of the piecewise linear function to generate a complete piecewise linear fitting function.

[0139] Optionally, the smoothing unit 304 is also specifically configured to:

[0140] Determine the transition region near each segmentation point according to the domain of the linear function corresponding to each segmentation point and a preset proportional value. The preset proportional value is a positive integer.

[0141] Optionally, the smoothing unit 304 is also specifically configured to:

[0142] Solve the smooth transition function through the conditions of equal boundary function values and matching derivatives at the segmentation points.

[0143] Optionally, the correction unit 305 is specifically configured to:

[0144] For each pixel in the image captured by the camera, find the corresponding target segmented interval according to the exposure time.

[0145] Calculate the corrected gray value using the linear function or smooth transition function corresponding to the target segmented interval, and apply the corrected gray value to the image captured by the camera to complete the non-linear correction.

[0146] In this embodiment, in the case where the variation law of the camera response curve is complex, the determination unit 302 divides it into multiple segmented intervals according to the actual variation of the camera response curve. The fitting unit 303 uses different linear functions for fitting within each segmented interval, so as to perform local optimization for the characteristics of each interval. At the same time, the smoothing unit 304 also uses a quadratic function to perform a smoothing transition process on the function near the segmentation point, so that the finally integrated piecewise linear fitting function can more accurately approximate the true response curve. Compared with the high-degree polynomial functions that may be involved in the polynomial fitting method, the calculation form of the linear function is simpler, avoiding complex high-power operations and the solution of a large number of coefficients. It not only ensures a high-precision fitting effect but also greatly reduces the computational complexity, and has the prospect of popularization and application. In the system of this embodiment, the specific functions of each unit correspond to the steps in the foregoing Figure 2 The steps in the method embodiment shown are corresponding, and will not be elaborated here.

[0147] This application also provides a device for non-linear correction of camera response. Please refer to Figure 4 , Figure 4 which is an embodiment of the device for non-linear correction of camera response provided by this application. The device includes:

[0148] A processor 401, a memory 402, an input / output unit 403, and a bus 404;

[0149] The processor 401 is connected to the memory 402, the input / output unit 403, and the bus 404;

[0150] The memory 402 stores a program, and the processor 401 calls the program to execute any one of the methods for non-linear correction of camera response as described above.

[0151] This application also relates to a computer-readable storage medium on which a program is stored. When the program runs on a computer, the computer is enabled to execute any one of the methods for non-linear correction of camera response as described above.

[0152] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated here.

[0153] In several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of devices or units can be in electrical, mechanical, or other forms.

[0154] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] In addition, each functional unit in various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0156] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of the present application. The foregoing storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs and other various media that can store program codes.

Claims

1. A method for correcting nonlinearity of camera response, characterized in that: The method comprises: Using the camera to be fitted to shoot a standard light source, obtaining a series of original images at different exposure times, and obtaining a data point set of exposure time and response value according to the original images, wherein the data point set is used to generate a camera response curve; Calculating the slopes between adjacent data points in the data point set, and calculating the differences between adjacent slopes, calculating a segmentation threshold according to the mean and standard deviation of the differences between adjacent slopes, and when the difference is greater than the segmentation threshold, determining the data point corresponding to the difference as a segmentation point; Dividing the camera response curve into a plurality of segmentation intervals based on the segmentation points; Fitting each segmented interval using the least square method to obtain a linear function corresponding to each segmented interval; Integrate the linear functions of all segmented intervals to obtain a piecewise linear function; Determine a transition region near each segmentation point, and define a smooth transition function in the transition region, wherein the smooth transition function is a quadratic function; The smooth transition function is introduced on the basis of the piecewise linear function to generate a complete piecewise linear fitting function; The piecewise linear fitting function is applied to correct the image taken by the camera.

2. The method according to claim 1, characterized in that The least square method is used to fit each segmented interval to obtain a linear function corresponding to each segmented interval, including: Substituting the data points in each segmented interval into a linear function, constructing the sum of squares of errors between the linear function and the data points according to the least squares principle, wherein the linear function contains coefficients to be fitted; Taking partial derivatives of the coefficients to be fitted based on minimization of the sum of squared errors to obtain a system of linear equations in two variables; The optimal fitting coefficient is obtained by solving the set of two-variable linear equations, and the linear function corresponding to each segmented interval is determined according to the optimal fitting coefficient.

3. The method according to claim 1, characterized in that Determine a transition area near each segmentation point, including: The transition area near each segmentation point is determined according to the domain of the linear function corresponding to each segmentation point and a preset ratio value, where the preset ratio value is a positive integer.

4. The method according to claim 1, characterized in that: The method further comprises: The smooth transition function is solved by the conditions that the boundary function values ​​are equal and the derivatives at the segmentation points match.

5. The method according to claim 1, characterized in that The applying the piecewise linear fitting function to correct the image taken by the camera includes: For each pixel in the image captured by the camera, searching for a corresponding target segmentation interval according to the exposure time; The linear function or the smooth transition function corresponding to the target segmentation interval is used to calculate the corrected grayscale value, and the corrected grayscale value is applied to the image captured by the camera to complete the nonlinear correction.

6. A system for correcting nonlinearity of camera response, characterized in that: The system comprises: An acquisition unit, used to use the camera to be fitted to shoot the standard light source to obtain a series of original images under different exposure times, and obtain a data point set of exposure time and response value according to the original images, wherein the data point set is used to generate a camera response curve; A determination unit, configured to calculate the slopes between adjacent data points in the data point set, and calculate the differences between adjacent slopes, calculate a segmentation threshold according to the mean and standard deviation of the differences between adjacent slopes, and when the difference is greater than the segmentation threshold, determine the data point corresponding to the difference as a segmentation point; and divide the camera response curve into a plurality of segmentation intervals based on the segmentation points; A fitting unit, used for fitting each segmented interval by using the least square method to obtain a linear function corresponding to each segmented interval; A smoothing unit is used to integrate the linear functions of all segmented intervals to obtain a piecewise linear function; determine a transition area near each segmentation point, and define a smooth transition function in the transition area, wherein the smooth transition function is a quadratic function; introduce the smooth transition function on the basis of the piecewise linear function to generate a complete piecewise linear fitting function; A correction unit is used to apply the piecewise linear fitting function to correct the image taken by the camera.

7. A device for correcting nonlinear camera response, characterized in that: The device comprises: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, and when the program is executed on a computer, the method according to any one of claims 1 to 5 is performed.

Images