Camera module compensation method and device, storage medium and program product

Abstract

The application provides a camera module compensation method and device, a storage medium and a program product. The method is applied to a terminal device deployed with a camera module, the camera module includes multiple cameras, and the camera module compensation method includes: in response to triggering compensation of the camera module, obtaining a rotation compensation angle of the camera module, the rotation compensation angle is determined based on image data, and the image data is obtained by the camera module under the condition that the terminal device is in a stable posture; and based on the rotation compensation angle, the camera module is rotationally compensated. In the application, the accuracy and stability of the rotation compensation angle are improved, the consistency of picture alignment and switching connection between multiple cameras is improved, and the picture position jump phenomenon during camera switching is reduced.

Description

Technical Field

[0001] This application relates to the field of image processing and camera calibration compensation technology, and in particular to a camera module compensation method, device, storage medium and program product. Background Technology

[0002] With the continuous evolution of imaging capabilities in mobile devices such as smartphones and tablets, multi-camera systems have become a core feature of high-end devices, providing users with a rich shooting experience including ultra-wide-angle, optical zoom, and portrait modes. These multi-camera systems integrate multiple camera modules with different focal lengths and angles of view within the device, allowing users to seamlessly switch between them during shooting to meet diverse shooting needs.

[0003] Currently, to address the issue of camera module misalignment caused by mechanical tolerances during the assembly of multi-camera systems, the relevant technologies mainly employ the following approaches: First, in the assembly stage, the installation tolerance between the camera module and the body is controlled through mechanical structure design and assembly processes. Second, at the software level, algorithmic calibration techniques are used to calibrate the relative rotation angles of different camera modules, aiming to keep the jumps during image switching within an acceptable range. Third, in the testing stage, the methods for evaluating the installation position of the camera module are relatively traditional, often relying on manual visual inspection or simple testing equipment with limited accuracy. Finally, in the compensation stage, software algorithms are primarily used to adjust the position of the captured images to correct the detected misalignment. However, the above methods result in low accuracy of camera module compensation. Summary of the Invention

[0004] This application provides a camera module compensation method, apparatus, storage medium, and program product to improve the accuracy of camera module compensation.

[0005] Firstly, this application provides a camera module compensation method, applied to a terminal device equipped with a camera module, the camera module comprising multiple cameras, the camera module compensation method including:

[0006] The camera module is triggered to compensate for the rotation, and the rotation compensation angle of the camera module is obtained. The rotation compensation angle is determined based on the image data, which is acquired by the camera module when the terminal device is in a stable posture.

[0007] Rotation compensation is performed on the camera module based on the rotation compensation angle.

[0008] In one possible embodiment, the rotation compensation angle is determined as follows:

[0009] Acquire the attitude data of the terminal device;

[0010] Based on the attitude data, determine whether the terminal device is in a stable attitude;

[0011] If the terminal device is in an unstable posture, adjust the terminal device to a stable posture;

[0012] When the terminal device is in a stable position, the image data captured by the camera module is obtained;

[0013] The rotation compensation angle is determined based on the image data.

[0014] In one possible embodiment, if the terminal device is in an unstable posture, adjusting the terminal device to a stable posture includes:

[0015] If the terminal device is in an unstable posture, calculate the tilt angle of the terminal device based on the three-axis acceleration data of the terminal device;

[0016] Determine if the tilt angle exceeds the tilt threshold;

[0017] If the tilt angle exceeds the tilt threshold, a rotary cylinder control command is generated to adjust the terminal device to a stable posture, which is either a horizontal or vertical posture.

[0018] In one possible embodiment, determining the rotation compensation angle based on image data includes:

[0019] Adaptive binarization is performed on the image data to generate a binary image;

[0020] Linear features are extracted from the binarized image to obtain the rotation compensation angle.

[0021] In one possible embodiment, adaptive binarization processing is performed on the image data to generate a binary image, including:

[0022] Determine whether the threshold parameter of the image data is a negative threshold;

[0023] If the determination result is yes, then for each target pixel in the image data, perform the following adaptive binarization steps:

[0024] Calculate the average gray value of the target pixel's neighborhood within a preset window;

[0025] The absolute value of the threshold parameter is used as the offset;

[0026] The average gray level of the neighborhood and the offset are added together to construct the adaptive binarization threshold corresponding to the target pixel;

[0027] The grayscale values ​​of the target pixels are compared based on the adaptive binarization threshold to generate a binarized image.

[0028] In one possible embodiment, straight line features are extracted from the binarized image to obtain the rotation compensation angle, including...

[0029] Linear features are extracted from the binarized image to obtain multiple detection lines;

[0030] Determine whether the angle between any two detection lines crosses the zero axis boundary;

[0031] If the judgment result is that it crosses the zero axis boundary, the angle is mapped to the continuous range of the target to obtain the target angle;

[0032] Determine the rotation compensation angle based on the target angle.

[0033] In one possible embodiment, determining the rotation compensation angle based on the target angle includes:

[0034] Determine if there is an angle in the target angle that is greater than or equal to an angle threshold;

[0035] If the judgment result indicates that the problem exists, a prompt message will be output, which indicates that there is a non-mechanical tolerance problem in the camera module.

[0036] If the judgment result indicates that the target angle does not exist, determine the number of target angles;

[0037] If the number of target angles is even, then the average of the target angles is determined as the rotation compensation angle;

[0038] If the number of target angles is odd, then the median of the target angles is determined as the rotation compensation angle.

[0039] In one possible embodiment, rotation compensation of the camera module is performed based on the rotation compensation angle, including:

[0040] The rotation compensation angle is invoked, and the real-time image captured by the camera module is reverse-mapped using a rotation matrix to obtain the processed real-time image.

[0041] The final image is obtained by interpolating the processed real-time image using a two-point interpolation method.

[0042] Secondly, this application provides a camera module compensation device, comprising:

[0043] An accelerometer sensor module is used to acquire attitude data of the terminal device;

[0044] A rotary actuator, connected to an acceleration sensor module, is used to adjust the terminal device to a stable posture if the terminal device is in an unstable posture.

[0045] A camera module, deployed in a terminal device, is used to collect image data when the terminal device is in a stable position.

[0046] The compensation parameter determination module is connected to the camera module and is used to determine the rotation compensation angle based on image data.

[0047] The parameter compensation execution module, connected to the compensation parameter determination module, is used to perform rotation compensation on the camera module based on the rotation compensation angle.

[0048] Thirdly, this application provides an electronic device, including: a memory and a processor;

[0049] The memory stores the instructions that the computer executes;

[0050] The processor executes computer execution instructions stored in memory, causing the execution of the first aspect and / or various possible implementations of the first aspect as described above.

[0051] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the first aspect and / or various possible embodiments of the first aspect.

[0052] Fifthly, this application provides a computer program product, including a computer program that, when executed, implements the first aspect and / or various possible implementations of the first aspect.

[0053] This application provides a camera module compensation method, apparatus, storage medium, and program product, relating to the fields of image processing and camera calibration compensation technology. The method is applied to a terminal device with a camera module, which includes multiple cameras. The camera module compensation method includes: responding to a trigger to compensate the camera module; obtaining the rotation compensation angle of the camera module, where the rotation compensation angle is determined based on image data acquired by the camera module when the terminal device is in a stable posture; and performing rotation compensation on the camera module based on the rotation compensation angle. In this application, the terminal device first acquires image data corresponding to the current state of the camera module under stable posture conditions, then determines the angle parameter reflecting the rotation deviation of the camera module from the image data, and uses this angle parameter for rotation correction in the image output link. This reduces the impact of changes in the terminal device's posture and interference factors during image acquisition on the camera module rotation deviation test results, improves the accuracy and stability of the rotation compensation angle, and thus enhances the consistency of image alignment and switching between multiple cameras, reducing image position jumps during camera switching. Attached Figure Description

[0054] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0055] Figure 1 A flowchart illustrating the camera module compensation method provided in this application embodiment. Figure 1 ;

[0056] Figure 2 This is a schematic diagram of the structure of the rotary cylinder provided in the embodiments of this application;

[0057] Figure 3 This is a schematic diagram of different rotation directions of the rotary cylinder provided in the embodiments of this application;

[0058] Figure 4 Example diagrams for acquiring accelerometer data provided in embodiments of this application;

[0059] Figure 5 This is a schematic diagram of the structure of the test equipment provided in the embodiments of this application;

[0060] Figure 6 This is a schematic diagram of a chart provided in an embodiment of this application;

[0061] Figure 7 A schematic diagram of a binarized image provided in an embodiment of this application;

[0062] Figure 8 This is a schematic diagram of a real-time image provided for an embodiment of this application;

[0063] Figure 9 The corrected final image provided for embodiments of this application;

[0064] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0065] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0066] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0067] Camera module compensation technology belongs to the field of mobile terminal image processing and is mainly used in terminal devices such as smartphones and tablets that have multiple cameras. These devices typically use multiple cameras, such as ultra-wide-angle, main camera, and telephoto, to work together to achieve zoom shooting, video recording, and switching between different field of view angles.

[0068] In relevant application scenarios, terminal devices need to quickly switch between multiple cameras while maintaining a relatively consistent image position before and after the switch. To achieve this, the system typically includes a camera module, a terminal image processing link, and a parameter acquisition and retrieval mechanism for compensation control.

[0069] Existing terminal devices typically reduce imaging discrepancies between different cameras through assembly control, factory calibration, and software correction. The basic idea is to align the images captured by the camera modules and call preset compensation parameters during switching to reduce image jumps. Some solutions also combine test images to evaluate the camera module status and then adjust subsequent image output accordingly.

[0070] However, existing solutions focus more on the relative relationships between multiple cameras, lacking sufficient ability to quantify the rotational deviation of individual cameras, resulting in inaccurate compensation criteria. At the same time, the testing process is easily affected by changes in terminal posture, fluctuations in shooting lighting, and errors in real-time image feature extraction, leading to poor stability of the obtained compensation parameters and making it difficult to accurately reflect the true deviation state of the camera module.

[0071] In the above situations, when users record videos, zoom continuously, or switch between multiple cameras, the image may still experience sudden changes in position, subject shift, or discontinuous transitions between front and back views. This not only affects the viewing experience but also reduces the imaging consistency of the multi-camera system, making it difficult to meet the requirements of high-end terminal devices for smooth switching and stable alignment.

[0072] In view of this, how to perform camera module compensation more accurately and stably in terminal devices has become an urgent technical problem to be solved.

[0073] To address the aforementioned issues, this application provides a camera module compensation method. When camera module compensation is triggered, the rotation compensation angle of the camera module is obtained. This rotation compensation angle is determined based on image data collected when the terminal device is in a stable posture. Then, the camera module is rotated and compensated based on the rotation compensation angle, thereby improving the continuity and consistency of the image when switching between multiple cameras.

[0074] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0075] Figure 1 A flowchart illustrating the camera module compensation method provided in this application embodiment. Figure 1 ,like Figure 1 As shown, the method includes:

[0076] S101: Response triggers the compensation camera module, obtains the rotation compensation angle of the camera module. The rotation compensation angle is determined based on image data, which is acquired by the camera module when the terminal device is in a stable posture.

[0077] In this step, the camera module is a multi-camera assembly integrated within the terminal device. This camera module participates in subsequent angle calculation and compensation processing as the compensation object. The terminal device is the shooting device that deploys the aforementioned camera module, and its internal processor executes the compensation process. The multiple cameras enable switching between imaging units, and maintaining consistent image positions before and after switching units necessitates rotational deviation correction for the camera module. The rotational compensation angle is an angular parameter characterizing the rotational deviation of the camera module relative to the terminal device's reference coordinate system. It can be represented by a single angle value indicating clockwise or counterclockwise deflection, or by a signed floating-point angle value indicating both direction and magnitude. The image data consists of image frames acquired by the camera module after the compensation process is triggered. Stable attitude refers to the state where the terminal device's attitude change is within a controlled range during image acquisition.

[0078] In practice, the compensation control module in the terminal device initiates the angle acquisition process after receiving the compensation trigger signal. When the terminal device is in a stable posture, it controls the camera module to acquire image data and determines the rotation compensation angle based on the image data.

[0079] Based on the above analysis, this step determines the rotation compensation angle by limiting it to image data acquired under stable posture, rather than directly using factory-fixed parameters or calculation results from unstable images. This ensures that the angle parameters correspond to the state of the camera module. The angle data obtained in this way reduces errors introduced by handheld shaking, posture shifts, and instantaneous image feature fluctuations. Therefore, it can serve as an effective input for subsequent rotation compensation, making the angle used for correction during multi-camera switching closer to the actual rotation state of the camera module.

[0080] S102: Perform rotation compensation on the camera module based on the rotation compensation angle.

[0081] In this step, rotation compensation refers to correcting the image output relationship of the camera module based on the aforementioned rotation compensation angle to offset the actual rotational deviation of the camera module. The rotation compensation angle is used as a driving parameter in the geometric transformation calculation in this step, and its value and sign directly determine the compensation direction and magnitude.

[0082] In practice, the rotation compensation angle obtained in S101 is first read, and the camera module is rotated and compensated according to this angle to correct the orientation of the image output by the camera module. After the compensation is performed, the terminal device can further output the corrected image frame so that subsequent display or processing is based on the compensation result.

[0083] Based on the above analysis, it can be seen that this step uses orientation correction processing based on rotation compensation angle to perform rotation compensation on the output image of the camera module. The compensation process directly affects the coordinate relationship of the imaging result, thus it can offset the manifestation of the rotation deviation of the camera module in the image.

[0084] In the aforementioned implementation process, by acquiring image data while the terminal device is in a stable posture, the interference of changes in the terminal device's posture (such as hand-held shaking or tilting) on ​​image feature extraction can be effectively eliminated. Specifically, a stable posture ensures that the physical state of the camera module is consistent when acquiring images, avoiding image distortion or feature deviation caused by fluctuations in the terminal device's posture. The rotation compensation angle is calculated based on the image data under this stable state, which can more accurately reflect the actual rotation deviation of the camera module relative to the target reference direction. By applying this rotation compensation angle to the geometric transformation in the image output link, the influence of the module's rotation deviation on the image position can be offset, thereby reducing image jumps and improving image continuity and consistency when switching between multiple cameras.

[0085] In summary, in this embodiment, the terminal device first acquires image data corresponding to the current state of the camera module under stable posture conditions, then determines the angle parameter reflecting the rotation deviation of the camera module from the image data, and uses the angle parameter for rotation correction in the image output link. This can reduce the impact of terminal device posture changes and interference factors during image acquisition on the camera module rotation deviation test results, improve the accuracy and stability of the rotation compensation angle, thereby enhancing the consistency of image alignment and switching between multiple cameras and reducing the phenomenon of image position jumps when switching cameras.

[0086] In one possible implementation, the rotation compensation angle is determined by: acquiring the attitude data of the terminal device; determining whether the terminal device is in a stable attitude based on the attitude data; if the terminal device is in an unstable attitude, adjusting the terminal device to a stable attitude; when the terminal device is in a stable attitude, acquiring real-time images captured by the camera module; and determining the rotation compensation angle based on the real-time images.

[0087] In this implementation, after acquiring the attitude data, the terminal device parses the attitude data and compares it with a preset threshold to determine whether it is in a stable attitude, such as whether it is in a horizontal or vertical attitude. When the determination result is an unstable attitude, the terminal device can output a prompt message and wait for the user to adjust the device to the target attitude, or combine with the attitude adjustment mechanism to complete the attitude correction so that the terminal device enters a stable attitude. After the attitude meets the stability conditions, the camera module acquires image data with the current optical center state. The image data can be a single frame image or a target frame in a series of frames.

[0088] After acquiring image data, the terminal device extracts and matches reference features from the image data. These reference features may include edge contours, straight line structures, corner points, or preset marked areas. Based on the directional deviation, rotation offset, or image coordinate change of the reference features, the terminal device calculates a rotation compensation angle. This rotation compensation angle corresponds to the rotation deviation of the camera module relative to the target reference direction and is output as a control parameter for subsequent camera module rotation compensation.

[0089] The above implementation method acquires real-time images and performs angle calculations under stable posture, so that the rotation compensation angle corresponds to the actual rotation state of the camera module, and is not affected by temporary tilting or shaking of the terminal device, thereby making the compensation parameters more stable and improving the consistency of image position when switching between multiple cameras.

[0090] Based on the aforementioned embodiments, if the terminal device is in an unstable posture, adjusting the terminal device to a stable posture includes: if the terminal device is in an unstable posture, calculating the tilt angle of the terminal device based on the three-axis acceleration data of the terminal device; determining whether the tilt angle exceeds a tilt threshold; if the tilt angle exceeds the tilt threshold, generating a rotary cylinder control command to adjust the terminal device to a stable posture, wherein the stable posture is a horizontal posture or a vertical posture.

[0091] In this specific implementation, the terminal device can be equipped with a three-axis accelerometer. The detection axes of the three-axis accelerometer are set to correspond to the three spatial coordinate axes of the terminal device. After the acceleration values ​​collected by the three-axis accelerometer are read by the attitude calculation module, the tilt angle can be calculated based on the projection relationship of the gravity component on each axis. The tilt angle can be represented by the pitch angle, roll angle, or a combination of both, and the specific expression method can be determined according to the installation method of the terminal device.

[0092] The controller compares the tilt angle with a preset tilt threshold. If the tilt angle exceeds the threshold, it outputs a corresponding rotary cylinder control command to the rotary cylinder drive unit, driving the rotary cylinder to rotate the support mechanism, thus returning the terminal device to a horizontal or vertical orientation. Optionally, the structure of the rotary cylinder can be found in [reference needed]. Figure 2 , Figure 2 The diagram below is a schematic representation of the rotary cylinder provided in an embodiment of this application. It should be noted that this is merely an example. Figure 3 This is a schematic diagram showing different rotation directions of the rotary cylinder provided in the embodiments of this application. Figure 3 A schematic diagram of rotation in three directions is given.

[0093] This application's embodiments calculate the tilt angle based on triaxial acceleration data and determine a tilt threshold. This allows for the timely generation of rotary cylinder control commands when the terminal device's posture deviates significantly, thereby adjusting the terminal device to a stable posture. Consequently, subsequent image data can be acquired under more stable posture conditions, providing a consistent posture basis for determining the camera module's rotation compensation angle.

[0094] Based on the aforementioned embodiments, further, determining the rotation compensation angle based on image data includes: performing adaptive binarization processing on the image data to generate a binarized image; and extracting straight line features from the binarized image to obtain the rotation compensation angle.

[0095] In actual processing, the camera module acquires image data when the terminal device is in a stable posture. After the image data is input to the image processing unit, the adaptive threshold module first performs local grayscale statistics on the image data and generates the corresponding segmentation threshold according to the preset window size. The window size can be set from 3 pixels to 31 pixels to adapt to images with different clarity and noise levels.

[0096] Subsequently, the pixel classification module maps pixels above a local threshold to a first grayscale value and pixels below the threshold to a second grayscale value, forming a binarized image. Then, the line feature extraction module performs edge connectivity analysis and line fitting on the binarized image to identify the principal line corresponding to the camera module's pose deviation, and calculates the rotation compensation angle based on the angle between the principal line and the horizontal reference axis. When multiple candidate lines exist in the binarized image, the target line can be determined based on line length, connectivity, and pixel density to improve the consistency of angle calculation.

[0097] By adopting this method, the line features in the image data can maintain high recognizability under changes in lighting and background interference, and the calculation results of the rotation compensation angle are closer to the actual rotation deviation of the camera module, thus improving the stability and consistency of the compensation parameters.

[0098] Based on the aforementioned embodiments, further, adaptive binarization processing is performed on the real-time image to generate a binarized image, including: determining whether the threshold parameter of the image data is a negative threshold; if the determination result is yes, then for each target pixel in the image data, the following adaptive binarization processing steps are performed: calculating the mean gray value of the neighborhood of the target pixel within a preset window; determining the absolute value of the threshold parameter as the offset; adding the mean gray value of the neighborhood and the offset to construct the adaptive binarization threshold corresponding to the target pixel; comparing the gray values ​​of the target pixels according to the adaptive binarization threshold to generate a binarized image.

[0099] In the specific implementation, the image data output by the camera module is first received, and the corresponding threshold parameter is read from it. When the threshold parameter is detected to be negative, the image processing module processes each target pixel in the image data one by one, constructing a preset window centered on the target pixel. The window size can be set according to the image resolution and texture density, for example, a square area of ​​several pixels wide can be taken to cover the local brightness changes near the target pixel. Subsequently, the gray values ​​of all neighboring pixels in the window are counted and the average is calculated. Then, the absolute value of the threshold parameter is taken to obtain the offset, and this offset is superimposed on the average gray value of the neighboring pixels to form the threshold corresponding to the current target pixel. Then, the threshold is compared with the gray value of the target pixel. If the gray value of the target pixel meets the preset judgment condition, the first binary result is output; otherwise, the second binary result is output, thus forming a binary image.

[0100] The above processing method uses the average gray level of the local neighborhood as the basic threshold and adjusts it through an offset, so that the threshold parameter can be updated as the local brightness of the image data changes. As a result, the binarization result can better preserve the line and edge information in the image data and suppress local noise and brightness fluctuations, thus providing a more stable binarized image input for subsequent line feature extraction.

[0101] In summary, the adaptive binarization process described above constructs a dynamically adjusted binarization threshold by calculating the average grayscale value of the target pixel's neighborhood within a preset window and combining it with the absolute value of the threshold parameter as an offset. This threshold can be automatically adjusted based on local brightness changes in the image, avoiding the loss of line features or noise enhancement caused by a fixed threshold under complex lighting conditions. For example, in low-light environments, the threshold is increased by raising the offset to preserve lines in dark areas; in bright light environments, background noise is suppressed by decreasing the offset. This dynamic adjustment mechanism improves the recognizability of line features in the binarized image, providing a more stable input for subsequent line detection, thereby enhancing the accuracy of rotation compensation angle calculation.

[0102] After adopting the above embodiments, the threshold construction in the image data no longer depends on a fixed constant, but is dynamically adjusted in combination with the gray-level distribution within the local window. This can improve the adaptability of the binarized image to local features and enhance the stability and consistency of subsequent rotation compensation angle extraction.

[0103] Based on the aforementioned embodiments, further, straight line features are extracted from the binarized image to obtain the rotation compensation angle, including extracting straight line features from the binarized image to obtain multiple detection lines; determining whether the angle constructed by any two detection lines crosses the zero axis boundary; if the determination result is that it crosses the zero axis boundary, mapping the angle to the target continuous interval range to obtain the target angle; and determining the rotation compensation angle based on the target angle.

[0104] In practical implementation, after completing the adaptive binarization processing of the image data, the terminal device inputs the binarized image into the line feature extraction module. The line feature extraction module can use Hough transform or a detection method based on line segment fitting to identify a set of pixels with continuous and approximately collinear edges from the image and output multiple detection lines. Each detection line carries its direction and angle parameters. The angle parameters can be established relative to the horizontal axis of the image coordinate system, and the range can be set to an angle expression method with the zero axis boundary as the dividing line.

[0105] After acquiring multiple detection lines, the angle calculation unit pairs and compares any two detection lines, calculates the included angle between them based on their respective angle parameters, and determines whether the included angle crosses the zero axis boundary. If the two detection lines are located on opposite sides of the zero axis boundary, the corresponding included angle may change from a positive value to a negative value or vice versa. In this case, the included angle is mapped to a continuous target interval. This continuous target interval can be set as a continuous angle expression within the same sign interval, so that angles that originally crossed the boundary are converted into continuously comparable target angles.

[0106] After obtaining the target angles, the angle aggregation module can calculate a representative angle value based on all target angles. The representative angle value can be the average, median, or weighted average of multiple target angles, and this representative angle value is output as the rotation compensation angle. This rotation compensation angle is used to characterize the rotational deviation of the camera module relative to the stable posture and is sent to the subsequent compensation control link to perform rotation compensation.

[0107] This embodiment maps angles across the zero axis boundary continuously, ensuring that the angle representation remains continuous near the boundary. This makes the rotation compensation angles calculated by multiple detection lines more stable and facilitates consistent calculation and calling in subsequent compensation processing, thereby improving the accuracy and stability of the camera module rotation compensation results.

[0108] Based on the aforementioned embodiments, further, determining the rotation compensation angle according to the target angle includes: determining whether there is an angle greater than or equal to an angle threshold among the target angles; if the determination result indicates that there is such an angle, then outputting a prompt message, which is used to indicate that there is a non-mechanical tolerance problem in the camera module; if the determination result indicates that there is no such angle, determining the number of target angles; if the number of target angles is even, then determining the average of the target angles as the rotation compensation angle; if the number of target angles is odd, then determining the median of the target angles as the rotation compensation angle.

[0109] In practice, after generating the target angles, the terminal device compares each target angle with a preset angle threshold. When any target angle is greater than or equal to the threshold, the control compensation module stops directly calculating the rotation compensation angle using that set of target angles. Instead, it outputs a prompt message corresponding to the abnormal state to indicate that the current camera module may have a non-mechanical tolerance problem. The prompt message can be recorded in the device operation information or displayed in the maintenance interface for easy troubleshooting in conjunction with the camera module calibration results.

[0110] If no angle that meets the angle threshold condition appears in the target angles, the number of target angles is determined. If the number of target angles is even, the average of the target angles is determined as the rotation compensation angle. If the number of target angles is odd, all target angles are sorted and the median is extracted from the sorting results as the rotation compensation angle. This median can suppress local abnormal fluctuations and keep the compensation results stable.

[0111] In this embodiment, the angle threshold is used to filter out angle samples that exceed the normal range, the prompt information is used to promptly identify abnormal states of the camera module, and the median is used to output a stable rotation compensation angle when all samples are within the normal range. Therefore, the compensation calculation can both provide prompts for abnormal camera module states and obtain compensation results resistant to outlier interference under normal conditions, thereby improving the reliability and consistency of the rotation compensation angle and enhancing the image alignment effect during subsequent camera switching.

[0112] In one possible implementation, rotation compensation is performed on the camera module based on the rotation compensation angle, including: calling the rotation compensation angle, performing reverse mapping processing on the real-time image acquired by the camera module through the rotation matrix to obtain the processed real-time image; and performing interpolation processing on the processed real-time image through the two-point interpolation method to obtain the final image.

[0113] In practical implementation, after obtaining the rotation compensation angle, the terminal device converts the rotation compensation angle into the corresponding rotation parameters, and establishes a rotation matrix based on the image center. For example, the rotation matrix can be expressed as the following formula:

[0114] .

[0115] in, Indicates the rotation compensation angle; (c) x ,c y () indicates the image center of the real-time image.

[0116] During the mapping process, each target pixel in the real-time image is first mapped by calculating its corresponding floating-point coordinates in the original sampling plane based on the rotation matrix, and then the floating-point coordinates are written into the processed real-time image as the result of the reverse mapping process.

[0117] Because rotation transformations cause some pixel coordinates to fall into non-integer positions, the processed real-time image still contains gaps or areas with insufficient grayscale transitions. Subsequently, a bilinear two-point interpolation method is used to supplement the pixel values ​​between two adjacent sampling points, and the interpolation weights are determined based on the distance relationship between the two sampling points to generate the final image. This two-point interpolation method has low computational complexity, making it suitable for continuous execution in real-time output scenarios from camera modules, balancing speed and image quality requirements.

[0118] The above processing allows the rotation compensation angle to directly affect the image space transformation. First, coordinate mapping is completed through a rotation matrix, and then the resampled pixel values ​​are completed through a two-point interpolation method. This ensures that the compensated image maintains continuous geometric relationships and smoother edge representation. As a result, a stable and consistent final image can be output even when there is rotational deviation in the camera module, and jagged edges and local distortions generated during rotation correction are reduced.

[0119] Furthermore, the following example uses a specific application scenario to explain how to utilize the camera module compensation method provided in this application embodiment.

[0120] The specific application scenario is as follows: In mobile terminal devices equipped with multi-camera systems, users achieve functions such as optical zoom and wide-angle shooting by switching between different cameras (e.g., ultra-wide-angle -> main camera -> telephoto periscope). However, due to assembly tolerances of the camera modules, image shifts and jumps occur, affecting the continuity of shooting. For example, when recording video, switching cameras may cause the center point of the image to shift suddenly; during optical zoom, the edges of the image may be misaligned due to rotation errors. This application embodiment ensures "seamless connection" of the image when switching between multiple cameras through a systematic testing and compensation scheme, improving the user's shooting experience.

[0121] In light of the above application scenarios, the solution of this application embodiment will be explained in detail. The solution includes the following steps:

[0122] 1. Mobile terminal device attitude calibration: The mobile terminal device obtains accelerometer data through the built-in three-axis accelerometer to calculate the tilt angle in order to monitor the current attitude (such as horizontal, vertical or tilted) in real time. Figure 4 This is an example diagram illustrating the acquisition of accelerometer data provided in an embodiment of this application. Figure 4 This diagram illustrates how accelerometer data is acquired when a mobile device, such as a smartphone, is in landscape mode. Figure 4 Based on this, the tilt angle of the phone is calculated in the following way: Furthermore, assuming the phone is in portrait mode and rotated 90° clockwise, the phone's tilt angle is calculated as follows: When the phone is in portrait mode and rotated 90° counterclockwise, the tilt angle is calculated as follows: Furthermore, ).in, "Y" indicates deviation; "roll" refers to the rotation of an object around its y-axis; "pitch" refers to the pitch of an object around its x-axis.

[0123] If the tilt angle of the mobile terminal device exceeds a preset threshold (e.g., 0.1°), the rotary cylinder of the testing equipment will automatically adjust the clamp angle to restore the mobile terminal device to a horizontal or vertical position. A schematic diagram of the testing equipment can be found here. Figure 5 , Figure 5 This is a schematic diagram of the structure of the testing equipment provided in the embodiments of this application. Figure 5 The diagram shows the test equipment in different directions.

[0124] This step ensures a stable acquisition environment for subsequent test images, avoiding angle calculation errors caused by the posture errors of mobile terminal devices.

[0125] 2. Adaptive Binarization Processing: Chart images captured by mobile terminal devices, Figure 6 This is a schematic diagram of a chart provided in an embodiment of this application. Figure 6 As can be seen, the chart contains straight lines or black and white edges. Next, an adaptive binarization algorithm is used to process the chart. This algorithm first determines if the input threshold of the chart is negative. If it is negative, the mean value of the chart is used as a benchmark, and the mean plus |threshold| is taken as the binarization threshold. For example, in low-light environments, the grayscale distribution of the chart is darker, and the adaptive binarization algorithm automatically increases the threshold to preserve line features; in bright light environments, the threshold is decreased to suppress background noise. The processed image generates a high-contrast binarized image, providing clear input for subsequent line detection.

[0126] 3. Cross-zero axis angle remapping: The HoughLines algorithm is applied to extract multiple detection lines from the processed binarized image and calculate their polar angles. Figure 7 This is a schematic diagram of a binarized image provided in an embodiment of this application. Figure 7 Multiple detection lines can be seen in the data.

[0127] If the polar angles of any two detection lines cross the 0° / 180° boundary (e.g., one line is 175° and the other is 5°), the HoughLines algorithm determines the boundary crossing by changing the sign of the sine value and maps the polar angles to a continuous interval (e.g., a continuous interval ∈ (-π / 2, 3π / 2)) to obtain the target angle. For example, the polar angles that originally crossed the boundary, 5° and 175°, will be mapped to 5° and 365°, ensuring the accuracy of subsequent median calculations.

[0128] This step addresses the periodic boundary problem of polar angles, avoiding rotation angle errors caused by discretization.

[0129] 4. Cross-Angle Verification: The target angle is compared with an angle threshold. If any target angle is greater than or equal to the threshold, the calculation of the rotation compensation angle using that set of target angles is stopped. Instead, a prompt message corresponding to the abnormal state is output to indicate that the current camera module may have a non-mechanical tolerance problem. If no angle meets the angle threshold condition among the target angles, the number of target angles is determined. If the number of target angles is odd, all target angles are sorted, and the median is extracted from the sorting results as the rotation compensation angle. If the number of target angles is even, the average value of all target angles is determined as the rotation compensation angle. Furthermore, by analyzing the cosine (or sine) sign relationship between the rotation compensation angle and the original angle, it can be determined whether the polar radius of the rotation compensation angle needs to be inverted, thereby ensuring the consistency of the polar coordinate representation.

[0130] 5. Compensation Parameter Storage and Application: The calculated rotation angle data is stored in a designated memory address (e.g., MiscTA) on the mobile terminal device. When the user activates the mobile terminal device's camera, the device uses this rotation angle data to perform reverse mapping processing on the captured real-time image using a rotation matrix. Furthermore, it combines a two-point interpolation method to optimize edge pixels, ensuring a smooth, jagged-free image after compensation. The final output image achieves seamless transitions during multi-camera switching, eliminating offset jumps. Figure 8 This is a schematic diagram of a real-time image provided in an embodiment of this application. Figure 9 The corrected final image provided for embodiments of this application. From Figure 8 It can be seen that the real-time image has a significant tilt angle, which is greater than 2°; from Figure 9 As can be seen, the final image after correction does not have a significant tilt angle; the tilt angle is less than 0.2°.

[0131] Through the above steps, during the user's video recording process, when switching cameras, posture calibration ensures the stability of the test environment and avoids image feature extraction errors caused by the tilt of the mobile terminal device; adaptive binarization preserves the line features of the chart and improves the accuracy of polar angle calculation; cross-zero axis angle remapping eliminates the polar angle discretization problem and ensures the continuity of rotation angle data; dynamic compensation combines inverse rotation matrix and interpolation optimization to correct image offset in real time.

[0132] Ultimately, when users switch cameras in a video, the center point of the image remains consistent with no abrupt changes at the edges, significantly improving the shooting experience.

[0133] Furthermore, this application embodiment also provides a camera module compensation device, including: an acceleration sensor module for acquiring attitude data of a terminal device; a rotation actuator connected to the acceleration sensor module for adjusting the terminal device to a stable attitude if the terminal device is in an unstable attitude; a camera module deployed in the terminal device for acquiring image data when the terminal device is in a stable attitude; a compensation parameter determination module connected to the camera module for determining a rotation compensation angle based on the image data; and a parameter compensation execution module connected to the compensation parameter determination module for performing rotation compensation on the camera module based on the rotation compensation angle.

[0134] The camera module compensation device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0135] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, a processing module can be a separate processing element, or it can be integrated into a chip within the device. Alternatively, it can be stored as program code in the device's memory, and its functions can be called and executed by a processing element. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0136] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). As another example, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a System-On-a-Chip (SOC).

[0137] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 10 As shown, the electronic device 1000 provided in this application embodiment may include: a processor 1001, and a memory 1002 communicatively connected to the processor, wherein:

[0138] The memory stores the instructions that the computer executes;

[0139] The processor executes computer execution instructions stored in memory to implement the method described in the foregoing method embodiments.

[0140] It should be understood that processor 1001 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor. Memory 1002 may include high-speed random access memory (RAM), and may also include non-volatile memory (NVM), such as at least one disk storage device, or a USB flash drive, external hard drive, read-only memory, disk, or optical disc, etc.

[0141] Optionally, the electronic device 1000 may also include a communication interface 1003. In specific implementations, if the communication interface 1003, memory 1002, and processor 1001 are implemented independently, they can be interconnected via a bus to complete communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc., but this does not imply that there is only one bus or one type of bus.

[0142] Optionally, in a specific implementation, if the communication interface 1003, memory 1002 and processor 1001 are integrated on a single chip, then the communication interface 1003, memory 1002 and processor 1001 can communicate through an internal interface.

[0143] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the methods described in any of the foregoing embodiments.

[0144] It is understood that the computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0145] An exemplary computer-readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the computer-readable storage medium. Of course, the computer-readable storage medium can also be a component of the processor. The processor and the computer-readable storage medium can reside in an ASIC. Alternatively, the processor and the computer-readable storage medium can exist as discrete components in an electronic device.

[0146] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a computer-readable storage medium, include several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application.

[0147] This application also provides a computer program product, including a computer program that, when executed, implements the method described in any of the foregoing embodiments.

[0148] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0149] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0150] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0151] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0152] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A camera module compensation method, characterized in that, The method for compensating a camera module is applied to a terminal device equipped with a camera module, the camera module comprising multiple cameras, and includes the following: The camera module is compensated in response to the trigger, and the rotation compensation angle of the camera module is obtained. The rotation compensation angle is determined based on image data, which is acquired by the camera module when the terminal device is in a stable posture. Based on the aforementioned rotation compensation angle, rotation compensation is performed on the camera module.

2. The method according to claim 1, characterized in that, The rotation compensation angle is determined in the following way: Obtain the attitude data of the terminal device; Based on the attitude data, determine whether the terminal device is in a stable attitude; If the terminal device is in an unstable posture, adjust the terminal device to a stable posture; When the terminal device is in a stable posture, the image data captured by the camera module is acquired; Based on the image data, the rotation compensation angle is determined.

3. The method according to claim 2, characterized in that, If the terminal device is in an unstable posture, adjusting the terminal device to a stable posture includes: If the terminal device is in an unstable posture, calculate the tilt angle of the terminal device based on the three-axis acceleration data of the terminal device; Determine whether the tilt angle exceeds the tilt threshold; If the tilt angle exceeds the tilt threshold, a rotary cylinder control command is generated to adjust the terminal device to the stable posture, wherein the stable posture is a horizontal posture or a vertical posture.

4. The method according to claim 2, characterized in that, Determining the rotation compensation angle based on the image data includes: The image data is subjected to adaptive binarization processing to generate a binary image; The rotation compensation angle is obtained by extracting straight line features from the binarized image.

5. The method according to claim 4, characterized in that, The image data is subjected to adaptive binarization processing to generate a binary image, including: Determine whether the threshold parameter of the image data is a negative threshold; If the determination result is yes, then for each target pixel in the image data, the following adaptive binarization steps are performed: Calculate the average gray value of the neighborhood of the target pixel within the preset window; The absolute value of the threshold parameter is determined as the offset; The average gray value of the neighborhood and the offset are added together to construct the adaptive binarization threshold corresponding to the target pixel; The grayscale values ​​of the target pixels are compared according to the adaptive binarization threshold to generate the binarized image.

6. The method according to claim 4, characterized in that, Linear feature extraction is performed on the binarized image to obtain the rotation compensation angle, including... Linear feature extraction is performed on the binarized image to obtain multiple detection lines; Determine whether the angle between any two detection lines crosses the zero axis boundary; If the judgment result is that the angle crosses the zero axis boundary, the angle is mapped to the target continuous interval range to obtain the target angle; The rotation compensation angle is determined based on the target angle.

7. The method according to claim 6, characterized in that, Determining the rotation compensation angle based on the target angle includes: Determine whether there is an angle greater than or equal to an angle threshold among the target angles; If the judgment result indicates that the problem exists, a prompt message is output, which is used to indicate that the camera module has a non-mechanical tolerance problem. If the judgment result indicates that the target angle does not exist, determine the number of target angles; If the number of target angles is even, then the average of the target angles is determined as the rotation compensation angle; If the number of target angles is odd, then the median of the target angles is determined as the rotation compensation angle.

8. The method according to any one of claims 1 to 7, characterized in that, The rotation compensation of the camera module based on the rotation compensation angle includes: The rotation compensation angle is invoked, and the real-time image captured by the camera module is reverse-mapped using a rotation matrix to obtain the processed real-time image. The processed real-time image is interpolated using a two-point interpolation method to obtain the final image.

9. A camera module compensation device, characterized in that, include: An accelerometer sensor module is used to acquire attitude data from the terminal device. A rotary actuator, connected to the acceleration sensor module, is used to adjust the terminal device to a stable posture if the terminal device is in an unstable posture. A camera module, deployed in the terminal device, is used to acquire image data when the terminal device is in a stable posture; The compensation parameter determination module is connected to the camera module and is used to determine the rotation compensation angle based on the image data. The parameter compensation execution module is connected to the compensation parameter determination module and is used to perform rotation compensation on the camera module based on the rotation compensation angle.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-8.

Images