Calibration method and device, electronic equipment, storage medium and program product

Abstract

The application discloses a calibration method, device, electronic equipment, storage medium and program product. The calibration method comprises the following steps: acquiring a plurality of groups of synchronously collected double-vision images to be processed and vehicle motion information, wherein the double-vision images to be processed are collected by a double-vision camera carried by a vehicle; filtering the double-vision images to be processed according to the vehicle motion information to obtain a plurality of groups of target double-vision images; performing feature point detection and matching on each group of target double-vision images, determining at least one pair of matched feature points included in each group of target double-vision images, and calculating a processing row difference of each pair of matched feature points; performing statistical analysis based on a plurality of processing row differences to obtain at least one statistical quantity, and performing parameter calibration on the double-vision camera according to the statistical quantity. The application can reduce the calculation burden caused by the filtered invalid double-vision images to be processed, reduce the error caused by the vehicle motion, improve the robustness of the double-vision system, and enable the double-vision system to accurately perform row difference calculation in motion.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a calibration method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] In computer vision and robotics, binocular vision systems capture images of the same object using two independent cameras. They then compare the pixel differences between the two images—the binocular disparity—to derive depth information, enabling depth perception and 3D reconstruction. Therefore, the accuracy and reliability of binocular camera systems heavily rely on precise binocular disparity calibration.

[0003] Among them, the acquisition and matching of feature points are the key steps in calculating the binocular line difference. However, in practical applications, when the binocular vision system is applied to a moving vehicle, the dynamic movement of the vehicle will change the relative position and angle between the camera and the objects in the scene. This causes feature points that are clear and stable under static conditions to become blurry, unstable or difficult to detect under dynamic conditions. If the feature points are inaccurate or mismatched, the calculated line difference will also be incorrect, which in turn reduces the accuracy and reliability of the binocular vision system. Summary of the Invention

[0004] This application provides a calibration method, apparatus, electronic device, storage medium, and program product to at least solve the problem in the related art where the dynamic movement of a vehicle causes inaccurate feature points or matching errors, thereby reducing the accuracy and reliability of the binocular vision system. The technical solution of this application is as follows: According to a first aspect of the embodiments of this application, a calibration method is provided, comprising: Multiple sets of synchronously acquired binocular images to be processed and vehicle motion information are acquired, wherein the binocular images to be processed are acquired by the binocular camera mounted on the vehicle; Based on the vehicle motion information, the binocular images to be processed are filtered to obtain multiple sets of target binocular images; For each group of target binocular images, feature point detection and matching are performed to determine at least one pair of matching feature points included in each group of target binocular images, and the unprocessed row difference of each pair of matching feature points is calculated. Statistical analysis is performed on multiple line differences to be processed to obtain at least one statistic, and the parameters of the binocular camera are calibrated based on the statistic.

[0005] Optionally, acquiring multiple sets of synchronously acquired binocular images to be processed and vehicle motion information includes: Based on timestamp synchronization technology, the binocular cameras are synchronously triggered to acquire the left and right images to be processed, and to acquire vehicle motion information. Distortion removal processing is performed on the left image and the right image to be processed to obtain a set of binocular images to be processed.

[0006] Optionally, the vehicle motion information is the motion change information between the current time and the previous time. The step of filtering the binocular image to be processed based on the vehicle motion information to obtain multiple sets of target binocular images includes: For the Nth group of binocular images to be processed, determine whether the motion change information at the same moment is greater than a preset change threshold, where N is a positive integer greater than 1; If the value is greater than the target value, the Nth group of stereo images to be processed is used as the target stereo image; if the value is not greater than the target value, the Nth group of stereo images to be processed is filtered out.

[0007] Optionally, each set of target binocular images includes a left target image and a right target image; the step of performing feature point detection and matching on each set of target binocular images to determine at least one pair of matching feature points included in each set of target binocular images includes: For each set of target binocular images, a pre-set feature point detection algorithm is used to detect feature points in the left image and the right image of the target respectively, so as to obtain multiple first feature points in the left image of the target and multiple second feature points in the right image of the target. Using a pre-defined image feature matching algorithm, feature point matching is performed on the left and right images of the target, and a first feature point and a second feature point that are successfully matched are taken as a pair of matched feature points.

[0008] Optionally, the statistical analysis based on multiple row differences to be processed to obtain at least one statistic includes: Calculate the mean and / or standard deviation of the multiple rows of differences to be processed as statistics.

[0009] Optionally, the step of calibrating the parameters of the binocular camera based on the statistics includes: If the statistical quantity does not meet the preset qualification conditions, the parameters of the binocular camera are calibrated, and the process of acquiring multiple sets of synchronously acquired binocular images and vehicle motion information is returned. If the statistical quantity meets the preset qualification conditions, the parameter calibration of the binocular camera is deemed qualified. The preset qualification conditions include at least one of the following: the average value is less than a preset average value threshold; the standard deviation is less than a preset standard deviation threshold.

[0010] According to a second aspect of the embodiments of this application, a calibration apparatus is provided, comprising: The acquisition module is used to acquire multiple sets of synchronously acquired binocular images to be processed and vehicle motion information. The binocular images to be processed are acquired by the binocular camera mounted on the vehicle. The filtering module is used to filter the binocular image to be processed based on the vehicle motion information to obtain multiple sets of target binocular images; The matching module is used to detect and match feature points for each group of target binocular images, determine at least one pair of matching feature points included in each group of target binocular images, and calculate the unprocessed row difference for each pair of matching feature points. The analysis module is used to perform statistical analysis based on multiple line differences to be processed, obtain at least one statistical quantity, and perform parameter calibration on the binocular camera according to the statistical quantity.

[0011] Optionally, the acquisition module is specifically used for: Based on timestamp synchronization technology, the binocular cameras are synchronously triggered to acquire the left and right images to be processed, and to acquire vehicle motion information. Distortion removal processing is performed on the left image and the right image to be processed to obtain a set of binocular images to be processed.

[0012] Optionally, the vehicle motion information is the motion change information between the current moment and the previous moment, and the filtering module is specifically used for: For the Nth group of binocular images to be processed, determine whether the motion change information at the same moment is greater than a preset change threshold, where N is a positive integer greater than 1; If the value is greater than the target value, the Nth group of stereo images to be processed is used as the target stereo image; if the value is not greater than the target value, the Nth group of stereo images to be processed is filtered out.

[0013] Optionally, each set of target binocular images includes a left target image and a right target image; the matching module is specifically used for: For each set of target binocular images, a pre-set feature point detection algorithm is used to detect feature points in the left image and the right image of the target respectively, so as to obtain multiple first feature points in the left image of the target and multiple second feature points in the right image of the target. Using a pre-defined image feature matching algorithm, feature point matching is performed on the left and right images of the target, and a first feature point and a second feature point that are successfully matched are taken as a pair of matched feature points.

[0014] Optionally, the analysis module includes: The calculation unit is used to calculate the average and / or standard deviation of the multiple row differences to be processed as statistics.

[0015] Optionally, the analysis module includes: A calibration unit is used to calibrate the parameters of the binocular camera when the statistical quantity does not meet the preset qualification conditions, and return to the step of acquiring multiple sets of synchronously acquired binocular images to be processed and vehicle motion information; when the statistical quantity meets the preset qualification conditions, the parameter calibration of the binocular camera is determined to be qualified; wherein, the preset qualification conditions include at least one of the following: the average value is less than a preset average value threshold; the standard deviation is less than a preset standard deviation threshold.

[0016] According to a third aspect of the embodiments of this application, an electronic device is provided, the electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the calibration method as described in any of the preceding claims.

[0017] According to a fourth aspect of the embodiments of this application, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the calibration method as described in any of the preceding claims.

[0018] According to a fifth aspect of the embodiments of this application, a computer program product is provided, wherein a program or instructions are stored on the computer program product, and the program or instructions, when executed by a processor, implement the calibration method as described in any of the preceding claims.

[0019] The technical solutions provided by the embodiments of this application have at least the following beneficial effects: By acquiring vehicle motion information synchronized with the binocular images to be processed, the calibration method not only relies on the binocular images to be processed but also incorporates real-time vehicle motion information to implement a filtering mechanism for the binocular images to be processed. Only when the synchronously acquired vehicle motion information meets certain conditions will the binocular image to be processed be considered as the target binocular image, thereby triggering the row difference calculation of the target binocular image. The filtering mechanism ensures that the binocular vision system can efficiently process valid target binocular images, reducing the computational burden caused by filtered invalid binocular images to be processed. Moreover, filtering based on real-time vehicle motion information helps reduce errors caused by vehicle motion. Performing row difference calculation based on multiple target binocular images obtained from filtering can improve the robustness of the binocular vision system, enabling the binocular vision system to perform row difference calculation more accurately during motion.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application, and do not constitute an undue limitation of this application.

[0022] Figure 1 This is a flowchart illustrating a calibration method according to an exemplary embodiment.

[0023] Figure 2 This is a logical schematic diagram illustrating a calibration method according to an exemplary embodiment.

[0024] Figure 3 This is a block diagram illustrating a calibration device according to an exemplary embodiment.

[0025] Figure 4 This is a block diagram illustrating an electronic device according to an exemplary embodiment.

[0026] Figure 5 This is a block diagram illustrating a readable storage medium according to an exemplary embodiment. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0028] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0029] Figure 1 This is a flowchart illustrating a calibration method according to an exemplary embodiment, such as... Figure 1 As shown, the calibration method includes: In step S11, multiple sets of synchronously acquired binocular images to be processed and vehicle motion information are obtained. The binocular images to be processed are acquired by the binocular camera mounted on the vehicle.

[0030] A binocular camera typically consists of two cameras positioned side-by-side, sharing the same field of view and focal length but slightly offset to mimic human binocular vision. These two cameras can simultaneously capture two images of the same scene; due to the positional difference between the cameras, objects will appear slightly different in the two images. By comparing the positional differences (parallax) of identical feature points in the two images, the distance between the object and the camera can be calculated.

[0031] This step typically involves acquiring multiple sets of simultaneously collected binocular images and vehicle motion information for subsequent data analysis and processing. This has broad application prospects in fields such as autonomous driving and intelligent transportation. For example, in autonomous driving systems, this information can be used for tasks such as vehicle localization, obstacle detection, and path planning.

[0032] The acquisition of binocular images and vehicle motion information can be achieved by continuously triggering the binocular cameras to acquire and record vehicle motion information. Vehicle motion information may include positioning information (x, y, z) and angular information (roll, pitch, yaw). This information describes the vehicle's position and orientation in space and can be obtained through sensors installed on the vehicle, such as GPS (Global Positioning System), accelerometers, and gyroscopes.

[0033] In one implementation, acquiring multiple sets of synchronously acquired binocular images to be processed and vehicle motion information includes: Based on timestamp synchronization technology, the binocular cameras are simultaneously triggered to acquire the left and right images to be processed, and vehicle motion information is also acquired. Distortion removal processing is performed on the left image to be processed and the right image to be processed to obtain a set of binocular images to be processed.

[0034] In other words, the stereo cameras should trigger exposures synchronously to ensure that the left and right images to be processed are captured at the same time, avoiding motion blur caused by time delay. Synchronous triggering is typically achieved by sending a synchronization trigger signal to the stereo cameras; this signal can be an electrical or optical signal. When the stereo cameras receive this signal, they will simultaneously begin acquiring images.

[0035] In practical applications, binocular cameras and vehicle motion information sensors (such as GPS, accelerometers, etc.) may be controlled by different hardware systems, so their sampling times and frequencies may not be completely consistent. To ensure that the acquired image information and motion information are synchronized, timestamp synchronization technology is required.

[0036] Timestamp synchronization is typically achieved by adding a timestamp to each sampling point. This timestamp records the specific time the sampling point occurred. In subsequent data processing, the timestamps can be used to match and synchronize data from different sources.

[0037] Furthermore, due to the distortion of the camera lens, the left and right images to be processed need to undergo distortion correction in order to obtain more accurate feature points and line difference calculations. Common distortion correction methods include using camera intrinsic parameters for correction, or it can be achieved through specialized image processing software or algorithms.

[0038] In step S12, the binocular images to be processed are filtered according to the vehicle motion information to obtain multiple sets of target binocular images.

[0039] After acquiring vehicle motion information and the stereo images to be processed, image filtering can begin. The purpose of this step is to select multiple sets of target stereo images with sufficient motion information from a large number of stereo images to be processed, so as to perform subsequent stereo matching and depth calculation.

[0040] Specifically, filtering conditions can be set based on certain key parameters in the vehicle motion information (such as changes in speed, etc.). For example, a change threshold can be set; only when the change in vehicle information between two consecutive time points exceeds this threshold is the currently acquired binocular image considered valid. Similarly, a speed change threshold or acceleration threshold can also be set for filtering. This method effectively reduces the number of binocular images to be processed, improving the efficiency and accuracy of subsequent processing.

[0041] In one implementation, the vehicle motion information is the motion change information between the current time and the previous time. Based on the vehicle motion information, the binocular images to be processed are filtered to obtain multiple sets of target binocular images, including: For the Nth group of binocular images to be processed, determine whether the motion change information at the same moment is greater than a preset change threshold, where N is a positive integer greater than 1; If the result is greater than the target image, the Nth group of stereo images to be processed will be used as the target stereo image; otherwise, the Nth group of stereo images to be processed will be filtered out.

[0042] In other words, firstly, it is necessary to obtain the vehicle motion information corresponding to the Nth group of binocular images to be processed at the same moment. This vehicle motion information can be the motion change information between the current moment and the previous moment. The current moment is the moment when the current group of binocular images to be processed was acquired, while the previous moment is the moment before the acquisition of the current group of binocular images to be processed. For example, the motion change information can include changes in the distance and / or angle of the vehicle between the current moment and the previous moment.

[0043] Next, the calculated motion change information is compared with a pre-set change threshold. If the motion displacement is greater than the change threshold, then the Nth group of binocular images to be processed is considered to have been acquired when the vehicle underwent a sufficiently large displacement. Therefore, the Nth group of binocular images to be processed contains enough stereo information to be used in the subsequent calibration process and can be retained as the target binocular images.

[0044] If the motion change information is not greater than the change threshold, it indicates that the motion change of the vehicle during the acquisition of the Nth group of binocular images to be processed is not significant enough. This may lead to insufficient stereo information or difficulty in accurate matching between the Nth group of binocular images to be processed. Therefore, the Nth group of binocular images to be processed is filtered out and not used in the subsequent calibration process.

[0045] For example, the threshold for change can be set to a distance change of 2 meters or an angle change of 30 degrees. Then, when the distance change between the current moment and the previous moment is greater than 2 meters or the angle change is greater than 30 degrees, the binocular image to be processed can be used as the target binocular image and become effective line difference calculation data.

[0046] In this way, target binocular images with sufficient motion information can be selected from a large number of binocular images to be processed, improving the efficiency and accuracy of subsequent feature point detection and matching, and reducing the computational burden caused by invalid data.

[0047] In step S13, feature point detection and matching are performed on each group of target binocular images to determine at least one pair of matching feature points included in each group of target binocular images, and the unprocessed row difference of each pair of matching feature points is calculated.

[0048] Feature point detection refers to the process of detecting representative and easily identifiable points or regions in a target binocular image, while feature point matching refers to the process of establishing the connection between each set of target binocular images through the correspondence of these feature points.

[0049] After accumulating feature points from multiple sets of effective target binocular images, feature point matching is completed. Then, for each pair of matched feature points, the vertical coordinate difference between the corresponding features in the left and right images is calculated to obtain the unprocessed row difference, which can reflect the depth difference of the object in the left and right views.

[0050] Pending row difference The calculation formula can be expressed as:

[0051] in, It is the vertical coordinate of the i-th matching point in the left figure. These are the vertical coordinates of the corresponding matching points in the right-hand diagram.

[0052] In one implementation, each set of target stereo images includes a left target image and a right target image; feature point detection and matching are performed on each set of target stereo images to determine at least one pair of matching feature points included in each set of target stereo images, including: For each set of target binocular images, a pre-defined feature point detection algorithm is used to detect feature points in the left and right images of the target respectively, resulting in multiple first feature points in the left image and multiple second feature points in the right image of the target. Using a pre-defined image feature matching algorithm, feature points are matched between the left and right images of the target. A first feature point and a second feature point that are successfully matched are taken as a pair of matched feature points.

[0053] In other words, different feature point detection algorithms and image feature matching algorithms can be selected according to actual needs, rather than using a single algorithm. Through adaptive adjustment of the algorithms, this combined optimization strategy can achieve the best results under different environmental conditions, cope with complex scene changes, such as changes in lighting and scene complexity, thereby improving the overall accuracy of line difference calculation.

[0054] Feature point detection algorithms include, but are not limited to: Harris Corner Detection: Harris corner detection is a classic feature point detection method, primarily used to find corners with high curvature in images. It identifies corners by calculating the image gradient and is robust to noise and lighting variations.

[0055] SIFT (Scale-Invariant Feature Transform): SIFT is a feature point detection method that is scale-invariant and rotation-invariant. It detects feature points at different scales by constructing a Gaussian pyramid and extracts a set of descriptors for matching. SIFT features are highly robust to changes in rotation, scale, and illumination, but have high computational complexity.

[0056] SURF (Speeded Up Robust Features): SURF is an improved version of SIFT, designed to increase computational speed. It uses integral images and Hessian matrices to quickly compute feature points and generate descriptors. SURF significantly reduces computation time while maintaining high accuracy.

[0057] ORB (Oriented FAST and Rotated BRIEF): ORB combines FAST (Features from Accelerated Segment Test) corner detection with BRIEF (Binary Robust Independent Elementary Features) descriptors, and introduces orientation information, giving it a degree of rotation invariance. ORB is more lightweight than SIFT and SURF, making it suitable for real-time applications.

[0058] Image feature matching algorithms include, but are not limited to: Brute-Force Matcher: Brute-force matching is the most direct matching method. For each feature point, it calculates the distance between it and all other feature points in another image (usually using Euclidean distance) and selects the point with the smallest distance as the match point. This method is simple but computationally intensive and inefficient.

[0059] FLANN (Fast Library for Approximate Nearest Neighbors): FLANN is an efficient nearest neighbor search algorithm library suitable for fast feature matching on large datasets. It quickly finds the approximate nearest neighbors of feature points by constructing an approximate KD (K-Dimensional) tree or other data structure.

[0060] KNN (K-Nearest Neighbors) matching: KNN matching matches each feature point with multiple feature points in another image, and then selects the k nearest points. Further, a ratio test is used to filter out unreliable matches; that is, if the distance difference between the nearest and second nearest neighbors is large, the match is considered reliable.

[0061] Bidirectional Matching: In bidirectional matching, feature points are matched once from the left image to the right image, and then once from the right image to the left image. Only when the matching results in both directions are consistent is the matching point pair considered reliable.

[0062] In this application, the feature point detection algorithm and the image feature matching algorithm can be switched manually to improve the performance and real-time nature of the calibration and reduce the occupation of additional computing resources; or, they can be automatically adjusted according to certain characteristics of the target binocular image through a pre-trained model or pre-set rules to improve the flexibility and adaptability of the calibration and reduce the need for manual intervention; no specific limitation is made.

[0063] In step S14, statistical analysis is performed based on multiple line differences to be processed to obtain at least one statistic, and the parameters of the binocular camera are calibrated according to the statistic.

[0064] In this step, statistical analysis is performed on multiple line differences to be processed, typically including calculating statistics such as mean, standard deviation, maximum, and minimum values. These statistics reflect the distribution and dispersion of the line differences, thereby identifying potential errors or deviations in image acquisition by the stereo camera. These errors or deviations may be caused by factors such as inaccurate camera intrinsic and extrinsic parameters, lens distortion, and changes in lighting conditions. Therefore, the purpose of statistical analysis is to identify these errors or deviations and provide a basis for subsequent parameter calibration.

[0065] Based on the statistical quantities obtained from the statistical analysis, the parameters that the binocular camera needs to be calibrated can be determined. Then, parameter calibration of the binocular camera can be performed. Parameter calibration is achieved by readjusting the camera's intrinsic and extrinsic parameters, or by reducing line error through epipolar correction. This enables the camera to acquire and process image information more accurately, improves the imaging quality and accuracy of the binocular camera, reduces errors and deviations caused by inaccurate camera parameters, and improves the accuracy of stereo matching and depth calculation, thereby enhancing the performance of the binocular vision system.

[0066] During the adjustment process, new pairs of stereo images to be processed can be continuously acquired for verification to ensure that the performance of the calibrated camera is improved. Once the calibrated camera meets certain accuracy requirements, the parameter calibration process is complete.

[0067] In one implementation, statistical analysis is performed based on multiple row differences to be processed to obtain at least one statistic, including: Calculate the mean and / or standard deviation of multiple row differences to be processed as statistics.

[0068] The average value is calculated by summing all the row differences to be processed and dividing by the total number of row differences. The result represents the average level of the row differences. It can be represented as:

[0069] The smaller the average value of the row difference, the more accurate the row difference calibration.

[0070] The standard deviation is calculated by first finding the square of the difference between each row difference and the mean, then summing these squares and dividing by the total number of row differences. The square root of this sum represents the degree of dispersion of the row differences. (Standard deviation of row differences) It can be represented as:

[0071] in, This represents the average of the row differences.

[0072] Standard deviation reflects the dispersion of row differences; a smaller standard deviation means that the row differences being processed are more consistent.

[0073] In one implementation, parameter calibration of the stereo camera is performed based on statistics, including: If the statistical quantity does not meet the preset qualification conditions, the parameters of the binocular camera are calibrated, and the process of acquiring multiple sets of synchronously acquired binocular images and vehicle motion information is returned. If the statistical quantity meets the preset qualification conditions, the parameter calibration of the binocular camera is deemed qualified. The preset qualification conditions include at least one of the following: the average value is less than a preset average value threshold; the standard deviation is less than a preset standard deviation threshold.

[0074] In other words, before performing parameter calibration, it is necessary to determine whether the statistics meet the preset acceptance criteria. These preset acceptance criteria may include, but are not limited to, at least one or more of the following combinations to obtain more comprehensive calibration results: the mean is less than a preset mean threshold; the standard deviation is less than a preset standard deviation threshold. The mean threshold and standard deviation threshold are determined based on the actual application scenario and calibration requirements; they represent the range and degree of dispersion of the row differences that are considered to require calibration.

[0075] The setting of the average threshold typically considers the imaging quality and accuracy requirements of the binocular camera, as well as the accuracy requirements of depth information in practical applications. For example, an average threshold of 0.5 can be used. The setting of the standard deviation threshold considers the impact of the dispersion of the line difference on the calibration results, as well as the adaptability to environmental changes in practical applications.

[0076] After obtaining the statistical values, it is necessary to determine whether the values ​​meet the preset acceptance criteria. If the values ​​do not meet the preset acceptance criteria, the parameters of the stereo camera need to be calibrated. Conversely, if the values ​​meet the preset acceptance criteria, the parameter calibration of the stereo camera can be considered successful, and no further calibration is required.

[0077] If the statistics still do not meet the preset qualification conditions after parameter calibration, it may be necessary to return to the "step of acquiring multiple sets of synchronously acquired binocular images and vehicle motion information" to re-acquire images and perform statistical analysis in order to further adjust and optimize the camera parameters.

[0078] Figure 2 This is a logical schematic diagram illustrating a calibration method according to an exemplary embodiment. Upon receiving a binocular image acquisition calibration request, the method systematically processes data acquisition and calibration issues in the binocular vision system by inputting binocular images and vehicle motion information, calculating binocular line difference, and determining acquisition calibration, and providing acquisition status feedback. This improves the accuracy and reliability of depth perception. Through precise feature point selection, matching, and line difference calculation, combined with vehicle motion information filtering and evaluation mechanisms, the performance of the stereo vision system in practical applications can be effectively improved.

[0079] As can be seen from the above, the technical solution provided by the embodiments of this application, by acquiring vehicle motion information synchronized with the binocular image to be processed, enables the calibration method to not only rely on the binocular image to be processed, but also to implement a filtering mechanism for the binocular image to be processed by combining real-time vehicle motion information. Only when the synchronously acquired vehicle motion information meets certain conditions will the binocular image to be processed be regarded as the target binocular image, thereby triggering the row difference calculation of the target binocular image. The filtering mechanism can ensure that the binocular vision system can efficiently process valid target binocular images, reduce the computational burden caused by the filtered invalid binocular images to be processed, and the filtering based on real-time vehicle motion information helps to reduce errors caused by vehicle motion. The row difference calculation based on multiple target binocular images obtained by filtering can improve the robustness of the binocular vision system, enabling the binocular vision system to perform row difference calculation more accurately during motion.

[0080] Figure 3 This is a block diagram illustrating a calibration apparatus according to an exemplary embodiment, comprising: The acquisition module 201 is used to acquire multiple sets of synchronously acquired binocular images to be processed and vehicle motion information, wherein the binocular images to be processed are acquired by the binocular camera mounted on the vehicle. The filtering module 202 is used to filter the binocular image to be processed according to the vehicle motion information to obtain multiple sets of target binocular images; The matching module 203 is used to perform feature point detection and matching on each group of target binocular images, determine at least one pair of matching feature points included in each group of target binocular images, and calculate the unprocessed row difference for each pair of matching feature points. The analysis module 204 is used to perform statistical analysis based on multiple line differences to be processed, obtain at least one statistical quantity, and perform parameter calibration on the binocular camera according to the statistical quantity.

[0081] Optionally, the acquisition module 201 is specifically used for: Based on timestamp synchronization technology, the binocular cameras are synchronously triggered to acquire the left and right images to be processed, and to acquire vehicle motion information. Distortion removal processing is performed on the left image and the right image to be processed to obtain a set of binocular images to be processed.

[0082] Optionally, the vehicle motion information is the motion change information between the current moment and the previous moment, and the filtering module 202 is specifically used for: For the Nth group of binocular images to be processed, determine whether the motion change information at the same moment is greater than a preset change threshold, where N is a positive integer greater than 1; If the value is greater than the target value, the Nth group of stereo images to be processed is used as the target stereo image; if the value is not greater than the target value, the Nth group of stereo images to be processed is filtered out.

[0083] Optionally, each set of target binocular images includes a left target image and a right target image; the matching module 203 is specifically used for: For each set of target binocular images, a pre-set feature point detection algorithm is used to detect feature points in the left image and the right image of the target respectively, so as to obtain multiple first feature points in the left image of the target and multiple second feature points in the right image of the target. Using a pre-defined image feature matching algorithm, feature point matching is performed on the left and right images of the target, and a first feature point and a second feature point that are successfully matched are taken as a pair of matched feature points.

[0084] Optionally, the analysis module 204 includes: The calculation unit is used to calculate the average and / or standard deviation of the multiple row differences to be processed as statistics.

[0085] Optionally, the analysis module 204 includes: A calibration unit is used to calibrate the parameters of the binocular camera when the statistical quantity does not meet the preset qualification conditions, and return to the step of acquiring multiple sets of synchronously acquired binocular images to be processed and vehicle motion information; when the statistical quantity meets the preset qualification conditions, the parameter calibration of the binocular camera is determined to be qualified; wherein, the preset qualification conditions include at least one of the following: the average value is less than a preset average value threshold; the standard deviation is less than a preset standard deviation threshold.

[0086] As can be seen from the above, the technical solution provided by the embodiments of this application, by acquiring vehicle motion information synchronized with the binocular image to be processed, enables the calibration method to not only rely on the binocular image to be processed, but also to implement a filtering mechanism for the binocular image to be processed by combining real-time vehicle motion information. Only when the synchronously acquired vehicle motion information meets certain conditions will the binocular image to be processed be regarded as the target binocular image, thereby triggering the row difference calculation of the target binocular image. The filtering mechanism can ensure that the binocular vision system can efficiently process valid target binocular images, reduce the computational burden caused by the filtered invalid binocular images to be processed, and the filtering based on real-time vehicle motion information helps to reduce errors caused by vehicle motion. The row difference calculation based on multiple target binocular images obtained by filtering can improve the robustness of the binocular vision system, enabling the binocular vision system to perform row difference calculation more accurately during motion.

[0087] Optionally, embodiments of this application also provide an electronic device.

[0088] Figure 4 This is a block diagram illustrating an electronic device according to an exemplary embodiment.

[0089] like Figure 4 As shown, the electronic device 11 includes a processor 111, a memory 112, and a program or instruction 1121 stored in the memory and executable on the processor. When the program or instruction 1121 is executed by the processor 111, it implements the various processes of any of the above-described calibration method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0090] Optionally, embodiments of this application also provide a readable storage medium.

[0091] Figure 5 This is a block diagram illustrating a readable storage medium according to an exemplary embodiment.

[0092] like Figure 5 As shown, a program or instruction 211 is stored on the readable storage medium 21. When the program or instruction 211 is executed by the processor, it implements the various processes of any of the above calibration method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0093] The processor may be the processor in the electronic device described in the above embodiments.

[0094] The processors mentioned above may include, but are not limited to: Central Processing Unit (CPU), Network Processor (NP), Digital Signal Processor (DSP), Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0095] The aforementioned readable storage media include computer-readable storage media, which may include, but are not limited to: read-only memory (ROM), random access memory (RAM), compact disc read-only memory (CD-ROM), electronically erasable programmable read-only memory (EEPROM), hard disk, floppy disk, flash memory, etc.

[0096] Optionally, embodiments of this application also provide a computer program product, on which a program or instructions are stored, and when the program or instructions are executed by a processor, the calibration method as described in any of the above embodiments is implemented.

[0097] As can be seen from the above, the technical solution provided by the embodiments of this application, by acquiring vehicle motion information synchronized with the binocular image to be processed, enables the calibration method to not only rely on the binocular image to be processed, but also to implement a filtering mechanism for the binocular image to be processed by combining real-time vehicle motion information. Only when the synchronously acquired vehicle motion information meets certain conditions will the binocular image to be processed be regarded as the target binocular image, thereby triggering the row difference calculation of the target binocular image. The filtering mechanism can ensure that the binocular vision system can efficiently process valid target binocular images, reduce the computational burden caused by the filtered invalid binocular images to be processed, and the filtering based on real-time vehicle motion information helps to reduce errors caused by vehicle motion. The row difference calculation based on multiple target binocular images obtained by filtering can improve the robustness of the binocular vision system, enabling the binocular vision system to perform row difference calculation more accurately during motion.

[0098] The various embodiments in this specification are related to each other and are described in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other.

[0099] It should be noted that all actions involving the acquisition of signals, information, or data in this application are carried out in compliance with the relevant data protection laws and regulations of the locality and with authorization from the owner of the relevant device.

[0100] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0101] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0102] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms. The functional units in the various embodiments of this application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0103] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0104] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A calibration method, characterized in that, include: Multiple sets of synchronously acquired binocular images to be processed and vehicle motion information are acquired, wherein the binocular images to be processed are acquired by the binocular camera mounted on the vehicle; Based on the vehicle motion information, the binocular images to be processed are filtered to obtain multiple sets of target binocular images; For each group of target binocular images, feature point detection and matching are performed to determine at least one pair of matching feature points included in each group of target binocular images, and the unprocessed row difference of each pair of matching feature points is calculated. Statistical analysis is performed on multiple line differences to be processed to obtain at least one statistic, and the parameters of the binocular camera are calibrated based on the statistic.

2. The calibration method according to claim 1, characterized in that, The acquisition of multiple sets of synchronously acquired binocular images and vehicle motion information includes: Based on timestamp synchronization technology, the binocular cameras are synchronously triggered to acquire the left and right images to be processed, and to acquire vehicle motion information. Distortion removal processing is performed on the left image and the right image to be processed to obtain a set of binocular images to be processed.

3. The calibration method according to claim 1, characterized in that, The vehicle motion information refers to the motion change information between the current moment and the previous moment. Based on the vehicle motion information, the binocular images to be processed are filtered to obtain multiple sets of target binocular images, including: For the Nth group of binocular images to be processed, determine whether the motion change information at the same moment is greater than a preset change threshold, where N is a positive integer greater than 1; If the value is greater than the target value, the Nth group of stereo images to be processed is used as the target stereo image; if the value is not greater than the target value, the Nth group of stereo images to be processed is filtered out.

4. The calibration method according to claim 1, characterized in that, Each set of target binocular images includes a left target image and a right target image; the step of performing feature point detection and matching on each set of target binocular images to determine at least one pair of matching feature points included in each set of target binocular images includes: For each set of target binocular images, a pre-set feature point detection algorithm is used to detect feature points in the left image and the right image of the target respectively, so as to obtain multiple first feature points in the left image of the target and multiple second feature points in the right image of the target. Using a pre-defined image feature matching algorithm, feature point matching is performed on the left and right images of the target, and a first feature point and a second feature point that are successfully matched are taken as a pair of matched feature points.

5. The calibration method according to claim 1, characterized in that, The statistical analysis based on multiple row differences to be processed yields at least one statistic, including: Calculate the mean and / or standard deviation of the multiple rows of differences to be processed as statistics.

6. The calibration method according to claim 5, characterized in that, The step of calibrating the parameters of the binocular camera based on the statistics includes: If the statistical quantity does not meet the preset qualification conditions, the parameters of the binocular camera are calibrated, and the process of acquiring multiple sets of synchronously acquired binocular images and vehicle motion information is returned. If the statistical quantity meets the preset qualification conditions, the parameter calibration of the binocular camera is deemed qualified. The preset qualification conditions include at least one of the following: the average value is less than a preset average value threshold; the standard deviation is less than a preset standard deviation threshold.

7. A calibration device, characterized in that, include: The acquisition module is used to acquire multiple sets of synchronously acquired binocular images to be processed and vehicle motion information. The binocular images to be processed are acquired by the binocular camera mounted on the vehicle. The filtering module is used to filter the binocular image to be processed based on the vehicle motion information to obtain multiple sets of target binocular images; The matching module is used to detect and match feature points for each group of target binocular images, determine at least one pair of matching feature points included in each group of target binocular images, and calculate the unprocessed row difference for each pair of matching feature points. The analysis module is used to perform statistical analysis based on multiple line differences to be processed, obtain at least one statistical quantity, and perform parameter calibration on the binocular camera according to the statistical quantity.

8. An electronic device, characterized in that, The electronic device includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the calibration method as described in any one of claims 1 to 6.

9. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the calibration method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product stores a program or instructions that, when executed by a processor, implement the calibration method as described in any one of claims 1 to 6.

Images