Robot vision system dynamic calibration method and system based on data fusion

Abstract

The invention discloses a dynamic calibration method and system for a robot vision system based on data fusion, and relates to the technical field of robot vision calibration, and the method comprises the steps: obtaining a calibration plate image sequence and a joint torque signal of a robot, extracting the vibration characteristics of a candidate angular point, such as the angular point vibration amplitude, the angular point vibration frequency and the dominant resonance frequency; and judging whether the system is in a quasi-static state. In a quasi-static state, according to comparison of an average value, a maximum value and a standard deviation of angular point vibration amplitudes and a preset threshold value, dynamically determining an adjustment coefficient based on a comparison relationship between current and historical vibration characteristics in a self-adaptive manner, further calculating a self-adaptive data acquisition period, and acquiring additional image data; and then, multi-dimensional screening is carried out based on the spatial distribution characteristics of the constructed vibration feature vectors, an optimized data set is obtained, and finally, calibration is completed by adopting a robust estimation algorithm. According to the method, the calibration period precision and robustness of the robot vision system in the dynamic vibration environment are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of robot vision calibration technology, and more specifically, this application relates to a dynamic calibration method and system for robot vision systems based on data fusion. Background Technology

[0002] In the field of robot vision system calibration technology, traditional calibration methods are generally based on the assumption of a static environment. These methods typically require the robot system to be completely stationary, and to acquire calibration board image data for parameter calculation under this ideal state. However, actual industrial application environments are full of various dynamic interference factors. Residual vibrations after the robot body stops moving, structural resonances caused by the operation of other equipment in the workshop, and low-frequency jitter caused by the characteristics of the servo system itself will all continuously act on the vision sensor, causing slight temporal drift and continuous vibration of the corner coordinates in the calibration board image sequence.

[0003] Meanwhile, vibration characteristics often change dynamically over time. The residual vibration of the robot body will gradually decay over time, while environmental interference may occur randomly, making it difficult for a fixed data acquisition cycle to adapt to this dynamically changing vibration environment.

[0004] Existing calibration methods have significant shortcomings in their data acquisition strategies. Due to the use of a fixed acquisition period, the duration of data acquisition cannot be adjusted based on real-time vibration characteristics. Data quality is poor during periods of strong vibration, while high-quality data may be missed if acquisition is terminated too early during periods of weak vibration. This mismatch between the acquisition strategy and the vibration environment makes it difficult to obtain the optimal quality dataset within a limited timeframe. Furthermore, traditional methods lack a response mechanism to dynamic changes in vibration characteristics, failing to automatically extend the acquisition time to obtain a sufficient amount of effective data during periods of sustained strong vibration, and also failing to optimize acquisition efficiency in a timely manner as vibration rapidly decays.

[0005] To address the aforementioned issues, there is an urgent need in this field for a calibration method capable of dynamically optimizing data acquisition strategies based on vibration characteristics. A key shortcoming of existing technologies is that their fixed data acquisition cycles cannot adapt to the dynamically changing vibration environments of industrial sites. This results in traditional visual calibration methods failing to guarantee the quality and efficiency of calibration data under conditions where vibration characteristics change significantly over time. Summary of the Invention

[0006] To address the aforementioned technical problems, this technical solution provides a dynamic calibration method for robot vision systems based on data fusion, thus resolving the issues raised in the background section.

[0007] In a first aspect, embodiments of this application provide a dynamic calibration method for a robot vision system based on data fusion, comprising the following steps: acquiring and processing image sequence data of a calibration board and joint torque signal sequence of a target robot, extracting vibration features of candidate corner points, the vibration features including corner point vibration amplitude, corner point vibration frequency, and dominant resonant frequency; determining that the system is in a quasi-static state when the corner point vibration amplitude of all candidate corner points is less than a stability judgment threshold, and the difference between the corner point vibration frequency and the dominant resonant frequency of any candidate corner point is greater than a preset frequency tolerance; when the system is in a quasi-static state, the system is further determined when the average value of the corner point vibration amplitude is greater than or equal to a first preset threshold, or the maximum value of the corner point vibration amplitude is greater than or equal to a first preset threshold. When the value is greater than or equal to the second preset threshold, or the standard deviation of the corner vibration amplitude is greater than or equal to the third preset threshold, the adjustment coefficient is determined based on the vibration characteristics of all candidate corners and the comparison relationship of historical vibration characteristics, and the data acquisition cycle is calculated accordingly. According to the data acquisition cycle, additional calibration plate image sequence data is collected, vibration features are extracted from the collected calibration plate image sequence data and additional calibration plate image sequence data, and vibration feature vectors are constructed accordingly. Based on the spatial distribution characteristics of the vibration feature vectors, the data is filtered to obtain an optimized dataset. Based on the optimized dataset, a robust estimation algorithm is used to calculate the calibration parameter data and output it to the calibration equipment of the target robot.

[0008] Secondly, embodiments of this application provide a dynamic calibration system for a robot vision system based on data fusion, comprising: a data acquisition module for acquiring and processing image sequence data of the calibration board and joint torque signal sequence of the target robot to obtain the corner vibration amplitude, corner vibration frequency, and dominant resonance frequency of candidate corner points; a corner judgment module for determining that the system is in a quasi-static state when the corner vibration amplitude of all candidate corner points is less than a stability judgment threshold and the difference between the corner vibration frequency and the dominant resonance frequency of any candidate corner point is greater than a preset frequency tolerance; and a quasi-static processing module for determining that, when the system is in a quasi-static state, the average value of the corner vibration amplitude is greater than or equal to a first preset threshold, or the maximum value of the corner vibration amplitude is greater than or equal to a first preset threshold. When the standard deviation of the corner vibration amplitude is greater than or equal to the second preset threshold, or greater than or equal to the third preset threshold, the adjustment coefficient is determined based on the vibration characteristics of all candidate corners and the comparison relationship of historical vibration characteristics, and the data acquisition cycle is calculated accordingly; Calibration parameter processing module: used to collect additional calibration board image sequence data according to the data acquisition cycle, extract vibration features from the collected calibration board image sequence data and additional calibration board image sequence data and construct vibration feature vectors accordingly, and perform filtering processing based on the spatial distribution characteristics of vibration feature vectors to obtain an optimized dataset; Output module: used to calculate calibration parameter data based on the optimized dataset using a robust estimation algorithm and output it to the calibration equipment of the target robot accordingly.

[0009] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0010] 1. Quasi-static timing is determined by analyzing corner vibration amplitude, corner vibration frequency, and dominant resonance frequency, and data enhancement strategies are dynamically triggered based on the average, maximum, and standard deviation of corner vibration amplitude. The data acquisition cycle is dynamically determined by combining the comparison between current and historical vibration characteristics, and intelligent filtering is performed based on the spatial distribution characteristics of vibration feature vectors. This enables the system to adapt to different vibration environments, solving the problem of incompatibility between traditional methods and the environment due to fixed acquisition modes.

[0011] 2. This method not only utilizes the corner coordinate data of the calibration board, but also integrates the edge contour data of the calibration board mesh and the optical flow vector data of adjacent image frames, and performs joint optimization by constructing a multi-source feature objective function. This multi-feature fusion mechanism ensures that when the reliability of a certain type of feature decreases due to vibration interference, other features can still provide effective constraints, thereby maintaining the stability of the calibration solution under vibration environment and overcoming the weakness of traditional methods that rely too heavily on a single feature.

[0012] 3. By constructing vibration feature vectors, calculating vibration feature variation rates, and combining historical period comparisons to quantify the contribution weight of changes in each feature, the vibration trend can be dynamically predicted and system parameters adjusted. This feedback mechanism based on historical data enables the system to intelligently allocate resources, strengthening data acquisition when vibration intensifies and optimizing efficiency when vibration improves, achieving optimal allocation of computational and time resources. Attached Figure Description

[0013] Figure 1 A schematic diagram illustrating the steps of the dynamic calibration method for a robot vision system based on data fusion provided in this application embodiment;

[0014] Figure 2 A schematic diagram of the logic flow for obtaining the data acquisition cycle provided in the embodiments of this application;

[0015] Figure 3 A schematic diagram of the logical flow for obtaining the optimized dataset provided in an embodiment of this application;

[0016] Figure 4 A logical diagram illustrating the iterative optimization process of the multi-source feature objective function provided in the embodiments of this application;

[0017] Figure 5 A logical diagram illustrating the calculation of fuzzy inference weights provided in an embodiment of this application;

[0018] Figure 6 This is a schematic diagram of the structure of a dynamic calibration system for a robot vision system based on data fusion, provided in an embodiment of this application. Detailed Implementation

[0019] This application's embodiments address the technical problem in the prior art where, under conditions where vibration characteristics change significantly over time, it is difficult to ensure that the data acquisition cycle cannot adapt to the dynamic changes in the industrial site. This is achieved through a dynamic calibration method for robot vision systems based on data fusion.

[0020] In high-precision manufacturing scenarios such as semiconductor wafer transport and precision electronic assembly, the core contradiction of traditional visual calibration methods is that the calibration process requires a static environment, while the actual industrial environment is full of dynamic vibrations. This contradiction leads to two main problems: the timing of calibration is difficult to grasp, and calibration before the vibration has fully decayed will introduce systematic errors; the calibration strategy is too simplistic and cannot be dynamically adjusted according to vibration characteristics, resulting in insufficient accuracy or low efficiency.

[0021] The technical solution employs a multi-source data fusion sensing method, simultaneously acquiring calibration plate image sequences and joint torque signals to obtain vibration information from both the visual and dynamic domains. The temporal changes in corner coordinates within the image sequences reflect the external manifestations of the vibration, while the joint torque signals reveal the system's inherent resonance characteristics. This dual-modal sensing lays the data foundation for subsequent intelligent decision-making.

[0022] Recognizing that vibration is unavoidable, it is necessary to establish standards that better align with engineering realities to define suitable calibration states. The quasi-static concept embodies this idea; it does not pursue absolute zero vibration, but rather requires vibration to be in a controllable and non-destructive state. This is achieved through two criteria: the vibration amplitude must be less than the engineering acceptable threshold, ensuring that the swaying is within tolerance; and the vibration frequency must be far from the system's dominant resonant frequency to avoid destructive structural resonance. This criterion allows the system to find a practical calibration window in a real industrial environment.

[0023] After determining the quasi-static window, the statistical characteristics of corner vibration amplitude are further analyzed. The average value reflects the overall disturbance level, the maximum value reveals the risk of abnormal frames, and the standard deviation characterizes data stability. The system can predict the quality level of the current data based on these characteristics and trigger different processing strategies. When all three statistics are relatively good, the standard strategy that pursues the ultimate accuracy is adopted; when any statistic deteriorates, the enhancement strategy of multi-source feature fusion is activated; when multiple statistics deteriorate significantly, the data enhancement strategy is initiated. This hierarchical processing achieves an optimal balance between accuracy, robustness, and efficiency.

[0024] The technical solution establishes a dynamically adjusted intelligent closed loop by introducing a historical state comparison mechanism. Specifically, it constructs vibration feature vectors and calculates their variation rate, enabling the system to perceive environmental trends. By analyzing the contribution weights of each vibration feature, the system can diagnose the current primary problem. Finally, through the dynamic calculation of adjustment coefficients, the diagnostic results are transformed into specific action parameters and data acquisition cycles. This mechanism allows the system not only to respond to the current vibration state but also to make adjustments in advance based on vibration trends.

[0025] In the data augmentation strategy, the technical solution ensures the final data quality through multi-dimensional optimization. In the temporal dimension, the acquisition cycle is dynamically extended based on the severity of vibration to ensure sufficient data samples are obtained. In the spatial dimension, intelligent filtering is performed based on the distribution characteristics of vibration feature vectors, retaining high-quality frames that simultaneously meet the requirements of low amplitude and high resonance deviation. In the quality dimension, final filtering is performed based on image sharpness. This multi-level, multi-dimensional optimization strategy ensures reliable calibration data can be obtained even under poor vibration conditions.

[0026] This approach forms a complete technical path to achieve the desired results: understanding the environmental state through multi-source sensing, determining the calibration timing through intelligent judgment, implementing a refined hierarchical matching strategy, achieving adaptive optimization through dynamic adjustment, and ensuring data quality through multi-dimensional filtering. The entire process forms a complete closed loop from environmental perception to decision execution, successfully transforming the obstacles faced by traditional calibration methods into quantifiable, manageable, and optimizable technical parameters, thereby achieving stable, high-precision calibration in real-world dynamic industrial environments.

[0027] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0028] like Figure 1The diagram illustrates the steps of a dynamic calibration method for a robot vision system based on data fusion, as provided in this application embodiment. The method includes the following steps: acquiring and processing image sequence data of the calibration board and joint torque signal sequence of the target robot; extracting vibration features of candidate corner points, including corner vibration amplitude, corner vibration frequency, and dominant resonant frequency; determining a quasi-static state when the corner vibration amplitude of all candidate corner points is less than a stability judgment threshold, and the difference between the corner vibration frequency and the dominant resonant frequency of any candidate corner point is greater than a preset frequency tolerance; and determining a quasi-static state when the average value of the corner vibration amplitude is greater than or equal to a first preset threshold. When the maximum value of the corner vibration amplitude is greater than or equal to the second preset threshold, or the standard deviation of the corner vibration amplitude is greater than or equal to the third preset threshold, the adjustment coefficient is determined based on the vibration characteristics of all candidate corners and the comparison relationship of historical vibration characteristics, and the data acquisition cycle is calculated accordingly. Additional calibration plate image sequence data is acquired according to the data acquisition cycle. Vibration features are extracted from the acquired calibration plate image sequence data and additional calibration plate image sequence data, and vibration feature vectors are constructed accordingly. Based on the spatial distribution characteristics of the vibration feature vectors, a filtering process is performed to obtain an optimized dataset. Based on the optimized dataset, a robust estimation algorithm is used to calculate calibration parameter data, and the data is output to the calibration equipment of the target robot.

[0029] The stability judgment threshold is determined through the following steps: Under standard experimental conditions, multiple sets of calibration plate image sequence data under different vibration states are collected, and the statistical distribution characteristics of the corner vibration amplitude are calculated for each set of data; based on the upper limit of vibration amplitude set according to the calibration accuracy requirements, the critical value of corner vibration amplitude that can guarantee calibration accuracy is determined through statistical analysis; this critical value is determined as the stability judgment threshold after multiple experimental verifications.

[0030] The preset frequency tolerance is determined through the following steps: analyze the robot's structural dynamics characteristics to obtain the bandwidth range of the dominant resonant frequency; determine the measurement error range of the resonant frequency through frequency domain analysis; and, taking into account the structural safety factor and calibration accuracy requirements, set the frequency tolerance as the weighted sum of the resonant frequency bandwidth and the measurement error to ensure effective avoidance of the resonant region.

[0031] The first, second, and third preset thresholds are determined through the following steps: a large amount of calibration data is collected under typical working conditions to establish a model relating the average, maximum, and standard deviation of the corner vibration amplitude to the calibration error; an acceptable upper limit for the calibration error is set based on the calibration accuracy requirements, and the corresponding vibration characteristic threshold is derived by reverse calculation; the threshold settings are optimized through cross-validation to ensure good robustness while maintaining accuracy.

[0032] Vibration characteristics can be understood as physical quantities that reflect the dynamic changes of candidate corner points in the calibration plate image sequence. Specifically, they can be extracted by analyzing the changes in corner point coordinates over time in the image sequence. For example, by calculating the displacement amplitude, frequency components, and frequency distribution of the main vibration modes of the corner point coordinates, the corner point vibration amplitude, corner point vibration frequency, and dominant resonance frequency can be obtained respectively.

[0033] By fusing calibration board image sequence data and joint torque signal sequence, comprehensive perception of vibration characteristics is achieved. Based on the vibration characteristics, the data acquisition cycle is dynamically adjusted and the dataset is filtered and optimized, thereby solving the problems of low efficiency and unstable quality of calibration data acquisition for robot vision systems in dynamic vibration environments in industrial settings.

[0034] Compared to traditional fixed-period data acquisition strategies, this application can flexibly adjust the acquisition duration according to real-time vibration characteristics, ensuring that the acquisition time is extended when the vibration is strong to obtain a sufficient amount of effective data, while optimizing the acquisition efficiency in a timely manner when the vibration is weak, thereby improving the overall reliability of the calibration data.

[0035] Further, the specific steps for obtaining the corner vibration amplitude, corner vibration frequency, and dominant resonance frequency are as follows: The target robot's visual and mechanical sensors continuously collect data on the calibration plate and robot joints for a predetermined duration, respectively, to obtain a calibration plate image sequence and a robot joint torque signal sequence; Frequency domain transformation analysis is performed on the robot joint torque signal sequence to extract the dominant resonance frequency data; Corner detection is performed on the calibration plate image of each image frame in the calibration plate image sequence to obtain initial two-dimensional coordinate data of multiple candidate corners; Based on the time series in the calibration plate image sequence, the initial two-dimensional coordinate data of the same candidate corner are associated and combined to obtain the corner coordinate time series data of each candidate corner; Frequency domain transformation analysis is performed on the corner coordinate time series data to extract the vibration frequency and vibration amplitude of each candidate corner, obtaining the corner vibration amplitude and corner vibration frequency; Frequency domain transformation analysis is performed on the robot joint torque signal sequence to extract the dominant resonance frequency.

[0036] In this embodiment, the calibration board image sequence refers to a set of images obtained by continuously shooting the calibration board at different time points, which can be achieved by an industrial camera or a high-precision vision sensor.

[0037] Among them, the robot joint torque signal sequence can be understood as the torque change data generated by each joint of the robot during the movement, which can be collected by built-in torque sensors or external measuring devices.

[0038] Specifically, corner detection refers to identifying points with significant geometric features from an image. It can be implemented using the Harris corner detection algorithm or the FAST corner detection algorithm, and is used to focus on the key geometric features of the calibration plate to improve the accuracy of subsequent vibration analysis.

[0039] In addition, frequency domain transform analysis refers to the technique of converting time-domain signals into frequency-domain representations. It can be achieved through fast Fourier transform or wavelet transform to separate periodic components and noise interference in signals, thereby accurately extracting vibration characteristics.

[0040] By synchronously acquiring calibration board image sequences and robot joint torque signal sequences using a vision sensor, the temporal consistency of multi-source data is ensured, avoiding feature distortion caused by asynchronous sampling from a single data source.

[0041] By performing frequency domain transformation analysis on the joint torque signal sequence, the dominant resonant frequency is extracted. The robot joint torque directly reflects the structural vibration characteristics, effectively separating environmental interference from the body vibration components, and providing an objective environmental vibration benchmark for subsequent quasi-static determination.

[0042] Meanwhile, corner point detection is performed on each frame of the calibration board image sequence and initial two-dimensional coordinate data is extracted. The changes in these coordinates can directly characterize the displacement state of the visual sensor affected by vibration, thus closely linking it to the core objective of the calibration task.

[0043] By associating and combining coordinate data of the same corner point based on time series, a continuous vibration trajectory is constructed, overcoming the time series breakage problem caused by isolated processing of corner point data in traditional methods. Through frequency domain transformation analysis of the corner point coordinate time series data, vibration frequency and amplitude are extracted, and the vibration energy distribution of each corner point is quantified, making the vibration characteristics physically interpretable.

[0044] The robustness of the dominant resonant frequency is enhanced through a dual verification mechanism, ensuring its consistency in vibration feature selection and preventing subsequent resonance deviation judgment failures due to single analysis errors. Overall, the coordinated design of data synchronization, time-series correlation, and frequency domain transformation solves the problem of unclear vibration feature acquisition, providing stable and reliable vibration feature inputs for quasi-static determination and data acquisition cycle optimization, thereby improving the accuracy of calibration parameter calculation.

[0045] Furthermore, the specific rules for determining the data acquisition cycle are as follows: Adjustment coefficients are obtained based on the comparison between current vibration characteristics and historical vibration characteristics. These adjustment coefficients include a first adjustment coefficient corresponding to the average corner vibration amplitude, a second adjustment coefficient corresponding to the maximum corner vibration amplitude, and a third adjustment coefficient corresponding to the standard deviation of the corner vibration amplitude. Based on the average corner vibration amplitude, the maximum corner vibration amplitude, and the standard deviation of the corner vibration amplitude, the data acquisition cycle is calculated according to the following formula: ,in, Indicates the data collection period. Indicates the baseline acquisition duration. This represents the average value of the corner vibration amplitude. This represents the first preset threshold. This represents the maximum amplitude of the corner vibration. This indicates the second preset threshold. The standard deviation of the corner vibration amplitude. This indicates the third preset threshold. This represents the first adjustment factor. This represents the second adjustment factor. This represents the third adjustment factor.

[0046] In this embodiment, Figure 2 This is a schematic diagram of the logical flow of the data acquisition cycle provided in the embodiments of this application; the baseline acquisition time is determined by the following steps: analyzing the basic requirements of the calibration algorithm for the amount of data, determining the minimum number of image frames required to complete one calibration; considering the performance parameters of the image acquisition device, calculating the shortest time required to acquire these images; and adding an appropriate safety margin to ensure that sufficient effective data can be obtained under optimal working conditions.

[0047] The adjustment coefficient is a dynamic parameter determined based on the degree of difference between the current vibration characteristics and historical vibration characteristics. It can be implemented using various mathematical models or algorithms, such as extracting vibration change trends through time series analysis and then quantifying them. This design enables the system to identify the evolution of vibration states, thereby avoiding over-response to instantaneous fluctuations.

[0048] The first, second, and third adjustment coefficients correspond to the average, maximum, and standard deviation of the corner vibration amplitude, respectively. This multi-dimensional design covers the overall level, extreme peak values, and dispersion of vibration, ensuring that the adjustment mechanism can comprehensively respond to different vibration characteristics. In addition, the baseline acquisition duration provides a basic periodic guarantee, while the preset threshold defines the acceptable range of vibration, together forming a reference framework for adaptive adjustment.

[0049] By establishing a quantitative correlation mechanism between vibration characteristics and data acquisition cycle, dynamic adaptive adjustment of the acquisition cycle was achieved.

[0050] In actual operation, the adjustment coefficient is first obtained based on the comparison between the current vibration characteristics and historical vibration characteristics. This process highlights the importance of historical vibration characteristics as a dynamic reference benchmark, enabling the adjustment coefficient to reflect the long-term pattern of vibration attenuation or enhancement. Subsequently, the data acquisition cycle is calculated using a formula, where the deviation term is used to quantify the difference between the actual vibration and the threshold. Positive values ​​trigger an extended cycle to capture more effective data, while negative values ​​shorten the cycle to improve efficiency.

[0051] The adjustment coefficient, as a dynamic weight, is determined in real time based on historical comparisons, ensuring that the periodic adjustment amplitude precisely matches the degree of vibration change and avoiding the rigidity of fixed parameters.

[0052] Combined with the aforementioned techniques for acquiring vibration characteristics of candidate corner points, this approach accurately matches data acquisition needs when the vibration environment changes, thereby effectively balancing data acquisition efficiency and quality and solving the problems caused by the lack of accurate mathematical models and adaptive mechanisms.

[0053] Further, the specific process for obtaining the adjustment coefficient is as follows: The average value of the corner vibration amplitude, the maximum value of the corner vibration amplitude, and the standard deviation of the corner vibration amplitude are each constructed as vectors in three dimensions of a preset three-dimensional spatial coordinate system, and these vectors are synthesized to obtain the vibration feature vector; the vibration feature vector of the previous data acquisition cycle is obtained; the Euclidean distance between the current vibration feature vector and the vibration feature vector of the previous data acquisition cycle is calculated to obtain the vibration feature Euclidean distance; the sum of the magnitudes of the current vibration feature vector and the vibration feature vector of the previous data acquisition cycle is calculated to obtain the feature magnitude sum; the ratio of the vibration feature Euclidean distance to the feature magnitude sum is recorded as the vibration feature variation rate; when the vibration feature variation rate is greater than a preset variation rate threshold, the following adjustment coefficient acquisition steps are performed: The calculated... The difference between the current average corner vibration amplitude and the average corner vibration amplitude of the previous data acquisition period is recorded as the first difference. The difference between the current maximum corner vibration amplitude and the maximum corner vibration amplitude of the previous data acquisition period is recorded as the second difference. The difference between the standard deviation of the corner vibration amplitude of the current period and the standard deviation of the historical corner vibration amplitude is calculated as the third difference. The proportions of the absolute values ​​of the first, second, and third differences in the sum of the three differences are calculated and used as the contribution weights for the change in the average value, the change in the maximum value, and the change in the standard deviation, respectively. Based on the positive and negative characteristics of the first, second, and third differences, combined with the corresponding change contribution weights and the vibration characteristic variation rate, the first adjustment coefficient, the second adjustment coefficient, and the third adjustment coefficient are calculated respectively.

[0054] In this embodiment, when calculating the vibration characteristic variation rate, the method involves obtaining the vibration characteristic vector of the previous data acquisition cycle. However, for the first data acquisition cycle, since there is no historical data, the following initialization strategy is adopted: In the first data acquisition cycle, the system uses a preset default vibration characteristic vector as the "vibration characteristic vector of the previous data acquisition cycle". This default vector is based on the theoretical value of the calibration plate under ideal steady state or obtained through pre-calibration experiments. For example, the average value, maximum value, and standard deviation of the corner vibration amplitude are all set to zero or small values ​​close to zero to simulate a vibration-free state.

[0055] Adjustment coefficient initialization: If the vibration characteristic variation rate is greater than the preset variation rate threshold, the adjustment coefficient is calculated; otherwise, the preset default adjustment coefficient is used directly. In the first cycle, it is recommended to temporarily adjust the variation rate threshold to a higher value to tolerate larger initial variations and ensure a smooth system startup.

[0056] The preset variability threshold is determined through the following steps: analyzing the correspondence between the vibration characteristic variability rate and the calibration success rate in historical calibration data; setting an acceptable minimum calibration success rate and determining the corresponding variability threshold; verifying the applicability of the threshold under different working conditions through experiments, and finally determining the preset variability threshold.

[0057] Vibration feature vectors are geometric representations formed by mapping the average value of corner vibration amplitude, the maximum value of corner vibration amplitude, and the standard deviation of corner vibration amplitude to a three-dimensional spatial coordinate system. They can be implemented using multi-dimensional feature fusion technology or principal component analysis to integrate multi-dimensional vibration features and provide a unified quantitative basis.

[0058] The vibration characteristic variation rate can be understood as a measure of the overall change of the current vibration characteristic relative to the historical benchmark. It can be calculated by the ratio of the Euclidean distance to the sum of the moduli. It is used to filter out small fluctuations and ensure the robustness of the adjustment strategy.

[0059] The contribution weights of mean change, maximum change, and standard deviation change refer to the relative importance of the changes in each vibration index in the overall change. They can be achieved through normalization or proportional allocation algorithms and are used to dynamically respond to the differences in importance of different vibration characteristics.

[0060] By establishing a vectorized representation of vibration characteristics and a dynamic variability mechanism, accurate adaptive calculation of the adjustment coefficient is achieved. The average, maximum, and standard deviation of the corner vibration amplitude are constructed as vectors in a three-dimensional spatial coordinate system and synthesized into a vibration characteristic vector, so that multi-dimensional vibration characteristics are integrated into a unified geometric representation, avoiding the one-sidedness of single index analysis.

[0061] By obtaining the vibration feature vector from the previous data acquisition period as a historical benchmark, the changes in current vibration characteristics can be compared based on the time series, ensuring the continuity of change analysis and the reliability of the reference.

[0062] The ratio of the Euclidean distance to the sum of the magnitudes of the current vibration feature vector and the vector from the previous data acquisition cycle is calculated as the vibration feature variation rate. This variation rate quantifies the overall change amplitude of the vibration feature through geometric relationships. The calculation formula is:

[0063] ,in, This represents the current vibration eigenvector. Let d(.) represent the historical vibration characteristic vector, and d(.) represent the Euclidean distance.

[0064] The adjustment coefficient calculation is triggered only when the variability rate exceeds a preset threshold, effectively filtering out minor vibration fluctuations and preventing invalid adjustments due to noise or transient interference. When the variability rate is significant, the specific differences between the current and historical vibration characteristics are calculated. These differences accurately capture the changes in each vibration index, providing data support for subsequent weight allocation. Furthermore, the proportion of the absolute value of the difference in the sum is used as the weight of the change contribution. This weight reflects the degree of contribution of the average, maximum, and standard deviation of the corner vibration amplitude to the overall vibration change, enabling the adjustment coefficient to dynamically respond to the differences in the importance of different vibration characteristics.

[0065] The adjustment coefficient is calculated based on the positive and negative characteristics of the difference, the weight of the change contribution, and the variability rate of the vibration characteristics. When the difference is positive, the adjustment coefficient is amplified in combination with the variability rate to extend the acquisition time to cope with strong vibrations. When the difference is negative, the adjustment coefficient is reduced to improve efficiency, thereby achieving precise adaptation of the data acquisition cycle to changes in the vibration environment.

[0066] The above technical solution solves the problem of lacking accurate quantification and response mechanism for vibration characteristic changes in the acquisition of adjustment coefficients, and ensures high-quality acquisition of calibration data under dynamic working conditions.

[0067] Furthermore, the specific process for obtaining the adjustment coefficients includes: if the first difference is positive, the sum of 1 and the contribution weight of the average value change multiplied by the vibration characteristic variability rate is recorded as the first adjustment coefficient; if the first difference is negative, the difference between 1 and the contribution weight of the average value change is recorded as the first adjustment coefficient; if the second difference is positive, the sum of 1 and the contribution weight of the maximum value change multiplied by the vibration characteristic variability rate is recorded as the second adjustment coefficient; if the second difference is negative, the difference between 1 and the contribution weight of the maximum value change is recorded as the first adjustment coefficient; if the third difference is positive, the sum of 1 and the contribution weight of the standard deviation change multiplied by the vibration characteristic variability rate is recorded as the third adjustment coefficient; if the third difference is negative, the difference between 1 and the contribution weight of the standard deviation change is recorded as the third adjustment coefficient.

[0068] In this embodiment, the first difference refers to the difference between the current average corner vibration amplitude and the average corner vibration amplitude of the previous data acquisition period. It can be achieved through a simple numerical subtraction operation to capture the changing trend of the average vibration amplitude.

[0069] The contribution weight of the average value change refers to the relative importance of the first difference in the overall vibration characteristic change, which can be determined by normalization or weighted allocation.

[0070] Vibration characteristic variation rate refers to the degree of difference between the current vibration characteristic vector and the historical vibration characteristic vector. It can be calculated in the form of the ratio of Euclidean distance to modulus length, and aims to quantify the intensity of dynamic changes in the vibration environment.

[0071] The second difference refers to the difference between the current maximum value of the corner vibration amplitude and the maximum value of the corner vibration amplitude in the previous data acquisition period. It can also be achieved by subtracting values ​​to reflect the change in local vibration peak values.

[0072] The contribution weight of the maximum value change refers to the proportion of the second difference in the overall vibration characteristic change, which can be obtained through a similar calculation method as the contribution weight of the average value change.

[0073] The third difference refers to the difference between the standard deviation of the current period corner vibration amplitude and the standard deviation of the historical corner vibration amplitude, which is used to characterize the changing trend of vibration dispersion.

[0074] The contribution weight of standard deviation variation reflects the degree of influence of the third difference on the overall vibration characteristic change, and its calculation method can be consistent with the former two.

[0075] By clarifying the specific calculation logic of the adjustment coefficient, dynamic optimization of the data acquisition cycle is achieved. First, a first adjustment coefficient is generated based on the positive or negative characteristics of the first difference. When the first difference is positive, the sum of 1 and the contribution weight of the average change, multiplied by the vibration characteristic variability rate, is used as the first adjustment coefficient. This not only captures the trend of increasing average vibration amplitude but also amplifies the adjustment amplitude through the variability rate, thereby extending the data acquisition cycle accordingly to adapt to the environment of enhanced vibration and avoid premature termination of acquisition during periods of poor data quality. When the first difference is negative, the difference between 1 and the contribution weight of the average change is used as the first adjustment coefficient. This is based on the signal of decreasing average vibration amplitude, and shortens the acquisition cycle by reducing the adjustment coefficient, thus improving efficiency, because the system can enter a stable state more quickly when vibration decays.

[0076] A second adjustment coefficient is generated based on the positive or negative characteristics of the second difference. When the second difference is positive, the sum of 1 and the contribution weight of the maximum value change is multiplied by the vibration characteristic variation rate and recorded as the second adjustment coefficient. In response to the local risk of an increase in the maximum vibration amplitude, the adjustment effect is enhanced by combining the variation rate, and the acquisition cycle is extended to accumulate more data redundancy to prevent calibration inaccuracies caused by instantaneous peak values. When the second difference is negative, the difference between 1 and the contribution weight of the maximum value change is recorded as the first adjustment coefficient. Taking advantage of the signs of a decrease in the maximum vibration amplitude, the acquisition process is accelerated by reducing the adjustment coefficient.

[0077] The third adjustment coefficient is generated based on the positive and negative characteristics of the third difference. When the third difference is positive, the sum of 1 and the contribution weight of the standard deviation change is multiplied by the vibration characteristic variability rate and recorded as the third adjustment coefficient. This responds to the increase in vibration dispersion by dynamically adjusting the period length through the variability rate to ensure data sufficiency when vibration fluctuations intensify. When the third difference is negative, the difference between 1 and the contribution weight of the standard deviation change is recorded as the third adjustment coefficient. This is based on the characteristic of improved vibration stability, which appropriately compresses the acquisition period to avoid resource waste.

[0078] By transforming subtle changes in vibration characteristics into quantifiable adjustment parameters, the data acquisition cycle can closely match the actual vibration state, thus solving the problem of fuzzy rules for calculating adjustment coefficients and improving the robustness and adaptability of the calibration process.

[0079] Furthermore, the specific process for optimizing the dataset acquisition is as follows: For the calibration board image sequence data and the supplementary calibration board image sequence data of any image frame in the data acquisition cycle, obtain the average value of the corner vibration amplitude of all candidate corner points of the image frame as the overall vibration amplitude characterization value of the image frame; calculate the absolute value of the difference between the corner vibration frequency of all candidate corner points of the image frame and the dominant resonance frequency to obtain the absolute value set of the image frame; extract the maximum absolute value from the absolute value set as the overall resonance deviation characterization value of the image frame; and combine the overall vibration amplitude characterization value and the overall resonance deviation... The characterization values ​​are normalized, and the normalized overall vibration amplitude characterization value is used as the abscissa, and the normalized overall resonance deviation characterization value is used as the ordinate, forming a resonance feature vector in a two-dimensional coordinate system. The angle between the resonance feature vector and the positive direction of the abscissa is calculated. When the angle is within a preset preferred angle range, the calibration board image sequence data and the additional calibration board image sequence data under that image frame are retained and merged to obtain a fine-screened image set. Based on a preset sharpness threshold, image frames with image sharpness lower than the sharpness threshold are removed from the fine-screened image set to obtain an optimized dataset.

[0080] Figure 3 This is a schematic diagram of the logical flow for obtaining the optimized dataset provided in an embodiment of this application. In this embodiment, the sharpness threshold is determined through the following steps: calculating the image sharpness index using an image gradient operator; acquiring calibration board images of different sharpness under standard conditions and establishing a relationship model between the sharpness index and the corner detection accuracy; and determining the lower limit of sharpness that can guarantee the detection accuracy based on the accuracy requirements of corner detection, and setting it as the sharpness threshold.

[0081] The overall vibration amplitude characterization value refers to a quantitative index that reflects the global vibration intensity of an image frame, generated by statistical methods. It can be achieved by calculating the arithmetic mean, weighted average, or other forms of central tendency measure of the vibration amplitude of all candidate corner points, and is used to provide a stable global vibration assessment benchmark.

[0082] The overall resonance deviation characterization value can be understood as a quantitative indicator used to measure the most severe frequency deviation in an image frame. It can be achieved by extracting the absolute value of the difference between the vibration frequency of all candidate corner points and the dominant resonance frequency and selecting the maximum value. This is used to capture potential resonance risks and avoid data quality degradation caused by some corner points being close to the resonance frequency.

[0083] The resonance eigenvector is a unified vector representation formed after normalization. It can be realized by mapping the overall vibration amplitude characterization value and the overall resonance deviation characterization value to a vector in two-dimensional space. It is used to eliminate differences in different feature dimensions and scales, which facilitates subsequent screening operations based on spatial distribution characteristics.

[0084] The optimal angle range can be adjusted according to the actual application scenario. It is used to ensure that the selected image frames have both low vibration amplitude and small frequency deviation, thereby guaranteeing data quality. The optimal angle range is determined through the following steps: constructing a two-dimensional distribution map of vibration feature vectors and analyzing the calibration data quality corresponding to different angle ranges; determining the angle range corresponding to high-quality data through cluster analysis; and setting the optimal angle range based on engineering experience to ensure that the requirements of low vibration amplitude and high resonance deviation are met simultaneously.

[0085] A multi-dimensional vibration feature evaluation system was implemented to dynamically filter image frames, effectively solving the problem of unstable data quality under vibration interference. First, a global vibration intensity index was generated using the overall vibration amplitude characterization value. This index avoids interference from random fluctuations at individual corner points on the overall assessment, thus providing a reliable quantitative basis for the stability of the vibration environment. Second, the overall resonance deviation characterization value was used to focus on the most severe frequency deviation, ensuring sensitive capture of potential resonance risks and avoiding data quality degradation caused by some corner point frequencies approaching resonance.

[0086] By normalizing the vibration amplitude and frequency deviation to the same space, a resonance feature vector is formed, which enables a unified representation of multidimensional vibration features and facilitates accurate screening in the future.

[0087] By calculating the angle between the resonance feature vector and the positive direction of the horizontal axis and filtering according to the preferred angle range, we ensure that only high-quality image frames with low vibration amplitude and small frequency deviation are retained.

[0088] Low-resolution images are removed based on a sharpness threshold, and image quality degradation caused by motion blur or other factors is further filtered out, thereby improving the overall reliability of the optimized dataset.

[0089] Furthermore, the aforementioned scheme plays a crucial role in the dynamic calibration process. By combining the spatial distribution characteristics of vibration feature vectors for screening, it can not only effectively distinguish image frames with different degrees of vibration interference, but also avoid introducing low-quality data with strong vibration or high resonance risk, thereby significantly improving the calculation accuracy of calibration parameters.

[0090] The combination of this multi-dimensional vibration feature evaluation system and dynamic screening mechanism enables the calibration process to adaptively optimize the data acquisition strategy under complex vibration environments, ensuring that the best quality dataset is obtained within a limited time, thus providing high-quality input for subsequent robust estimation algorithms.

[0091] Furthermore, in the quasi-static state, the method also includes determining whether to implement a highly robust calibration strategy: when the average value of the corner vibration amplitude is less than a first preset threshold, the maximum value of the corner vibration amplitude is less than a second preset threshold, and the standard deviation of the corner vibration amplitude is less than a third preset threshold; select a predetermined number of calibration board image sequence data with the smallest vibration amplitude under the image frames from the calibration board image sequence data, and calculate the calibration parameter data using the bundle adjustment method based on the coordinate data of the candidate corner points under the corresponding image frames; when the average value of the corner vibration amplitude is less than the first preset threshold, or the maximum value of the corner vibration amplitude is less than the second preset threshold, or the standard deviation of the corner vibration amplitude is less than the third preset threshold; select a predetermined number of calibration board image sequence data from the calibration board image sequence data, extract the calibration board corner coordinate data and calibration board mesh edge contour data from the selected calibration board image sequence data, and calculate the optical flow vector data of adjacent image frames; construct a multi-source feature objective function based on the calibration board corner coordinate data, calibration board mesh edge contour data, and optical flow vector data; and calculate the calibration parameter data by minimizing the multi-source feature objective function.

[0092] In this embodiment, the high robustness calibration strategy refers to dynamically selecting different calibration methods based on the specific vibration characteristics under quasi-static conditions. This can be achieved using a dual-path decision mechanism, which designs different processing flows for highly stable and partially stable states respectively, thereby improving the calibration process's adaptability to different vibration environments.

[0093] Bundle adjustment is a technique based on geometric constraint optimization. It can obtain more accurate calibration results by iteratively optimizing the relationship between the three-dimensional spatial position of candidate corner points and camera parameters.

[0094] The construction of multi-source feature objective functions aims to integrate multiple information sources to enhance the robustness of the calibration process. It can comprehensively consider multi-dimensional information such as geometric structure, edge consistency and dynamic change trends through weighted fusion.

[0095] Under quasi-static conditions, the average, maximum, and standard deviation of the corner vibration amplitudes need to be evaluated first. When these indicators are all below their respective preset thresholds, it indicates that the system is in a state of extremely low vibration. In this case, the image frame data with the smallest vibration amplitude is selected first, and the calibration parameters are calculated using the bundle adjustment method based on the coordinate information of the candidate corner points.

[0096] By focusing on the most stable image frame data, the high precision advantage of the bundle adjustment method under ideal static conditions is fully utilized.

[0097] ,in, , Let be the rotation matrix and translation vector of the camera in the i-th frame. Represents the three-dimensional coordinates of the j-th corner point. This represents the camera's projection matrix. Let F be the observed coordinates of the j-th corner point in the i-th frame of the image, where F is the number of image frames and P is the number of corner points.

[0098] In a partially stable state, i.e. when any vibration index is below the threshold, a multi-source feature fusion method is used.

[0099] By extracting and integrating the corner coordinate data of the calibration board, the mesh edge contour data, and the optical flow vector data of adjacent image frames, a multi-source feature objective function is constructed.

[0100] This objective function effectively compensates for coordinate drift caused by vibration by comprehensively balancing the contributions of geometric constraints, edge consistency, and inter-frame motion information through a minimization process. This strategy selection mechanism based on vibration feature details avoids the shortcomings of traditional methods that simply apply a single calibration algorithm in environments with slight vibration, and achieves dynamic optimization of calibration accuracy and environmental adaptability.

[0101] The above technical solution not only solves the problem of insufficient calibration accuracy under slight vibration environment, but also significantly improves the adaptability of the calibration process to dynamically changing vibration environment, thus providing a reliable guarantee for high-precision calibration of robot vision system.

[0102] Furthermore, the specific construction process of the multi-source feature objective function is as follows: The three-dimensional spatial coordinates of the template calibration board corner points are projected onto the image plane, and the difference between these coordinates and the calibration board corner point coordinate data is processed and normalized to obtain the corner point coordinate reprojection error term; the three-dimensional edge contour of the template calibration board mesh is projected onto the image plane, and the difference between this edge contour and the calibration board mesh edge contour data is processed and normalized to obtain the edge alignment error term; the optical flow vector error term is obtained by comparing the actual motion vectors of the corner points between adjacent frames with the theoretical motion vectors predicted by the camera motion model; the weights corresponding to the corner point coordinate reprojection error term, edge alignment error term, and optical flow vector error term are obtained through fuzzy inference; and the multi-source feature objective function is obtained by weighted summation of the corner point coordinate reprojection error term, edge alignment error term, and optical flow vector error term.

[0103] In this embodiment, the corner coordinate reprojection error term refers to the deviation between the projected position of the three-dimensional spatial coordinates of the corner of the template calibration board on the image plane and the actual detected corner coordinates. It can be achieved by calculating the Euclidean distance between the two and performing normalization processing to eliminate the scale difference under different vibration amplitudes, thereby accurately reflecting the change law of corner positioning deviation in dynamic environment.

[0104] The edge alignment error term can be understood as the deviation between the projection of the three-dimensional edge contour of the template calibration board mesh onto the image plane and the actual detected edge contour. It can be normalized by extracting edge pixels and calculating the matching degree between the projected contour and the actual contour. This is used to make the edge contour more robust to image blurring caused by vibration, while ensuring that this error term is comparable to the corner error term under the same dimensions.

[0105] The optical flow vector error term refers to the deviation between the actual motion vector of the corner point between adjacent frames and the theoretical motion vector predicted by the camera motion model. It can be realized by calculating the vector difference between the two and quantifying the instantaneous displacement, and is used to verify the degree of motion consistency in the time series affected by vibration.

[0106] Fuzzy reasoning refers to the process of generating fuzzy sets based on input variables and reasoning through an expert experience rule base. It can be implemented using a fuzzy logic controller to dynamically allocate the weights of each error term according to vibration characteristics, avoiding insufficient adaptability caused by fixed weights.

[0107] By systematically constructing multi-source characteristic objective functions, the problem of inaccurate parameter estimation caused by noise interference in calibration data in vibration environments is effectively solved.

[0108] After the three-dimensional spatial coordinates of the corner points of the template calibration board are projected onto the image plane, they are processed and normalized with the actual detection coordinates to generate a reference error to reflect the changing pattern of the corner point positioning deviation.

[0109] After the 3D edge contour of the template calibration board mesh is projected onto the image plane, it undergoes difference processing and normalization with the actual contour to provide stable supplementary features to cope with interference in corner detection. Then, by comparing the actual motion vectors of corner points between adjacent frames with the theoretical motion vectors, the instantaneous displacement caused by vibration is quantified, providing a basis for dynamic compensation.

[0110] The fuzzy inference system uses vibration features as input variables and dynamically adjusts the weights of each error term based on expert experience rules, enabling the system to reduce its dependence on easily disturbed features when vibrations are strong.

[0111] The expert experience fuzzy rule base is constructed through the following steps: collecting calibration experience from domain experts under different vibration conditions; converting expert experience into fuzzy rules in IF-THEN form; verifying and optimizing the effectiveness of the rules through experimental data; and establishing a complete rule base covering various operating conditions from slight vibration to strong vibration.

[0112] By fusing multi-dimensional feature information through a weighted summation mechanism, minimizing the objective function can simultaneously optimize geometric structure, edge continuity, and motion consistency, such as... Figure 4The diagram shows the logical schematic of the iterative optimization process of the multi-source feature objective function provided in the embodiment of this application: After starting the calibration parameter optimization, the calibration parameters are initialized; then the corner reprojection error term, edge alignment error term, and optical flow vector error term are calculated; then the fuzzy inference weights are obtained; the multi-source feature objective function is constructed; it is determined whether the objective function has converged. If it has not converged, the calibration parameters are updated and the error terms are recalculated. If it has converged, the calibration parameter data is output.

[0113] The multi-source feature objective function can be expressed in the following mathematical form:

[0114] Let the corner point coordinate reprojection error term be... The edge alignment error term is The optical flow vector error term is The corresponding weights are respectively , , Then the multi-source feature objective function as follows:

[0115] ;

[0116] ;in, These are the actual detected corner coordinates. To determine the projection coordinates of the corner points of the template, This represents the number of corner points.

[0117] ;in, These are the actual coordinates of the detected edge points. To determine the projection coordinates of the grid edge of the template calibration board, This represents the number of edge points.

[0118] ;in, These are the actual motion vectors of the corner points between adjacent frames. The theoretical motion vectors predicted by the camera motion model. Let be the number of motion vectors. By minimizing the objective function, To optimize calibration parameters.

[0119] This significantly improves the robustness of calibration parameter estimation under vibration conditions. Furthermore, the proposed scheme incorporates vibration characteristics such as the average, maximum, and standard deviation of corner vibration amplitudes, and achieves adaptive weight allocation through a fuzzy logic controller. This effectively suppresses noise interference with calibration accuracy in vibration environments, thereby enhancing the reliability of calibration results.

[0120] Furthermore, the specific process of fuzzy inference is as follows: the average value, maximum value, and standard deviation of the corner vibration amplitude are used as input variables of the fuzzy logic controller; the corner coordinate reprojection error term, edge alignment error term, and optical flow vector error term are used as output variables of the fuzzy logic controller; the input variables are converted into fuzzy sets by the fuzzy logic controller and fuzzy inference is performed based on these sets using an expert experience fuzzy rule base to obtain the weights corresponding to the corner coordinate reprojection error term, edge alignment error term, and optical flow vector error term.

[0121] In this embodiment, the fuzzy logic controller refers to an intelligent control device based on fuzzy mathematics theory, which can be implemented using a multi-input multi-output fuzzy inference system to handle the uncertainty and continuous change characteristics in vibration features.

[0122] The construction of a fuzzy logic controller includes the following steps: determining the average, maximum, and standard deviation of the corner vibration amplitude and the domain range of the weights of each error term; designing the membership functions of the input and output variables, using triangular or Gaussian membership functions; establishing a fuzzy rule base based on expert experience, covering weight allocation strategies under different vibration states; selecting a defuzzification method, using the centroid method to convert the fuzzy output into precise values.

[0123] Its membership function is represented by a triangular membership function:

[0124] ;

[0125] ;

[0126] ;

[0127] An expert-experienced fuzzy rule base can be understood as a database storing domain experts' experiential knowledge about the relationship between vibration and error. It can be constructed through pre-defined rule tables or dynamic learning mechanisms, aiming to ensure that the inference process can intelligently adjust the weight allocation strategy based on subtle changes in the vibration state. The average, maximum, and standard deviation of the corner vibration amplitude are used as input variables to reflect the overall vibration level, capture extreme vibration events, and characterize the degree of vibration fluctuation, respectively, thus providing a multi-dimensional description of the vibration state.

[0128] By constructing a fuzzy inference mechanism driven by vibration features, the problem of unsuitable weight allocation of multi-source feature objective functions under dynamic vibration environments is solved.

[0129] Using the mean, maximum, and standard deviation of the corner vibration amplitude as input variables, these variables comprehensively quantify the statistical characteristics of vibration intensity. The mean avoids interference from local fluctuations, the maximum identifies key interference sources, and the standard deviation assesses stability, thus providing a multi-dimensional vibration state representation for fuzzy inference.

[0130] The corner coordinate reprojection error term, edge alignment error term, and optical flow vector error term are used as output variables and directly associated with the optimization objective of the objective function, ensuring that the weight allocation is sensitive to the differences in the different error sources. The input variables are converted into fuzzy sets through a fuzzy logic controller, and fuzzy logic is used to handle the uncertainty and continuous change characteristics of vibration features, avoiding the abrupt change problem in the gradual vibration process of traditional hard threshold decision-making.

[0131] Fuzzy reasoning is based on an expert experience fuzzy rule base. The rule base integrates domain experts’ experience knowledge on the relationship between vibration and error, enabling the reasoning process to intelligently adjust the weight allocation strategy according to subtle changes in vibration state.

[0132] The obtained weights dynamically respond to changes in the vibration environment. When the vibration is strong, the weight of the optical flow vector error term that is susceptible to vibration interference may be reduced to suppress the influence of noise, or the weight of the diagonal point coordinate reprojection error term may be increased to improve accuracy when the vibration is weak. This ensures that the multi-source feature objective function can still effectively optimize the robustness of calibration parameter calculation under dynamic vibration conditions.

[0133] Through the above technical solutions, the fuzzy inference mechanism can dynamically adjust the weight allocation strategy according to real-time vibration characteristics, which significantly improves the robustness of calibration parameter calculation, and shows stronger adaptability and reliability, especially in complex industrial environments.

[0134] like Figure 5 The diagram illustrates the logic of fuzzy inference weight calculation provided in this embodiment: First, the current vibration feature is input, and it is determined whether it is quasi-static. If so, the average, maximum, and standard deviation of the corner vibration amplitude are extracted, and fuzzification is performed to convert them into a fuzzy set. Then, the expert experience fuzzy rule base is queried for fuzzy inference. Next, defuzzification is performed, and the weights are calculated. Finally, the weights of the corner reprojection error term, the edge alignment error term, and the optical flow vector error term are output. If it is not quasi-static, the default weights are maintained.

[0135] Figure 6This is a schematic diagram of the structure of a dynamic calibration system for a robot vision system based on data fusion provided in this application embodiment. The system includes: a data acquisition module for acquiring and processing image sequence data of the calibration board and joint torque signal sequence of the target robot to obtain the corner vibration amplitude, corner vibration frequency, and dominant resonance frequency of candidate corner points; a corner judgment module for determining that the system is in a quasi-static state when the corner vibration amplitude of all candidate corner points is less than a stability judgment threshold and the difference between the corner vibration frequency and the dominant resonance frequency of any candidate corner point is greater than a preset frequency tolerance; and a quasi-static processing module for determining that, when the system is in a quasi-static state, the average value of the corner vibration amplitude is greater than or equal to a first preset threshold. When the maximum value of the corner vibration amplitude is greater than or equal to the second preset threshold, or the standard deviation of the corner vibration amplitude is greater than or equal to the third preset threshold, the adjustment coefficient is determined based on the vibration characteristics of all candidate corners and the comparison relationship of historical vibration characteristics, and the data acquisition cycle is calculated accordingly; Calibration parameter processing module: used to collect additional calibration board image sequence data according to the data acquisition cycle, extract vibration features from the collected calibration board image sequence data and additional calibration board image sequence data and construct vibration feature vectors accordingly, and perform filtering processing based on the spatial distribution characteristics of vibration feature vectors to obtain an optimized dataset; Output module: used to calculate calibration parameter data based on the optimized dataset using a robust estimation algorithm and output it to the calibration equipment of the target robot accordingly.

[0136] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0137] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0138] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0139] These computer program instructions can also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0140] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0141] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A dynamic calibration method for robot vision systems based on data fusion, characterized in that, Includes the following steps: The calibration board image sequence data and joint torque signal sequence of the target robot are acquired and processed accordingly to extract the vibration features of candidate corner points. The vibration features include corner point vibration amplitude, corner point vibration frequency and dominant resonant frequency. When the corner vibration amplitude of all candidate corner points is less than the stability judgment threshold, and the difference between the corner vibration frequency of any candidate corner point and the dominant resonance frequency is greater than the preset frequency tolerance, it is determined to be in a quasi-static state. When in a quasi-static state, when the average value of the corner vibration amplitude is greater than or equal to the first preset threshold, or the maximum value of the corner vibration amplitude is greater than or equal to the second preset threshold, or the standard deviation of the corner vibration amplitude is greater than or equal to the third preset threshold, the adjustment coefficient is determined based on the vibration characteristics of all candidate corners and the comparison relationship of historical vibration characteristics, and the data acquisition cycle is calculated accordingly. According to the data acquisition cycle, image sequence data of the additional calibration plate is collected. Vibration features are extracted from the collected calibration plate image sequence data and additional calibration plate image sequence data, and vibration feature vectors are constructed accordingly. Based on the spatial distribution characteristics of the vibration feature vectors, the dataset is filtered to obtain an optimized dataset. Based on the optimized dataset, a robust estimation algorithm is used to calculate calibration parameter data and output it to the calibration device of the target robot.

2. The dynamic calibration method for robot vision systems based on data fusion according to claim 1, characterized in that, The specific steps for obtaining the corner vibration amplitude, corner vibration frequency, and dominant resonance frequency are as follows: The target robot's vision and mechanical sensors are used to continuously collect data on the calibration plate and robot joints for a predetermined duration, respectively, to obtain the calibration plate image sequence and the robot joint torque signal sequence. Frequency domain transformation analysis was performed on the robot joint torque signal sequence to extract the dominant resonant frequency data; Corner detection is performed on the calibration board image of each image frame in the calibration board image sequence to obtain the initial two-dimensional coordinate data of multiple candidate corner points; Based on the time series in the calibration board image sequence, the initial two-dimensional coordinate data of the same candidate corner point are associated and combined to obtain the corner point coordinate time series data of each candidate corner point; Frequency domain transformation analysis is performed on the time series data of the corner coordinates to extract the vibration frequency and vibration amplitude of each candidate corner, thus obtaining the corner vibration amplitude and corner vibration frequency. The dominant resonant frequency is extracted by performing frequency domain transformation analysis on the robot joint torque signal sequence.

3. The dynamic calibration method for robot vision systems based on data fusion according to claim 1, characterized in that, The specific rules for determining the data collection period are as follows: The adjustment coefficients are obtained based on the comparison between the current vibration characteristics and the historical vibration characteristics. The adjustment coefficients include a first adjustment coefficient corresponding to the average value of the corner vibration amplitude, a second adjustment coefficient corresponding to the maximum value of the corner vibration amplitude, and a third adjustment coefficient corresponding to the standard deviation of the corner vibration amplitude. Based on the average value, maximum value, and standard deviation of the corner vibration amplitude, the data acquisition period is calculated using the following formula: ,in, Indicates the data collection period. Indicates the baseline acquisition duration. This represents the average value of the corner vibration amplitude. This represents the first preset threshold. This represents the maximum amplitude of the corner vibration. This indicates the second preset threshold. The standard deviation of the corner vibration amplitude. This indicates the third preset threshold. This represents the first adjustment factor. This represents the second adjustment factor. This represents the third adjustment factor.

4. The dynamic calibration method for robot vision systems based on data fusion according to claim 3, characterized in that, The specific process for obtaining the adjustment coefficient is as follows: The average value of the corner vibration amplitude, the maximum value of the corner vibration amplitude, and the standard deviation of the corner vibration amplitude are respectively constructed as vectors in three dimensions in a preset three-dimensional spatial coordinate system, and then vectors are synthesized to obtain the vibration feature vector; Obtain the vibration feature vector from the previous data acquisition cycle; Calculate the Euclidean distance between the current vibration feature vector and the vibration feature vector of the previous data acquisition cycle to obtain the vibration feature Euclidean distance; Calculate the sum of the magnitude of the current vibration feature vector and the magnitude of the vibration feature vector in the previous data acquisition cycle to obtain the feature magnitude sum; The ratio of the vibration feature Euclidean distance to the sum of the feature moduli is denoted as the vibration feature variation rate. When the vibration characteristic variation rate is greater than the preset variation rate threshold, the following adjustment coefficient acquisition steps are performed: The difference between the current average corner vibration amplitude and the average corner vibration amplitude of the previous data acquisition period is calculated and recorded as the first difference. The difference between the current maximum corner vibration amplitude and the maximum corner vibration amplitude of the previous data acquisition period is calculated and recorded as the second difference. The difference between the standard deviation of the corner vibration amplitude of the current period and the standard deviation of the historical corner vibration amplitude is calculated. Calculate the proportions of the absolute values ​​of the first, second, and third differences in the sum of the three, and use them as the contribution weights for the change in the mean, the change in the maximum value, and the change in the standard deviation, respectively. Based on the positive and negative characteristics of the first, second, and third differences, and combined with the corresponding change contribution weights and vibration characteristic variation rates, the first adjustment coefficient, the second adjustment coefficient, and the third adjustment coefficient are calculated respectively.

5. The dynamic calibration method for robot vision systems based on data fusion according to claim 4, characterized in that, The specific process for obtaining the adjustment coefficient also includes: If the first difference is positive, the sum of 1 and the contribution weight of the change in the average value multiplied by the vibration characteristic variability rate is recorded as the first adjustment coefficient; If the first difference is negative, then the difference between 1 and the weight of the contribution of the change in the average value is recorded as the first adjustment coefficient; If the second difference is positive, the result of multiplying the sum of 1 and the contribution weight of the maximum value change by the vibration characteristic variability rate is recorded as the second adjustment coefficient; If the second difference is negative, then the difference between 1 and the contribution weight of the change in the maximum value is recorded as the first adjustment coefficient; If the third difference is positive, the sum of 1 and the contribution weight of the standard deviation change, multiplied by the vibration characteristic variability rate, is recorded as the third adjustment coefficient. If the third difference is negative, then the difference between 1 and the contribution weight of the standard deviation change is recorded as the third adjustment coefficient.

6. The dynamic calibration method for robot vision systems based on data fusion according to claim 1, characterized in that, The specific process for obtaining the optimized dataset is as follows: For the calibration board image sequence data and the supplementary calibration board image sequence data of any image frame in the data acquisition cycle, the average value of the corner vibration amplitude of all candidate corner points of the image frame is obtained as the overall vibration amplitude characterization value of the image frame. Calculate the absolute value of the difference between the corner vibration frequency and the dominant resonance frequency of all candidate corner points of the image frame to obtain the set of absolute values ​​of the image frame; Extract the maximum absolute value from the set of absolute values ​​and use it as the overall resonance deviation representation value of the image frame. The overall vibration amplitude characterization value and the overall resonance deviation characterization value are normalized respectively. The normalized overall vibration amplitude characterization value is used as the abscissa and the normalized overall resonance deviation characterization value is used as the ordinate to form a resonance feature vector in a two-dimensional coordinate system. Calculate the angle between the resonance feature vector and the positive direction of the horizontal axis. When the angle is within a preset preferred angle range, retain the calibration board image sequence data and the additional calibration board image sequence data under that image frame, and merge them to obtain the fine screening image set. Based on a preset sharpness threshold, image frames with image sharpness lower than the sharpness threshold are removed from the finely screened image set to obtain an optimized dataset.

7. The dynamic calibration method for robot vision systems based on data fusion according to claim 1, characterized in that, In the quasi-static state, it also includes determining whether to implement a highly robust calibration strategy: When the average value of the corner vibration amplitude is less than a first preset threshold, the maximum value of the corner vibration amplitude is less than a second preset threshold, and the standard deviation of the corner vibration amplitude is less than a third preset threshold; From the calibration plate image sequence data, select a predetermined number of image frames with the smallest vibration amplitude, and calculate the calibration parameter data based on the coordinate data of the candidate corner points in the corresponding image frames using the bundle adjustment method. When the average value of the corner vibration amplitude is less than the first preset threshold, or the maximum value of the corner vibration amplitude is less than the second preset threshold, or the standard deviation of the corner vibration amplitude is less than the third preset threshold; A predetermined number of calibration board image sequence data are selected from the calibration board image sequence data. The calibration board corner coordinate data and calibration board grid edge contour data are extracted from the selected calibration board image sequence data, and the optical flow vector data of adjacent image frames are calculated. Based on the calibration board corner coordinate data, calibration board mesh edge contour data, and optical flow vector data, a multi-source feature objective function is constructed. The calibration parameter data is calculated by minimizing the multi-source feature objective function.

8. The dynamic calibration method for robot vision systems based on data fusion according to claim 7, characterized in that, The specific construction process of the multi-source feature objective function is as follows: By projecting the three-dimensional spatial coordinates of the corner points of the template calibration board onto the image plane, and performing difference processing and normalization on the coordinates of the corner points of the calibration board, the corner point coordinate reprojection error term is obtained. By projecting the 3D edge contour of the template calibration board mesh onto the image plane, and performing difference processing and normalization on the edge contour data of the calibration board mesh, the edge alignment error term is obtained. The optical flow vector error term is obtained by comparing the actual motion vectors of adjacent inter-frame corner points with the theoretical motion vectors predicted by the camera motion model. The weights corresponding to the corner coordinate reprojection error term, edge alignment error term, and optical flow vector error term are obtained through fuzzy inference. The multi-source feature objective function is obtained by weighted summation of the corner coordinate reprojection error term, the edge alignment error term, and the optical flow vector error term.

9. The dynamic calibration method for robot vision systems based on data fusion according to claim 8, characterized in that, The specific process of the fuzzy reasoning is as follows: The average value, maximum value, and standard deviation of the corner vibration amplitude are used as input variables for the fuzzy logic controller. The corner coordinate reprojection error term, edge alignment error term, and optical flow vector error term are used as output variables of the fuzzy logic controller. The input variables are converted into fuzzy sets by a fuzzy logic controller, and fuzzy inference is performed based on the fuzzy rule base of expert experience to obtain the weights corresponding to the corner coordinate reprojection error term, edge alignment error term, and optical flow vector error term.

10. A dynamic calibration system for robot vision systems based on data fusion, characterized in that, include: Data acquisition module: used to acquire the calibration board image sequence data and joint torque signal sequence of the target robot and process them to obtain the corner vibration amplitude, corner vibration frequency and dominant resonance frequency of the candidate corner points; Corner point judgment module: When the corner vibration amplitude of all candidate corner points is less than the stability judgment threshold, and the difference between the corner vibration frequency of any candidate corner point and the dominant resonance frequency is greater than the preset frequency tolerance, it is determined that the position is in a quasi-static state. Quasi-static processing module: When in a quasi-static state, if the average value of the corner vibration amplitude is greater than or equal to a first preset threshold, or the maximum value of the corner vibration amplitude is greater than or equal to a second preset threshold, or the standard deviation of the corner vibration amplitude is greater than or equal to a third preset threshold, it determines the adjustment coefficient based on the vibration characteristics of all candidate corner points and the comparison relationship of historical vibration characteristics, and calculates the data acquisition cycle accordingly. Calibration parameter processing module: It is used to collect image sequence data of the additional calibration plate according to the data acquisition cycle, extract vibration features from the collected calibration plate image sequence data and additional calibration plate image sequence data, construct vibration feature vectors based on them, and perform filtering based on the spatial distribution characteristics of the vibration feature vectors to obtain an optimized dataset; Output module: Used to calculate calibration parameter data based on the optimized dataset using a robust estimation algorithm and output the data to the calibration device of the target robot.

Images