AI visual camera external parameter calibration system and method

Abstract

The invention discloses an AI visual camera external parameter calibration system and method, and relates to the technical field of camera external parameter calibration, and the method comprises the steps: collecting environment parameters through an environment perception sensor array, and forming an environment feature vector; performing deep fusion on the environment feature vector and the image data to obtain fusion data; inputting the fused data into a multi-layer depth feature extraction network to form a high-dimensional feature matrix; predicting an optimal camera extrinsic parameter by using a camera extrinsic parameter prediction model based on deep learning and generating a feedback signal; calculating an environment adaptation coefficient according to a feedback signal to perform self-calibration on the model; and the calibrated model parameters are adopted to determine external parameters of the camera so as to realize dynamic accurate calibration. Through combination of environment perception and deep learning, high-precision dynamic calibration of external parameters of the camera is realized, the imaging accuracy and stability are improved, a closed-loop optimization system is formed, the calibration effect is continuously improved, the method adapts to complex and changeable environments, and reliable calibration technical support is provided for AI visual application.

Description

Technical Field

[0001] The present invention relates to the technical field of camera extrinsic parameter calibration, and in particular to an AI vision camera extrinsic parameter calibration system and method. Background Art

[0002] With the development of computer vision technology, extrinsic calibration of AI vision cameras has become a research hotspot. Traditional calibration methods rely primarily on manual feature extraction and fixed model calculations, such as using checkerboard patterns to extract corner points for calibration. These methods have achieved some success in stable environments, and related technologies are widely used in fields such as intelligent driving and industrial inspection, and research interest continues to grow.

[0003] However, existing technologies have many shortcomings. They cannot adapt to complex and changing environments, and changes in environmental parameters have a significant impact on the accuracy of external parameters, resulting in unstable calibration results. At the same time, traditional methods only use a single data source of images, ignoring the value of multi-source data fusion, and the information is insufficient. In addition, the lack of a feedback optimization mechanism means that the model cannot self-improve based on errors, making it difficult to meet the needs of high-precision dynamic calibration. It is impossible to achieve high-precision dynamic calibration by deeply integrating environmental and image data and introducing a feedback self-calibration mechanism, as in the present invention. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides an AI vision camera extrinsic parameter calibration method to solve the problems that the existing AI vision camera extrinsic parameter calibration method cannot adapt to complex environments, cannot make full use of multi-source data fusion, and how to achieve high-precision dynamic calibration.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, the present invention provides an AI visual camera extrinsic parameter calibration method, which is characterized by comprising: S1: Use the environmental perception sensor array to collect environmental parameters in real time and at high frequency, and form an environmental feature vector after preliminary filtering.

[0007] S2: Deeply fuse the environmental feature vector with the image data captured by the AI ​​vision camera, and accurately align them in the time and space dimensions through the feature alignment algorithm to obtain fused data.

[0008] S3: Input the fused data into a multi-layer deep feature extraction network to extract deep feature representation to form a high-dimensional feature matrix.

[0009] S4: Input the high-dimensional feature matrix into the camera extrinsic parameter prediction model based on deep learning, predict the optimal camera extrinsic parameters, and generate a feedback signal through quantitative evaluation by the feedback evaluation module.

[0010] S5: Calculate the environmental adaptation coefficient according to the feedback signal and the environmental feature vector, and perform self-calibration on the deep learning model through the coefficient to obtain calibrated model parameters.

[0011] S6: Finally determine the camera external parameters using the calibrated model parameters, realize system cyclic operation, and complete dynamic accurate calibration of the camera external parameters.

[0012] As a preferred scheme of the AI vision camera external parameter calibration method, wherein: the environmental perception sensor array is used to collect environmental parameters in real time and high frequency, and the environmental feature vector is formed after preliminary filtering, and the specific steps are, According to the preset geometric arrangement mode, install light, temperature, humidity and color temperature sensors around the AI vision camera to form an ordered sensor layout; Start the sensor array, set the high-frequency collection frequency, and each sensor real-time acquires environmental light intensity, temperature, humidity and color temperature value to generate original environmental parameter data sequence; The original environmental parameter data sequence is processed by Kalman filtering algorithm in time domain, and the filtered environmental parameter estimation value is obtained by taking each sensor data sequence as input, initializing the filter state parameter, and recursively processing through the prediction and update steps. The filtered light intensity value, temperature value, humidity value and color temperature value are integrated according to the pre-defined data structure to form the environmental feature vector.

[0013] As a preferred scheme of the AI vision camera external parameter calibration method, wherein: the environmental feature vector and the image data shot by the AI vision camera are deeply fused, and the fusion data is obtained by accurately aligning in time and space dimensions through the feature alignment algorithm, and the specific steps are, According to the trigger signal synchronization mechanism of the camera and the sensor array, ensure that the original environmental parameter data and the image data correspond in time dimension; The light intensity, temperature, humidity and color temperature values in the environmental feature vector are projected onto the corresponding positions on the image plane respectively by using the camera internal and external parameter matrix to realize spatial dimension alignment; The bilinear interpolation method is used to interpolate the distribution of environmental features in the image to make the environmental features continuously distributed on the image; The processed environmental features and image pixel values are fused pixel by pixel to obtain the fusion data.

[0014] As a preferred scheme of the AI vision camera external parameter calibration method, wherein: the fusion data is input into the multi-layer deep feature extraction network to extract deep feature representation and form a high-dimensional feature matrix, and the specific steps are, The fusion data is normalized and organized into a four-dimensional tensor to obtain input data; Use the first layer of convolutional neural network to perform convolution operation on the input data, extract local features, and pass the result to the batch normalization layer for data normalization; After the nonlinearity is introduced by the ReLU activation function, the second layer of the convolutional neural network further extracts deep local features, enhances the feature fusion effect, and performs a maximum pooling operation to reduce the spatial size of the feature map; After repeated convolution, batch normalization, activation, and pooling operations, the final feature map is flattened and input into the fully connected layer to integrate all features to form a high-dimensional feature matrix.

[0015] As a preferred solution of the AI ​​visual camera extrinsic parameter calibration method of the present invention, wherein: the high-dimensional feature matrix is ​​input into the camera extrinsic parameter prediction model based on deep learning, the optimal camera extrinsic parameter is predicted and quantitatively evaluated by the feedback evaluation module to generate a feedback signal, the specific steps are as follows: The high-dimensional feature matrix is ​​input into the LSTM-based camera extrinsic parameter prediction model. The model receives and processes the input data and captures the temporal dependency between features. The LSTM network learns long-term dependencies through internal memory units and gating mechanisms, establishes a mapping from input features to camera extrinsics, and outputs preliminary predicted camera extrinsics. The predicted external parameters are fed into the actual image data into the feedback evaluation module to generate a feedback signal containing the evaluation score and error gradient.

[0016] As a preferred solution of the AI ​​visual camera external parameter calibration method of the present invention, wherein: the environmental adaptation coefficient is calculated based on the feedback signal and the environmental feature vector, and the deep learning model is self-calibrated by the coefficient to obtain the calibrated model parameters. The specific steps are: Extract the evaluation score and error gradient based on the feedback signal and concatenate them with the environment feature vector to form a joint feature vector; The joint feature vector is input into the multiple linear regression model to calculate the current environmental adaptation coefficient; The environmental adaptation coefficient is used to adjust the weight matrix and bias term of the deep learning model, and the calibrated model parameters are output.

[0017] As a preferred solution of the AI ​​vision camera extrinsic parameter calibration method of the present invention, wherein: the calibrated model parameters are used to finally determine the camera extrinsic parameters, realize the system cycle operation, and complete the dynamic and accurate calibration of the camera extrinsic parameters. The specific steps are: Apply the calibrated model parameters to the camera extrinsic parameter prediction model, replacing the original model parameters to complete the model update; Use the updated model to process the new fusion data and predict the optimal camera extrinsics under the current environment; Apply the predicted camera extrinsics to the camera system to perform real-time correction on the camera imaging to ensure the accuracy and stability of the camera imaging; The currently predicted camera extrinsics are compared with the actual imaging results to generate new feedback signals for the feedback evaluation process, thus realizing cyclic operation, continuously optimizing the camera extrinsics, and completing dynamic and precise calibration.

[0018] In a second aspect, the present invention provides an AI visual camera extrinsic parameter calibration system, comprising: The perception filtering model uses an environmental perception sensor array to collect environmental parameters in real time and at high frequency, and forms an environmental feature vector after preliminary filtering processing.

[0019] The fusion alignment model deeply fuses the environmental feature vector with the image data captured by the AI ​​vision camera, and accurately aligns them in the time and space dimensions through the feature alignment algorithm to obtain fused data.

[0020] The feature extraction model inputs the fused data into a multi-layer deep feature extraction network to extract deep feature representations to form a high-dimensional feature matrix.

[0021] The prediction and evaluation model inputs the high-dimensional feature matrix into the camera extrinsic parameter prediction model based on deep learning, predicts the optimal camera extrinsic parameters, and generates a feedback signal through quantitative evaluation by the feedback evaluation module.

[0022] The self-calibration model calculates the environmental adaptation coefficient based on the feedback signal and the environmental feature vector, and uses the coefficient to self-calibrate the deep learning model to obtain the calibrated model parameters.

[0023] The dynamic calibration model uses the calibrated model parameters to finalize the camera extrinsic parameters, realizes the system cycle operation, and completes the dynamic and accurate calibration of the camera extrinsic parameters.

[0024] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the AI ​​visual camera extrinsic parameter calibration method as described in the first aspect of the present invention is implemented.

[0025] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the AI ​​visual camera extrinsic parameter calibration method as described in the first aspect of the present invention is implemented.

[0026] The present invention achieves the following beneficial effects: It uses an environmental perception sensor array to collect environmental parameters in real time and form environmental feature vectors. It then uses a feature alignment algorithm to deeply integrate environmental features with image data, fully exploiting multi-source data information. This significantly improves the accuracy of camera extrinsic calibration compared to traditional single-source data source calibration methods. The introduction of an LSTM-based deep learning prediction model captures the complex temporal dependencies between features and accurately predicts optimal camera extrinsics. Furthermore, a feedback evaluation module quantitatively evaluates the prediction results, and the model is self-calibrated by calculating the environmental adaptability coefficient based on the feedback signal and the environmental feature vector. This achieves dynamic model optimization, enabling the system to maintain high-precision calibration results even in complex and changing environments, effectively addressing the shortcomings of existing technologies in dynamic environmental adaptability. Furthermore, the dynamic calibration model enables system cyclic operation and continuous optimization of camera extrinsics, further improving the accuracy and stability of camera imaging. This provides more reliable calibration technology support for the application of AI vision cameras in fields such as intelligent transportation and industrial automation. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0028] Figure 1 This is a flow chart of the AI ​​visual camera extrinsic calibration method in Example 1. DETAILED DESCRIPTION

[0029] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0030] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0031] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0032] Example 1, with reference to Figure 1, is the first embodiment of the present invention, which provides an AI visual camera external parameter calibration method, characterized by: including, S1: Use the environmental perception sensor array to collect environmental parameters in real time and at high frequency, and form an environmental feature vector after preliminary filtering.

[0033] S2: Deeply fuse the environmental feature vector with the image data captured by the AI ​​vision camera, and accurately align them in the time and space dimensions through the feature alignment algorithm to obtain fused data.

[0034] S3: Input the fused data into a multi-layer deep feature extraction network to extract deep feature representation to form a high-dimensional feature matrix.

[0035] S4: Input the high-dimensional feature matrix into the camera extrinsic parameter prediction model based on deep learning, predict the optimal camera extrinsic parameters, and generate a feedback signal through quantitative evaluation by the feedback evaluation module.

[0036] S5: Calculate the environmental adaptation coefficient based on the feedback signal and the environmental feature vector, and use the coefficient to self-calibrate the deep learning model to obtain the calibrated model parameters.

[0037] S6: Use the calibrated model parameters to finalize the camera extrinsic parameters, realize the system cycle operation, and complete the dynamic and accurate calibration of the camera extrinsic parameters.

[0038] Specifically, the environmental parameters are collected in real time and at high frequency by using an environmental perception sensor array, and an environmental feature vector is formed after preliminary filtering. The specific steps are as follows: Install light, temperature, humidity, and color temperature sensors around the AI ​​vision camera in a preset geometric arrangement to form an orderly sensor layout; Start the sensor array and set the high-frequency acquisition frequency. Each sensor obtains the ambient light intensity, temperature, humidity and color temperature in real time to generate the original environmental parameter data sequence. The Kalman filter algorithm is used to perform time domain smoothing on the original environmental parameter data sequence. Each sensor data sequence is used as input to initialize the filter state parameters. After recursive processing of prediction and update steps, the filtered environmental parameter estimation value is obtained. The filtered light intensity value, temperature value, humidity value and color temperature value are integrated according to a predefined data structure to form an environmental feature vector.

[0039] It should be noted that light, temperature, humidity, and color temperature sensors are installed around the AI ​​vision camera in a pre-set geometric arrangement, forming an orderly sensor layout. This step ensures that each sensor effectively covers the camera's field of view and does not interfere with each other, providing spatial rationality for subsequent data collection. The sensor array is activated and a high-frequency acquisition frequency is set. Each sensor acquires ambient light intensity, temperature, humidity, and color temperature in real time, generating a raw environmental parameter data sequence. Each sensor operates based on its own physical principle: the light sensor is based on the photoelectric effect, the temperature sensor utilizes thermistors or semiconductor properties, the humidity sensor detects changes in capacitance or resistance, and the color temperature sensor is based on optical filter technology and a photodiode array. A high-frequency acquisition frequency of 1000 times per second is set to ensure that subtle changes in environmental parameters are captured. The raw environmental parameter data sequence is smoothed in the time domain using the Kalman filter algorithm. Each sensor data sequence is used as input to initialize the filter state parameters, including the state estimate and the estimation error covariance matrix. The data is recursively processed through the prediction and update steps to remove high-frequency noise interference, resulting in filtered environmental parameter estimates. The filtered light intensity, temperature, humidity, and color temperature values ​​are then integrated according to a predefined data structure to form an environmental feature vector. This vector has four dimensions, with each dimension corresponding to the filtered value of a specific environmental parameter. These dimensions are arranged in a fixed order to form a compact and ordered environmental feature vector.

[0040] By rationally arranging the sensor array, comprehensive coverage of environmental parameters within the camera's field of view is achieved. This organized layout ensures the integrity and validity of data acquisition, providing a solid foundation for subsequent data fusion and calibration. It enables real-time, high-frequency acquisition of environmental parameters, enabling sensitive capture of subtle environmental changes. The high acquisition frequency ensures data timeliness and accuracy, providing high-quality raw data for subsequent filtering and feature vector formation. The application of the Kalman filter algorithm effectively removes noise from the raw data, improving its purity and reliability. The filtered environmental parameter estimates are closer to the true values, providing accurate data support for subsequent feature vector formation. Integrating multiple environmental parameters into a single, ordered feature vector not only simplifies the data structure but also facilitates subsequent processing and analysis. This integrated feature vector more intuitively reflects environmental conditions and provides an efficient input format for camera extrinsic calibration.

[0041] Specifically, the environmental feature vector is deeply fused with the image data captured by the AI ​​visual camera, and the fused data is obtained by accurately aligning the image in the time and space dimensions through the feature alignment algorithm. The specific steps are as follows: According to the trigger signal synchronization mechanism of the camera and sensor array, the original environmental parameter data and the image data are ensured to correspond in the time dimension; The camera's internal and external parameter matrices are used to project the light intensity, temperature, humidity, and color temperature values ​​in the environmental feature vector to corresponding positions on the image plane to achieve spatial dimension alignment. The bilinear interpolation method is used to interpolate the distribution of environmental features in the image so that the environmental features are continuously distributed on the image. The processed environmental features are fused with the image pixel values ​​pixel by pixel to obtain fused data.

[0042] It should be noted that the trigger signal synchronization mechanism between the camera and the sensor array ensures that the raw environmental parameter data and the image data correspond in the temporal dimension. This step, through precise time synchronization, strictly aligns the environmental parameter data sequence with the image data sequence, ensuring that both are in the same time coordinate system. Through the time synchronization mechanism, precise temporal matching of the environmental parameter data and the image data is achieved. This provides a unified temporal benchmark for subsequent data fusion, effectively avoiding data mismatches caused by time differences and providing an accurate temporal data foundation for camera extrinsic calibration. Using the camera's extrinsic parameter matrix, the light intensity, temperature, humidity, and color temperature values ​​in the environmental feature vector are projected onto corresponding positions on the image plane, achieving spatial alignment. The camera's extrinsic parameter matrix plays a key role in coordinate transformation in this process, mapping three-dimensional environmental feature information onto the two-dimensional image plane. This step spatially aligns the environmental features with the image data, ensuring that each environmental parameter value accurately corresponds to a specific location on the image. This provides a spatially correlated basis for subsequent feature fusion, enabling environmental features and image features to be fused in the same spatial coordinate system. This provides accurate spatial positioning information for camera extrinsic calibration. Bilinear interpolation is used to interpolate the distribution of environmental features in the image, ensuring a continuous distribution of environmental features across the image. Bilinear interpolation smoothes the distribution of environmental features across the image plane by calculating the weighted average of adjacent pixels, filling in feature gaps caused by discrete sampling. Through bilinear interpolation, environmental features appear continuously distributed across the image. This not only enhances the expressive power of environmental features but also provides richer detail for subsequent feature fusion, ensuring that the fused data more realistically reflects the combined information of the environment and image, providing a more refined feature representation for camera extrinsic calibration. The processed environmental features are then fused pixel by pixel with the image pixel values ​​to produce fused data. During the fusion process, the value of each pixel is a weighted combination of the image pixel value at that location and the corresponding environmental feature value, ultimately forming a fused dataset containing both environmental and image features.

[0043] This step achieves a deep fusion of environmental and image features at the pixel level, fully utilizing two different types of data. The fused data not only contains the visual information of the image but also incorporates the multi-dimensional characteristics of the environment, providing a more comprehensive and richer data foundation for camera extrinsic calibration, helping to improve calibration accuracy and reliability.

[0044] Specifically, the fusion data is input into a multi-layer deep feature extraction network to extract deep feature representation to form a high-dimensional feature matrix. The specific steps are: Normalize the fused data and organize it into a four-dimensional tensor to obtain input data; Use the first layer of convolutional neural network to perform convolution operation on the input data, extract local features, and pass the result to the batch normalization layer for data normalization; After the nonlinearity is introduced by the ReLU activation function, the second layer of the convolutional neural network further extracts deep local features, enhances the feature fusion effect, and performs a maximum pooling operation to reduce the spatial size of the feature map; After repeated convolution, batch normalization, activation, and pooling operations, the final feature map is flattened and input into the fully connected layer to integrate all features to form a high-dimensional feature matrix.

[0045] It should be noted that the fused data is normalized and scaled to the range [0, 1] to eliminate dimensional differences between different features. The normalized data is then organized into a four-dimensional tensor, which contains the number of samples, image height, image width, and number of channels (including the original image's RGB channels and environmental feature channels). This format is suitable for deep learning network input. Through normalization and four-dimensional tensor organization, the data is standardized and structured. This step provides a unified data input format for subsequent deep feature extraction, effectively improving data compatibility and processing efficiency, and laying the foundation for stable model training and feature extraction. The first layer of the convolutional neural network performs a convolution operation on the input data with a kernel size of 3×3, a stride of 1, and padding of the same to maintain the size of the feature map consistent with the input image. The convolution operation linearly transforms the input data using a sliding window mechanism to extract local features. The convolution results are then passed to the batch normalization layer, which normalizes the mean and variance of each mini-batch to ensure data stability and training speed. This step initially extracts local features from the fused data, while batch normalization improves model training efficiency and stability. Local feature extraction captures fundamental image features such as edges and texture, providing the necessary information for further feature fusion and abstraction. The normalized feature map undergoes a nonlinear transformation using the ReLU activation function, setting negative values ​​to zero and retaining positive values. A second convolutional neural network layer then performs further convolution on the activated feature map to extract deeper local features and enhance feature fusion. This second convolutional layer also uses a 3×3 convolution kernel with a stride of 1 and the same padding. The ReLU activation function introduces nonlinearity, enabling the model to learn more complex feature patterns. The second convolution layer further abstracts features, extracting deeper local features. This enhances the model's ability to express features and helps improve their discriminability and representativeness. The feature map after the second convolution layer undergoes a maximum pooling operation with a 2×2 pooling window size and a stride of 2. Maximum sampling is used to reduce the spatial size of the feature map and minimize computational effort. After repeated convolution, batch normalization, activation, and pooling operations, the final feature map is flattened and fed into a fully connected layer. The number of neurons in the fully connected layer is set based on actual needs, integrating all features to form a high-dimensional feature matrix.

[0046] The max pooling operation achieves spatial dimensionality reduction in the feature map, reducing the computational effort for subsequent processing while retaining important information in the feature map and enhancing the model's robustness to image scale changes. The fully connected layer integrates all local and global depth features to form a high-dimensional feature matrix, providing a comprehensive feature representation for the prediction of camera extrinsic parameters and improving the model's prediction accuracy and generalization capabilities.

[0047] Specifically, the high-dimensional feature matrix is ​​input into the camera extrinsic parameter prediction model based on deep learning, the optimal camera extrinsic parameter is predicted and quantitatively evaluated by the feedback evaluation module to generate a feedback signal. The specific steps are as follows: The high-dimensional feature matrix is ​​input into the LSTM-based camera extrinsic parameter prediction model. The model receives and processes the input data and captures the temporal dependency between features. The LSTM network learns long-term dependencies through internal memory units and gating mechanisms, establishes a mapping from input features to camera extrinsics, and outputs preliminary predicted camera extrinsics. The predicted external parameters are fed into the actual image data into the feedback evaluation module to generate a feedback signal containing the evaluation score and error gradient.

[0048] It should be noted that the high-dimensional feature matrix is ​​input into the LSTM-based camera extrinsic parameter prediction model. After receiving the input data, the model processes the data through its internal structure. The core of the LSTM model lies in its ability to capture temporal dependencies in sequential data, which is crucial for handling the temporally continuous camera extrinsic parameter calibration task. The input high-dimensional feature matrix is ​​processed step by step in the LSTM model, which captures the temporal evolution of features by updating its internal state. By inputting the high-dimensional feature matrix into the LSTM model, the temporal dependencies of features are effectively captured. This step enables the model to understand the temporal evolution of features, providing a dynamic feature foundation for subsequent camera extrinsic parameter prediction, improving prediction accuracy and adaptability. The LSTM network learns long-term dependencies through internal memory cells and gating mechanisms. Memory cells are responsible for storing and transmitting long-term information, while the gating mechanisms (including input gates, forget gates, and output gates) control the flow of information. The input gate determines which new information is stored in the memory cell, the forget gate determines which information is forgotten, and the output gate determines which information is output. Through these mechanisms, the LSTM network effectively learns the mapping between input features and camera extrinsic parameters, outputting preliminary predicted camera extrinsic parameters. This step establishes a mapping from input features to camera extrinsic parameters through the internal mechanisms of the LSTM network. This mapping captures the long-term dependencies between features and extrinsic parameters, enabling the model to more accurately predict camera extrinsic parameters and improving the accuracy and reliability of the calibration results. The predicted extrinsic parameters are then fed into the feedback evaluation module. The feedback evaluation module quantitatively evaluates the accuracy of the prediction results by comparing the differences between the predicted extrinsic parameters and the actual image data. Specifically, the module calculates the reprojection error of image feature points under the predicted extrinsic parameters and the matching consistency of feature points under different viewpoints, generating a feedback signal consisting of an evaluation score and an error gradient. This feedback signal, generated by the feedback evaluation module, enables quantitative evaluation of the prediction results. This step not only provides an indicator of the prediction accuracy but also provides guidance for subsequent model optimization, enabling the model to self-adjust and improve based on the feedback signal, forming a closed-loop optimization system.

[0049] Specifically, the environmental adaptation coefficient is calculated based on the feedback signal and the environmental feature vector, and the deep learning model is self-calibrated by the coefficient to obtain the calibrated model parameters. The specific steps are: Extract the evaluation score and error gradient based on the feedback signal and concatenate them with the environment feature vector to form a joint feature vector; The joint feature vector is input into the multiple linear regression model to calculate the current environmental adaptation coefficient; The environmental adaptation coefficient is used to adjust the weight matrix and bias term of the deep learning model, and the calibrated model parameters are output.

[0050] It should be noted that the evaluation score and error gradient information are extracted from the feedback signal, and an environmental feature vector is obtained simultaneously. The extracted evaluation score and error gradient are concatenated with the environmental feature vector to form a new joint feature vector. During this concatenation process, the order and correspondence of the feature elements are ensured to facilitate subsequent processing. By concatenating the key information in the feedback signal with the environmental feature vector into a joint feature vector, multi-source information integration is achieved. This step provides comprehensive feature input for the subsequent calculation of the environmental adaptation coefficient, enabling the model to simultaneously consider feedback information and environmental characteristics, improving the comprehensiveness and accuracy of model calibration. The joint feature vector is then input into a multiple linear regression model. The multiple linear regression model uses the components of the joint feature vector as independent variables and the performance improvement rate of the calibrated model in historical data as the dependent variable. The regression coefficients are estimated using the least squares method, establishing a quantitative relationship between the joint feature vector and the environmental adaptation coefficient. Based on the input joint feature vector, the environmental adaptation coefficient at the current moment is calculated. The environmental adaptation coefficient is calculated using the multiple linear regression model, establishing a quantitative relationship between the feedback information, environmental characteristics, and model calibration parameters. This step accurately calculates the environmental adaptation coefficient, providing key parameter support for subsequent model calibration and enhancing the model's adaptability to environmental changes. The calculated environmental adaptation coefficient is then used to adjust the deep learning model's weight matrix and bias terms. Specifically, the environmental adaptation coefficient is element-wise multiplied by the model's current weight matrix to scale the weights. Simultaneously, the bias terms are scaled and translated based on the environmental adaptation coefficient. After these operations, the calibrated model parameters are output, completing the model's self-calibration process.

[0051] By using the environmental adaptation coefficient to self-calibrate the deep learning model, dynamic adjustment of model parameters is achieved. This step enables the model to automatically optimize its own parameters based on the current environmental characteristics and feedback information, improving the model's prediction accuracy and stability under different environmental conditions, and enhancing the model's generalization and adaptability.

[0052] Specifically, the calibrated model parameters are used to finally determine the camera extrinsic parameters, realize the system cycle operation, and complete the dynamic and accurate calibration of the camera extrinsic parameters. The specific steps are: Apply the calibrated model parameters to the camera extrinsic parameter prediction model, replacing the original model parameters to complete the model update; Use the updated model to process the new fusion data and predict the optimal camera extrinsics under the current environment; Apply the predicted camera extrinsics to the camera system to perform real-time correction on the camera imaging to ensure the accuracy and stability of the camera imaging; The currently predicted camera extrinsics are compared with the actual imaging results to generate new feedback signals for the feedback evaluation process, thus realizing cyclic operation, continuously optimizing the camera extrinsics, and completing dynamic and precise calibration.

[0053] It should be noted that the calibrated model parameters obtained in the self-calibration step are applied to the camera extrinsic parameter prediction model, replacing the original model parameters and completing the model update. This step ensures that the model can promptly reflect the latest environmental characteristics and feedback information, providing optimized parameter support for subsequent predictions. By updating the model parameters, dynamic model optimization is achieved. This step enables the model to continuously adapt to environmental changes and new data feedback, improving the model's prediction accuracy and adaptability, and ensuring the accuracy and reliability of camera extrinsic parameter prediction. The updated model is then used to process the new fused data. Through calculation and analysis, the model predicts the optimal camera extrinsic parameters for the current environment. The new fused data contains the latest environmental characteristics and image information. The model uses this data for feature extraction and analysis, outputting the camera extrinsic parameters that best match the current environmental conditions. This step achieves dynamic and accurate prediction of camera extrinsic parameters. By using the updated model parameters and the new fused data, the model can more accurately predict camera extrinsic parameters that are suitable for the current environment, improving the accuracy and stability of camera imaging and providing a reliable foundation for subsequent imaging applications. The predicted camera extrinsic parameters are applied to the camera system for real-time correction of the camera's imaging. The camera system adjusts imaging parameters based on the new extrinsic parameters, optimizing the imaging effect and ensuring imaging accuracy and stability. This improves imaging quality through real-time correction of camera imaging. This step effectively reduces imaging errors caused by inaccurate camera extrinsics, improving imaging clarity and reliability, and providing high-quality image data for subsequent image analysis and applications. The currently predicted camera extrinsics are compared with the actual imaging results to generate a new feedback signal. This feedback signal contains an evaluation score and error gradient information, which is used to feed back into the evaluation process. The system triggers the feedback evaluation module again based on the new feedback signal, implementing a loop and continuously optimizing the camera extrinsics.

[0054] This step achieves closed-loop optimization and dynamic calibration of the system. By continuously generating new feedback signals and using them to further optimize the model, the system can continuously improve the prediction accuracy of camera extrinsic parameters, adapting to environmental changes and new data inputs, forming a continuously optimizing dynamic system and ensuring the long-term accuracy and stability of camera extrinsic calibration.

[0055] This embodiment also provides an AI visual camera extrinsic parameter calibration system, including: The perception filtering model uses an environmental perception sensor array to collect environmental parameters in real time and at high frequency, and forms an environmental feature vector after preliminary filtering processing.

[0056] The fusion alignment model deeply fuses the environmental feature vector with the image data captured by the AI ​​vision camera, and accurately aligns them in the time and space dimensions through the feature alignment algorithm to obtain fused data.

[0057] The feature extraction model inputs the fused data into a multi-layer deep feature extraction network to extract deep feature representations to form a high-dimensional feature matrix.

[0058] The prediction and evaluation model inputs the high-dimensional feature matrix into the camera extrinsic parameter prediction model based on deep learning, predicts the optimal camera extrinsic parameters, and generates a feedback signal through quantitative evaluation by the feedback evaluation module.

[0059] The self-calibration model calculates the environmental adaptation coefficient based on the feedback signal and the environmental feature vector, and uses the coefficient to self-calibrate the deep learning model to obtain the calibrated model parameters.

[0060] The dynamic calibration model uses the calibrated model parameters to finalize the camera extrinsic parameters, realizes the system cycle operation, and completes the dynamic and accurate calibration of the camera extrinsic parameters.

[0061] This embodiment also provides a computer device suitable for the AI ​​vision camera extrinsic parameter calibration method, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the AI ​​vision camera extrinsic parameter calibration method proposed in the above embodiment.

[0062] The computer device may be a terminal, comprising a processor, memory, a communication interface, a display, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. The internal memory provides an environment for the operating system and computer programs stored in the non-volatile storage media. The communication interface of the computer device is used to communicate with external terminals via wired or wireless communication. Wireless communication may be achieved via Wi-Fi, a carrier network, NFC (near-field communication), or other technologies. The display of the computer device may be a liquid crystal display or an electronic ink display. The input device may be a touchscreen overlay on the display, buttons, a trackball, or a touchpad on the computer device housing, or an external keyboard, touchpad, or mouse.

[0063] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for implementing the external parameter calibration of an AI visual camera as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or optical disk.

[0064] In summary, the present invention significantly improves the accuracy of camera extrinsic calibration compared to traditional single-source data source calibration methods by: using an environmental perception sensor array to collect environmental parameters in real time and form environmental feature vectors; combining them with a feature alignment algorithm to deeply integrate environmental features with image data, fully exploiting multi-source data information. The introduction of an LSTM-based deep learning prediction model captures the complex temporal dependencies between features and accurately predicts optimal camera extrinsics. Furthermore, a feedback evaluation module quantifies the prediction results and self-calibrates the model by calculating the environmental adaptability coefficient based on the feedback signal and the environmental feature vector. This achieves dynamic model optimization, enabling the system to maintain high-precision calibration results even in complex and changing environments, effectively addressing the shortcomings of existing technologies in dynamic environmental adaptability. Furthermore, the dynamic calibration model enables system cyclic operation and continuous optimization of camera extrinsics, further improving the accuracy and stability of camera imaging. This provides more reliable calibration technology support for the application of AI vision cameras in fields such as intelligent transportation and industrial automation.

[0065] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for extrinsic parameter calibration of an AI visual camera, characterized by: include, S1: Use the environmental perception sensor array to collect environmental parameters in real time and at high frequency, and form environmental feature vectors after preliminary filtering; S2: Deeply fuse the environmental feature vector with the image data captured by the AI ​​vision camera, and accurately align them in the spatiotemporal dimensions using a feature alignment algorithm to obtain fused data; S3: Input the fused data into a multi-layer deep feature extraction network to extract deep feature representations to form a high-dimensional feature matrix; S4: Input the high-dimensional feature matrix into the camera extrinsic parameter prediction model based on deep learning, predict the optimal camera extrinsic parameters, and generate a feedback signal through quantitative evaluation by the feedback evaluation module; S5: Calculate the environmental adaptation coefficient based on the feedback signal and the environmental feature vector, and use the coefficient to self-calibrate the deep learning model to obtain the calibrated model parameters; S6: Use the calibrated model parameters to finalize the camera extrinsic parameters, realize the system cycle operation, and complete the dynamic and accurate calibration of the camera extrinsic parameters.

2. The AI ​​visual camera extrinsic parameter calibration method according to claim 1, wherein: The environmental parameters are collected in real time and at high frequency by the environment perception sensor array, and the environmental feature vector is formed after preliminary filtering. The specific steps are as follows: Install light, temperature, humidity, and color temperature sensors around the AI ​​vision camera in a preset geometric arrangement to form an orderly sensor layout; Start the sensor array and set the high-frequency acquisition frequency. Each sensor obtains the ambient light intensity, temperature, humidity and color temperature in real time to generate the original environmental parameter data sequence. The Kalman filter algorithm is used to perform time domain smoothing on the original environmental parameter data sequence. Each sensor data sequence is used as input to initialize the filter state parameters. After recursive processing of prediction and update steps, the filtered environmental parameter estimation value is obtained. The filtered light intensity value, temperature value, humidity value and color temperature value are integrated according to a predefined data structure to form an environmental feature vector.

3. The AI ​​visual camera extrinsic parameter calibration method according to claim 2, wherein: The environmental feature vector is deeply fused with the image data captured by the AI ​​visual camera, and the fused data is obtained by accurately aligning the image in the time and space dimensions through the feature alignment algorithm. The specific steps are as follows: According to the trigger signal synchronization mechanism of the camera and sensor array, the original environmental parameter data and the image data are ensured to correspond in the time dimension; The camera's internal and external parameter matrices are used to project the light intensity, temperature, humidity, and color temperature values ​​in the environmental feature vector to corresponding positions on the image plane to achieve spatial dimension alignment. The bilinear interpolation method is used to interpolate the distribution of environmental features in the image so that the environmental features are continuously distributed on the image. The processed environmental features are fused with the image pixel values ​​pixel by pixel to obtain fused data.

4. The AI ​​visual camera extrinsic parameter calibration method according to claim 3, wherein: The fusion data is input into a multi-layer deep feature extraction network to extract deep feature representation to form a high-dimensional feature matrix. The specific steps are: Normalize the fused data and organize it into a four-dimensional tensor to obtain input data; Use the first layer of convolutional neural network to perform convolution operation on the input data, extract local features, and pass the result to the batch normalization layer for data normalization; After the nonlinearity is introduced by the ReLU activation function, the second layer of the convolutional neural network further extracts deep local features, enhances the feature fusion effect, and performs a maximum pooling operation to reduce the spatial size of the feature map; After repeated convolution, batch normalization, activation, and pooling operations, the final feature map is flattened and input into the fully connected layer to integrate all features to form a high-dimensional feature matrix.

5. The AI ​​visual camera extrinsic parameter calibration method according to claim 4, wherein: The high-dimensional feature matrix is ​​input into the camera extrinsic parameter prediction model based on deep learning, the optimal camera extrinsic parameter is predicted and quantitatively evaluated by the feedback evaluation module to generate a feedback signal. The specific steps are: The high-dimensional feature matrix is ​​input into the LSTM-based camera extrinsic parameter prediction model. The model receives and processes the input data and captures the temporal dependency between features. The LSTM network learns long-term dependencies through internal memory units and gating mechanisms, establishes a mapping from input features to camera extrinsics, and outputs preliminary predicted camera extrinsics. The predicted external parameters are fed into the actual image data into the feedback evaluation module to generate a feedback signal containing the evaluation score and error gradient.

6. The AI ​​vision camera extrinsic parameter calibration method according to claim 5, wherein: The environmental adaptation coefficient is calculated based on the feedback signal and the environmental feature vector. The deep learning model is self-calibrated by the coefficient to obtain the calibrated model parameters. The specific steps are as follows: Extract the evaluation score and error gradient based on the feedback signal and concatenate them with the environment feature vector to form a joint feature vector; The joint feature vector is input into the multiple linear regression model to calculate the current environmental adaptation coefficient; The environmental adaptation coefficient is used to adjust the weight matrix and bias term of the deep learning model, and the calibrated model parameters are output.

7. The AI ​​vision camera extrinsic parameter calibration method according to claim 6, wherein: The calibrated model parameters are used to finally determine the camera extrinsic parameters, realize the system cycle operation, and complete the dynamic and accurate calibration of the camera extrinsic parameters. The specific steps are: Apply the calibrated model parameters to the camera extrinsic parameter prediction model, replacing the original model parameters to complete the model update; Use the updated model to process the new fusion data and predict the optimal camera extrinsics under the current environment; Apply the predicted camera extrinsics to the camera system to perform real-time correction on the camera imaging to ensure the accuracy and stability of the camera imaging; The currently predicted camera extrinsics are compared with the actual imaging results to generate new feedback signals for the feedback evaluation process, thus realizing cyclic operation, continuously optimizing the camera extrinsics, and completing dynamic and precise calibration.

8. An AI vision camera extrinsic parameter calibration system, based on the AI ​​vision camera extrinsic parameter calibration method according to any one of claims 1 to 7, characterized in that: include, The perception filtering model uses an environmental perception sensor array to collect environmental parameters in real time and at high frequency, and forms an environmental feature vector after preliminary filtering. The fusion alignment model deeply fuses the environmental feature vector with the image data captured by the AI ​​vision camera, and accurately aligns them in the time and space dimensions through the feature alignment algorithm to obtain fused data; Feature extraction model, which inputs the fused data into a multi-layer deep feature extraction network to extract deep feature representations to form a high-dimensional feature matrix; The prediction and evaluation model inputs the high-dimensional feature matrix into the camera extrinsic parameter prediction model based on deep learning, predicts the optimal camera extrinsic parameters, and generates a feedback signal through quantitative evaluation by the feedback evaluation module; The self-calibration model calculates the environmental adaptation coefficient based on the feedback signal and the environmental feature vector, and uses this coefficient to self-calibrate the deep learning model to obtain the calibrated model parameters; The dynamic calibration model uses the calibrated model parameters to finalize the camera extrinsic parameters, realizes the system cycle operation, and completes the dynamic and accurate calibration of the camera extrinsic parameters.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the AI ​​visual camera extrinsic parameter calibration method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the AI ​​visual camera extrinsic parameter calibration method according to any one of claims 1 to 7 are implemented.

Images