Automatic lens switching method and system based on intelligent light sensitivity test

Abstract

The present application relates to the field of lens switching technology, and discloses a lens automatic switching method and system based on intelligent light perception testing. The method includes: collecting original light field data through a three-dimensional light field sensor array and generating a three-dimensional illumination distribution map of the scene; performing forward propagation calculations through an end-to-end deep convolutional neural network to obtain a spatial transformation matrix and an optical parameter vector; performing mirror parameter adjustment and lens parameter optimization to obtain a target lens optical system; collecting target optical images and adjusting weight parameters to obtain a globally optimized end-to-end model; obtaining continuous multi-frame light field data and generating illumination change prediction data; inputting illumination change prediction data into a globally optimized end-to-end model to perform lens switching prediction, obtaining a lens switching prediction result and generating a lens parameter adjustment strategy to achieve automatic lens switching. The present application achieves automatic lens switching through intelligent light perception testing, thereby improving the quality of the captured image.

Description

Technical Field

[0001] The present application relates to the technical field of lens switching, and in particular to an automatic lens switching method and system based on intelligent light perception testing. Background Art

[0002] With the rapid development of smart cameras and computational photography technology, the automatic lens switching function plays an increasingly important role in improving shooting effects and user experience. Traditional lens switching methods mainly rely on preset scene modes or simple light sensing, which is difficult to cope with complex and changing shooting environments, often resulting in untimely or inaccurate switching. At the same time, lens switching algorithms usually separate light sensing, image analysis, and lens control, lacking integrity and synergy, and making it difficult to fully utilize multi-source information to achieve optimal lens parameter configuration.

[0003] Most existing automatic lens switching methods are limited to static scenes and are difficult to adapt to dynamically changing shooting environments. How to predict the dynamic changes in scene lighting and switch lenses accordingly is an urgent problem to be solved. Summary of the invention

[0004] The present application provides a lens automatic switching method and system based on intelligent light sensitivity testing. The present application realizes automatic lens switching through intelligent light sensitivity testing, thereby improving the quality of captured images.

[0005] In a first aspect, the present application provides a lens automatic switching method based on intelligent light perception testing, and the lens automatic switching method based on intelligent light perception testing includes:

[0006] Collecting raw light field data through a three-dimensional light field sensor array and generating a three-dimensional illumination distribution map of the scene;

[0007] Inputting the three-dimensional illumination distribution map of the scene into a preset end-to-end deep convolutional neural network for forward propagation calculation to obtain a spatial transformation matrix and an optical parameter vector;

[0008] According to the space transformation matrix and the optical parameter vector, mirror parameter adjustment and lens parameter optimization are performed on the initial lens optical system to obtain a target lens optical system;

[0009] The target optical image is collected through the target lens optical system, and a global optimization loss function is calculated to adjust the weight parameters of the end-to-end deep convolutional neural network to obtain a globally optimized end-to-end model;

[0010] Acquire multiple frames of continuous light field data and calculate the motion vectors between adjacent frames using the optical flow method to generate illumination change prediction data;

[0011] The illumination change prediction data is input into the globally optimized end-to-end model to perform lens switching prediction, obtain a lens switching prediction result and generate a lens parameter adjustment strategy to achieve automatic lens switching.

[0012] In a second aspect, the present application provides an automatic lens switching system based on intelligent light perception testing, and the automatic lens switching system based on intelligent light perception testing includes:

[0013] An acquisition module, used to acquire raw light field data through a three-dimensional light field sensor array and generate a three-dimensional illumination distribution map of the scene;

[0014] A calculation module, used for inputting the three-dimensional illumination distribution map of the scene into a preset end-to-end deep convolutional neural network for forward propagation calculation to obtain a spatial transformation matrix and an optical parameter vector;

[0015] An adjustment module, used for adjusting mirror parameters and optimizing lens parameters of an initial lens optical system according to the space transformation matrix and the optical parameter vector, so as to obtain a target lens optical system;

[0016] An adjustment module, used to collect a target optical image through the target lens optical system, and calculate a global optimization loss function to adjust weight parameters of the end-to-end deep convolutional neural network to obtain a globally optimized end-to-end model;

[0017] A generation module is used to obtain continuous multi-frame light field data and calculate the motion vectors between adjacent frames through the optical flow method to generate illumination change prediction data;

[0018] The switching module is used to input the illumination change prediction data into the globally optimized end-to-end model to perform lens switching prediction, obtain the lens switching prediction result and generate a lens parameter adjustment strategy to achieve automatic lens switching.

[0019] In the technical solution provided by the present application, the original light field data is collected by a three-dimensional light field sensor array, which can more accurately capture and represent the three-dimensional illumination distribution of the scene. The end-to-end deep convolutional neural network is used for forward propagation calculation. Through the spatial transformation prediction subnetwork and the optical parameter prediction subnetwork, the joint optimization of the spatial transformation matrix and the optical parameter vector is realized, and the accuracy and flexibility of the lens parameter adjustment are improved. The multi-objective optimization and parameter coupling relationship model are introduced to adjust the mirror parameters and optimize the lens parameters of the initial lens optical system, which can better balance the various optical performance indicators and obtain a better target lens optical system. Combined with the generative adversarial network and the global optimization loss function, the weight parameters of the end-to-end deep convolutional neural network are adjusted to achieve closed-loop optimization of lens switching and image quality evaluation. The motion vector between adjacent frames is calculated by the optical flow method. Combined with the light field propagation model, the three-dimensional illumination distribution at the next moment can be effectively predicted, which improves the robustness and reliability of the switching decision. The real-time and accuracy of the lens parameter adjustment strategy are ensured, and the automatic lens switching is realized, thereby improving the quality of the collected image. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0021] Figure 1 A schematic diagram of an embodiment of a lens automatic switching method based on intelligent light perception testing in an embodiment of the present application;

[0022] Figure 2 This is a schematic diagram of an embodiment of an automatic lens switching system based on intelligent light sensitivity testing in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The embodiments of the present application provide a lens automatic switching method and system based on intelligent light perception testing. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 In the embodiments of the present application, an embodiment of the automatic lens switching method based on intelligent light perception testing includes:

[0025] Step 101: Collecting original light field data through a three-dimensional light field sensor array and generating a three-dimensional illumination distribution map of the scene;

[0026] It is understandable that the execution subject of the present application may be a lens automatic switching system based on intelligent light perception test, or a terminal or a server, which is not limited here. The present application embodiment is described by taking the server as the execution subject as an example.

[0027] Specifically, each light field sensor in the three-dimensional light field sensor array is calibrated, and the intrinsic parameter matrix and distortion coefficient of each sensor are obtained through calibration experiments and calculations. These parameters are used in the subsequent distortion correction process. The original light field data is distorted by using the calibrated intrinsic parameter matrix and distortion coefficient to eliminate the image distortion caused by the inherent defects of the sensor, and the corrected light field data is obtained. The corrected light field data is decoded by four-dimensional light field, and the light field data is converted into a four-dimensional light field function. Through the algorithm and decoding process, a four-dimensional light field function that can describe the light information in all directions in the light field is obtained. According to the four-dimensional light field function, the depth map of the scene is calculated. The initial depth estimation result may have noise and error. The initial depth estimation result is subjected to multi-scale edge-preserving filtering. This filtering method can remove noise while maintaining edge details, and obtain a more accurate and optimized depth map. Ray tracing is performed according to the optimized depth map and four-dimensional light field function, and the three-dimensional space light distribution is generated by simulating the propagation path of light in three-dimensional space. The three-dimensional space light distribution is voxelized, and the continuous light distribution is discretized into a three-dimensional light intensity field. Voxelization is to divide the three-dimensional space into small voxel units, and the light intensity in each voxel unit is calculated and distributed through the light distribution data. The discretized three-dimensional light intensity field is processed by a three-dimensional convolutional neural network. Through the deep learning model, the light distribution information is extracted and optimized to obtain a continuous three-dimensional light distribution. According to the continuous three-dimensional light distribution, the light gradient field is calculated, the light change characteristics are extracted, and the light change feature map is generated. The light gradient field reflects the change trend and direction of the light intensity, which helps the automatic switching of the lens. The light change feature map is feature fused with the continuous three-dimensional light distribution. Through the fusion process, the static information and change characteristics of the light distribution are organically combined to obtain the three-dimensional light distribution map of the scene.

[0028] Step 102: input the three-dimensional illumination distribution map of the scene into a preset end-to-end deep convolutional neural network for forward propagation calculation to obtain a spatial transformation matrix and an optical parameter vector;

[0029] Specifically, a multi-scale pyramid decomposition is performed on the three-dimensional illumination distribution map of the scene to obtain a set of feature maps of different scales, and a spatially adaptive convolution operation is performed on each feature map in the feature map set to obtain an initial feature map. Through multi-scale pyramid decomposition, illumination features of different scales and levels in the scene are captured. A spatially adaptive convolution operation is performed on the multi-scale feature map, and the size and shape of the convolution kernel are adjusted so that the convolution operation can adapt to the feature changes at different spatial positions, thereby obtaining a more accurate initial feature map. An attention mechanism module is constructed based on the initial feature map. The attention mechanism highlights important features and suppresses irrelevant information by learning a weighted method, and performs weighted processing on the feature map to obtain an attention-enhanced feature representation. The attention-enhanced feature representation can better reflect the key illumination information in the scene. Residual connections and skip connections are performed on the attention-enhanced feature representation to construct a multi-layer feature fusion network and obtain a multi-scale feature fusion result. The introduction of residual connections and skip connections enables the network to better retain feature information at different levels, promote multi-level fusion of features, and improve the richness and robustness of feature representation. On this basis, a spatial transformation prediction subnetwork is constructed based on the multi-scale feature fusion results, and the spatial transformation parameters are output through the fully connected layer to obtain the initial transformation matrix. In order to ensure the effectiveness and stability of the spatial transformation matrix, the initial transformation matrix is ​​subjected to singular value decomposition. The decomposed singular value matrix can effectively eliminate possible numerical instability and obtain a more reliable spatial transformation matrix. The spatial transformation matrix is ​​used to describe the displacement and rotation of the lens in three-dimensional space, providing a basis for subsequent lens adjustment. At the same time, an optical parameter prediction subnetwork is constructed based on the multi-scale feature fusion results, and the optical parameters are output through the convolution layer and the fully connected layer to obtain the initial parameter vector. The optical parameter prediction subnetwork is specifically used to extract parameters related to the optical characteristics of the lens. The initial parameter vector is regularized. The regularization method can prevent parameter overfitting, enhance the generalization ability of the model, and finally obtain the optical parameter vector.

[0030] Step 103: adjusting mirror parameters and optimizing lens parameters of the initial lens optical system according to the space transformation matrix and the optical parameter vector to obtain a target lens optical system;

[0031] Specifically, the optical path of the initial lens optical system is geometrically transformed through the spatial transformation matrix, so that the optical path model can accurately reflect the light propagation path in the actual scene. The transformed optical path model is subjected to ray tracing simulation, and the light propagation path and the light spot distribution on the imaging surface are obtained by simulating the light propagation path in the lens system. The ray tracing simulation can accurately depict the refraction, reflection and diffraction behavior of light in the lens system, and obtain the distribution of the light spot on the imaging surface. Based on the light propagation path and the light spot distribution on the imaging surface, the aberration function is calculated to obtain the system aberration evaluation index. The system aberration evaluation index can quantify the imaging quality of the lens system and reflect the degree of image distortion caused by the optical defects of the lens system. According to the system aberration evaluation index, the initial lens optical system is multi-objective optimized, and the various parameters of the lens system are adjusted through the optimization algorithm to obtain the mirror adjustment coefficient. The multi-objective optimization process comprehensively considers multiple indicators such as aberration, focal length, and field of view, aiming to improve the comprehensive performance of the lens system. The mirror adjustment coefficient is the key to adjusting the parameters of the lens optical system. By reasonably adjusting the mirror, the aberration can be effectively corrected and the imaging quality can be improved. At the same time, the lens components are parameter mapped according to the optical parameter vector to obtain the lens benchmark parameters. The optical parameter vector contains the basic optical parameters of the lens system, such as focal length, aperture, and field of view. These parameters are converted into specific lens benchmark parameters through the mapping process. A sensitivity analysis is performed on the lens benchmark parameters to identify the parameters that have a greater impact on the imaging quality, and a parameter coupling relationship model is constructed based on this. The parameter coupling relationship model describes the interdependence between various optical parameters. The parameters of the lens components are adjusted according to the mirror adjustment coefficient and the parameter coupling relationship model. By comprehensively considering the coupling relationship between the mirror adjustment coefficient and the optical parameters, the various parameters of the lens system are adjusted to obtain the target lens optical system.

[0032] Step 104: collect the target optical image through the target lens optical system, and calculate the global optimization loss function to adjust the weight parameters of the end-to-end deep convolutional neural network to obtain a globally optimized end-to-end model;

[0033] Specifically, multispectral imaging is performed on the optical system of the target lens to obtain multispectral image data, and the lens modulation transfer function is calculated based on the multispectral image data. Through multispectral imaging, light information of different wavelengths is captured to comprehensively evaluate the optical performance and imaging effect of the lens. The lens modulation transfer function is Fourier transformed to obtain the point spread function, and the Rayleigh criterion and Strehl ratio of the lens are calculated based on the point spread function to obtain the resolution and imaging quality index. The Rayleigh criterion and Strehl ratio are important indicators for evaluating the resolution and imaging quality of the lens. These indicators can be used to judge the imaging ability of the lens under different spectra. Edge detection is performed on the multispectral image data to obtain an edge sharpness map, and local contrast and global contrast are calculated based on the edge sharpness map to obtain an image clarity index. Edge detection can highlight the detailed features in the image, and local contrast and global contrast are used to measure the clarity and contrast of the image. These indicators are used to comprehensively evaluate the visual quality of the image. The resolution and imaging quality index and the image clarity index are combined into an image quality evaluation vector, which is input into the pre-trained generative adversarial network, and back-propagated through the discriminator of the generative adversarial network to obtain the image quality loss gradient. Generative adversarial networks can accurately capture quality defects in images through feedback from the discriminator and provide accurate loss gradients for optimization. Based on the image quality loss gradient and the output of the end-to-end deep convolutional neural network, a global optimization loss function is constructed, and the global optimization loss function is minimized through the stochastic gradient descent algorithm. The weight parameters of the end-to-end deep convolutional neural network are updated to obtain a globally optimized end-to-end model. The stochastic gradient descent algorithm is an efficient optimization method that iteratively updates the weight parameters of the neural network to gradually approach the minimum value of the global optimization loss function, thereby improving the overall performance of the model.

[0034] Step 105, acquiring multiple frames of continuous light field data and calculating motion vectors between adjacent frames by optical flow method to generate illumination change prediction data;

[0035] Specifically, by sampling multiple frames of continuous light field data in time series, the illumination information and dynamic changes of the scene at different times are captured. The adjacent frames in the light field data sequence are feature decomposed to obtain a feature map sequence, and the brightness constant constraint and the gradient constant constraint are calculated based on the feature map sequence to obtain the optical flow equation. The purpose of feature decomposition is to extract key features in the light field data, while the brightness constant constraint and the gradient constant constraint are used to ensure that the features remain consistent during the motion process, thereby constructing an accurate optical flow equation. The optical flow equation is solved by variational method to obtain a pixel-level optical flow field, and the pixel motion vectors between adjacent frames are calculated by variational method. The motion vectors are clustered based on the pixel-level optical flow field to obtain the moving targets in the scene. Motion vector clustering can classify pixels with similar motion features into one category, thereby identifying the moving targets in the scene. The moving targets in the scene are tracked by Kalman filtering to obtain the target motion trajectory. Kalman filtering is an effective target tracking algorithm that obtains a smoother and more accurate motion trajectory by continuously updating the target state. After obtaining the target motion trajectory, a light field propagation model is constructed based on the target motion trajectory and the current frame light field data, and a light field propagation matrix is ​​generated. The light field propagation model is used to describe the propagation law of light field data in a time series, and the light field propagation matrix is ​​calculated based on the model and used to perform time extrapolation on the current frame light field data. The light field propagation matrix is ​​used to perform time extrapolation on the current frame light field data to obtain the light field prediction for the next moment. Time extrapolation can predict the light field data at future moments and provide more timely lighting information. The three-dimensional lighting distribution is reconstructed based on the light field prediction for the next moment and the corresponding lighting change prediction data is calculated. The reconstructed three-dimensional lighting distribution can accurately reflect the lighting conditions of the scene at future moments, and the lighting change prediction data is used as the decision basis for automatic lens switching.

[0036] Step 106: Input the illumination change prediction data into the globally optimized end-to-end model to perform lens switching prediction, obtain the lens switching prediction result and generate a lens parameter adjustment strategy to achieve automatic lens switching.

[0037] Specifically, through feature extraction technology, key illumination change feature vectors are extracted from illumination change prediction data. These feature vectors can effectively reflect the dynamic characteristics of illumination changes in the scene. The extracted illumination change feature vectors are input into the global optimization end-to-end model. Through the calculation and analysis of the model, the lens switching prediction result is obtained. This prediction result shows the possibility and necessity of lens switching under the current illumination conditions. The lens switching prediction result is subjected to probability distribution reasoning to obtain the lens switching probability distribution, and the expected risk is calculated according to the lens switching probability distribution to obtain the optimal lens switching decision. Probability distribution reasoning can convert the prediction result into the probability distribution of lens switching. By calculating the expected risk, the potential risks of different switching strategies are evaluated, so as to select the optimal lens switching decision with the lowest risk. The optimal lens switching decision is discretized to obtain a discretized lens parameter adjustment instruction, and a lens parameter adjustment strategy is generated according to the discretized lens parameter adjustment instruction. Continuous decisions are converted into specific operation instructions, so as to facilitate actual execution. The lens parameter adjustment strategy is subjected to dynamic constraint analysis to generate a pulse sequence of the lens drive motor, and the pulse sequence is subjected to closed-loop feedback control to obtain a real-time lens position feedback signal. Dynamic constraint analysis can ensure that the generated pulse sequence conforms to the kinematic characteristics of the lens system, thereby avoiding system instability caused by too fast or too slow parameter adjustment. Through closed-loop feedback control, the lens position is monitored and adjusted in real time to ensure that the lens can be accurately moved to the predetermined position. The corresponding automatic lens switching result is generated based on the real-time lens position feedback signal. The real-time lens position feedback signal provides accurate information about the current state of the lens. Based on this information, the final automatic lens switching result is generated to ensure the accuracy and real-time performance of the lens switching.

[0038] In the embodiment of the present application, the three-dimensional light field sensor array is used to collect raw light field data, which can more accurately capture and represent the three-dimensional illumination distribution of the scene. The end-to-end deep convolutional neural network is used for forward propagation calculation. Through the spatial transformation prediction subnetwork and the optical parameter prediction subnetwork, the joint optimization of the spatial transformation matrix and the optical parameter vector is realized, and the accuracy and flexibility of the lens parameter adjustment are improved. The multi-objective optimization and parameter coupling relationship model are introduced to adjust the mirror parameters and optimize the lens parameters of the initial lens optical system, which can better balance the various optical performance indicators and obtain a better target lens optical system. Combined with the generative adversarial network and the global optimization loss function, the weight parameters of the end-to-end deep convolutional neural network are adjusted to achieve closed-loop optimization of lens switching and image quality evaluation. The motion vector between adjacent frames is calculated by the optical flow method. Combined with the light field propagation model, the three-dimensional illumination distribution at the next moment can be effectively predicted, which improves the robustness and reliability of the switching decision. The real-time and accuracy of the lens parameter adjustment strategy are ensured, and automatic lens switching is achieved, thereby improving the quality of the captured image.

[0039] In a specific embodiment, the process of executing step 101 may specifically include the following steps:

[0040] (1) Calibrate each light field sensor in the three-dimensional light field sensor array to obtain the intrinsic parameter matrix and distortion coefficient of each light field sensor, and collect raw light field data through the three-dimensional light field sensor array;

[0041] (2) Performing distortion correction on the original light field data according to the internal parameter matrix and the distortion coefficient to obtain corrected light field data;

[0042] (3) Performing four-dimensional light field decoding on the corrected light field data to obtain a four-dimensional light field function, and calculating a scene depth map based on the four-dimensional light field function to obtain an initial depth estimation result;

[0043] (4) Perform multi-scale edge-preserving filtering on the initial depth estimation result to obtain an optimized depth map, and perform ray tracing based on the optimized depth map and the four-dimensional light field function to obtain the three-dimensional spatial light distribution;

[0044] (5) Voxelize the three-dimensional light distribution to obtain a discretized three-dimensional light intensity field, and perform three-dimensional convolutional neural network processing on the three-dimensional light intensity field to obtain a continuous three-dimensional light distribution;

[0045] (6) Calculate the illumination gradient field based on the continuous three-dimensional illumination distribution to obtain an illumination change feature map, and then fuse the illumination change feature map with the continuous three-dimensional illumination distribution to obtain a three-dimensional illumination distribution map of the scene.

[0046] Specifically, each light field sensor in the three-dimensional light field sensor array is calibrated to obtain the intrinsic parameter matrix and distortion coefficient of each sensor. The intrinsic parameter matrix describes the internal parameters of the camera, including focal length, principal point coordinates, and pixel scaling factor, while the distortion coefficient describes the image distortion caused by the physical properties of the lens. The calibration process usually takes a series of calibration plate images of known structures and uses computer vision algorithms to calculate the intrinsic parameter matrix and distortion coefficients. The raw light field data is collected through a three-dimensional light field sensor array. These data usually contain light information captured by the light field sensor at different viewing angles and positions. The raw light field data is usually affected by sensor distortion, which is corrected for distortion. According to the intrinsic parameter matrix and distortion coefficient obtained by calibration, the following formula is used to correct the distortion of the raw light field data:

[0047] ;

[0048] in, are the corrected image coordinates, is the internal parameter matrix, are the original image coordinates, is the distortion coefficient, is the radius of radial distortion. The corrected light field data is decoded in four dimensions and converted into a four-dimensional light field function, which can describe the propagation of light in different directions and positions. The four-dimensional light field function is obtained through the decoding algorithm. Based on the four-dimensional light field function, the depth map of the scene is calculated, and the depth map describes the distance from each point in the scene to the sensor. A common method for calculating the depth map is to use phase coherent imaging technology, the basic principle of which is to estimate the depth by analyzing the phase information in the light field data. After obtaining the initial depth estimation result, the depth map may not be smooth and accurate enough due to noise and measurement errors. The initial depth estimation result is subjected to multi-scale edge-preserving filtering. Multi-scale edge-preserving filtering can smooth noise while retaining edge details. Through this filtering technology, an optimized depth map is obtained. Ray tracing is performed based on the optimized depth map and the four-dimensional light field function. Ray tracing is a technology that simulates the process of light propagation. It can calculate the propagation path of light in three-dimensional space and generate light distribution in three-dimensional space. The three-dimensional light distribution is voxelized, the continuous three-dimensional light distribution is discretized into a three-dimensional light intensity field, the three-dimensional space is divided into small cubic units (voxels), and the light intensity in each voxel is calculated. The three-dimensional light intensity field after voxelization can be processed by a three-dimensional convolutional neural network. The three-dimensional convolutional neural network is a deep learning model that can automatically extract and learn the characteristics of the light distribution. Through the processing of this network, a continuous three-dimensional light distribution is obtained. The light gradient field is calculated based on the continuous three-dimensional light distribution. The light gradient field describes the change of light intensity in three-dimensional space and reflects the gradient and direction of light. The calculation formula of the light gradient field is:

[0049] ;

[0050] in, is the illumination gradient field, is the gradient of light intensity. By calculating the light gradient field, we get the light change feature map. The light change feature map combines the information of light intensity and gradient, and can reflect the changing trend of light in the scene. We fuse the light change feature map with the continuous three-dimensional light distribution, and combine the light intensity and the change information to get a more comprehensive three-dimensional light distribution map of the scene.

[0051] In a specific embodiment, the process of executing step 102 may specifically include the following steps:

[0052] (1) Perform multi-scale pyramid decomposition on the three-dimensional illumination distribution map of the scene to obtain a set of feature maps of different scales, and perform a spatially adaptive convolution operation on each feature map in the feature map set to obtain an initial feature map;

[0053] (2) Construct an attention mechanism module based on the initial feature map, perform weighted processing on the feature map to obtain the attention-enhanced feature representation, and perform residual connection and skip connection on the attention-enhanced feature representation to construct a multi-layer feature fusion network to obtain a multi-scale feature fusion result;

[0054] (3) Based on the multi-scale feature fusion results, a spatial transformation prediction subnetwork is constructed, and the spatial transformation parameters are output through the fully connected layer to obtain the initial transformation matrix. The initial transformation matrix is ​​then subjected to singular value decomposition to obtain the spatial transformation matrix.

[0055] (4) Based on the multi-scale feature fusion results, an optical parameter prediction subnetwork is constructed. The optical parameters are output through the convolutional layer and the fully connected layer to obtain the initial parameter vector. The initial parameter vector is regularized to obtain the optical parameter vector.

[0056] Specifically, the three-dimensional illumination distribution map is subjected to multi-scale pyramid decomposition to obtain a set of feature maps of different scales. Multi-scale pyramid decomposition is a technology used to process images and light field data. By decomposing the original image into images of different resolutions and scales, different levels and detail information in the image can be captured. The process of multi-scale pyramid decomposition can be achieved by downsampling and filtering layer by layer to obtain multiple images with gradually decreasing resolutions, namely, a set of feature maps. These feature map sets can reflect the illumination characteristics of the scene at different scales. A spatially adaptive convolution operation is performed on each feature map in the feature map set to obtain an initial feature map. Spatially adaptive convolution is a convolution operation that can adaptively adjust the convolution kernel according to the spatial changes of the image, so as to better capture the local features and details in the image. The specific formula of spatially adaptive convolution is as follows:

[0057] ;

[0058] in, is the output feature map at position The value of is the input image at position The value of is the weight of the adaptive convolution kernel, is half the size of the convolution kernel. Through adaptive convolution operations, the initial feature map of each feature map is obtained. According to the initial feature map, the attention mechanism module is constructed. The attention mechanism is a technology that can perform weighted processing according to the importance of different parts of the input data. In this way, key features are highlighted and irrelevant information is suppressed. The attention mechanism module performs weighted processing on the feature map to obtain an attention-enhanced feature representation. The attention-enhanced feature representation can better reflect the important lighting information in the scene. On this basis, the attention-enhanced feature representation is subjected to residual connection and jump connection to construct a multi-layer feature fusion network. Residual connection and jump connection are a method of directly adding input and output, which helps to alleviate the gradient vanishing problem in deep networks and can better retain the detail information in feature maps. In this way, multi-scale feature fusion results are obtained. The spatial transformation prediction subnetwork is constructed based on the multi-scale feature fusion results. The spatial transformation prediction subnetwork outputs spatial transformation parameters through the fully connected layer. These parameters are used to describe the displacement and rotation of the lens in three-dimensional space. The output of spatial transformation parameters can be expressed by the following formula:

[0059] ;

[0060] in, is the spatial transformation parameter, is the weight matrix of the fully connected layer, is the result of multi-scale feature fusion. is the bias vector. In this way, the initial transformation matrix is ​​obtained. Perform singular value decomposition on the initial transformation matrix. Singular value decomposition is a matrix decomposition technique that decomposes the matrix into the product of several sub-matrices to ensure the stability and orthogonality of the transformation matrix, and finally obtains the spatial transformation matrix. At the same time, an optical parameter prediction subnetwork is constructed based on the multi-scale feature fusion results. The optical parameter prediction subnetwork outputs optical parameters through convolutional layers and fully connected layers. These parameters are used to describe the optical properties of the lens, such as focal length and aperture. The output of optical parameters can be expressed by the following formula:

[0061] ;

[0062] in, is the optical parameter, is the weight matrix of the fully connected layer, is the result of multi-scale feature fusion. is the bias vector. After obtaining the initial parameter vector, in order to ensure the rationality and effectiveness of the parameters, the initial parameter vector is regularized. Regularization is a method of adding a regularization term to the objective function to prevent parameter overfitting, enhance the generalization ability of the model, and finally obtain the optical parameter vector.

[0063] In a specific embodiment, the process of executing step 103 may specifically include the following steps:

[0064] (1) Performing geometric transformation on the optical path of the initial lens optical system according to the spatial transformation matrix to obtain a transformed optical path model;

[0065] (2) Perform ray tracing simulation on the transformed optical path model to obtain the light propagation path and the light spot distribution on the imaging surface, and calculate the aberration function based on the light propagation path and the light spot distribution on the imaging surface to obtain the system aberration evaluation index;

[0066] (3) Perform multi-objective optimization on the initial lens optical system according to the system aberration evaluation index to obtain the mirror adjustment coefficient;

[0067] (4) Map the parameters of the lens components according to the optical parameter vector to obtain the lens benchmark parameters, perform sensitivity analysis on the lens benchmark parameters, and construct a parameter coupling relationship model;

[0068] (5) Adjust the parameters of the lens assembly according to the mirror adjustment coefficient and the parameter coupling relationship model to obtain the target lens optical system.

[0069] Specifically, the optical path of the initial lens optical system is geometrically transformed according to the spatial transformation matrix. The spatial transformation matrix defines the position and direction changes of the lens in three-dimensional space. Assume that the spatial transformation matrix is , which represents the transformation of the lens system from the initial position to the target position. By applying The optical path is transformed to obtain the transformed optical path model. The formula of geometric transformation can be expressed as:

[0070] ;

[0071] in, are the coordinates of the points in the initial light path model, is the coordinate of the transformed point, and T is the space transformation matrix. Perform ray tracing simulation on the transformed optical path model to simulate the propagation path of light in the lens system. Ray tracing simulation is a technology that calculates the propagation path of light and the distribution of light spots on the imaging surface by simulating the reflection, refraction and diffraction behavior of light. Through ray tracing, the propagation path of light in the lens system and the final distribution of light spots on the imaging surface are obtained. Based on these data, the aberration function is calculated. The aberration function is used to evaluate the optical aberration in the lens system, that is, the degree of image distortion. The formula of the aberration function is:

[0072] ;

[0073] in, is the aberration value, Is the ideal light position, is the actual light position, is the total number of rays. According to the calculated aberration function, the aberration of the system is evaluated to obtain the system aberration evaluation index. According to the system aberration evaluation index, the initial lens optical system is multi-objective optimized. Multi-objective optimization is a process of optimizing the parameters of the lens by comprehensively considering multiple performance indicators to improve the overall performance. Through the optimization algorithm, the mirror adjustment coefficient is obtained, and the mirror adjustment coefficient is used to adjust the optical surface of the lens to reduce aberration and improve the imaging quality. At the same time, the lens component is parameter mapped according to the optical parameter vector. The optical parameter vector contains the basic optical parameters of the lens, such as focal length, aperture, field of view, etc. Through parameter mapping, the optical parameter vector is converted into specific lens benchmark parameters. The benchmark parameters are used to describe the initial state of the lens component and are the basis for optimization and adjustment. Sensitivity analysis is performed on the lens benchmark parameters to identify the parameters that have a greater impact on the imaging quality. Sensitivity analysis is a method to evaluate the degree of influence of each parameter on the system performance. Through this analysis, it is determined which parameters are the focus of optimization. Based on the results of sensitivity analysis, a parameter coupling relationship model is constructed. The parameter coupling relationship model describes the interdependence between various optical parameters and provides a theoretical basis for parameter optimization. The lens assembly parameters are adjusted according to the mirror adjustment coefficient and parameter coupling relationship model. By comprehensively considering the coupling relationship between the mirror adjustment coefficient and the optical parameters, the various parameters of the lens system are adjusted to obtain the target lens optical system.

[0074] In a specific embodiment, the process of executing step 104 may specifically include the following steps:

[0075] (1) Perform multispectral imaging on the target lens optical system to obtain multispectral image data, and calculate the lens modulation transfer function based on the multispectral image data;

[0076] (2) Perform Fourier transform on the lens modulation transfer function to obtain the point spread function, and calculate the Rayleigh criterion and Strehl ratio of the lens based on the point spread function to obtain the resolution and imaging quality indicators;

[0077] (3) Perform edge detection on the multispectral image data to obtain an edge sharpness map, and calculate the local contrast and global contrast based on the edge sharpness map to obtain the image clarity index;

[0078] (4) The resolution, imaging quality index, and image clarity index are combined into an image quality assessment vector, which is input into the pre-trained generative adversarial network and back-propagated through the discriminator of the generative adversarial network to obtain the image quality loss gradient;

[0079] (5) Based on the image quality loss gradient and the output of the end-to-end deep convolutional neural network, a global optimization loss function is constructed. The global optimization loss function is minimized through the stochastic gradient descent algorithm, and the weight parameters of the end-to-end deep convolutional neural network are updated to obtain a globally optimized end-to-end model.

[0080] Specifically, a multispectral imaging system is configured, which can capture spectral data in different bands from ultraviolet to infrared. Through multispectral imaging, image data of the scene in different bands is obtained. The modulation transfer function (MTF) of the lens is calculated based on the multispectral image data. MTF is an important parameter to describe the imaging quality of the optical system, which reflects the system's ability to transmit different spatial frequencies. The specific calculation method can be based on the contrast change of the multispectral image. After calculating the MTF, it is Fourier transformed to obtain the point spread function (PSF). PSF describes the light intensity distribution of a point light source on the imaging plane after passing through the optical system, which is another key performance indicator of the optical system. The Fourier transform formula is as follows:

[0081] ;

[0082] in, is the point spread function, is the Fourier transform result of the modulation transfer function, Represents the inverse Fourier transform. The Rayleigh criterion and Strehl ratio of the lens are calculated based on the point spread function. The Rayleigh criterion is used to evaluate the resolution of the optical system. According to the Rayleigh criterion, if the center distance of the PSF of two point light sources on the imaging plane is equal to the half-peak width of the PSF, the two point light sources can be resolved. The Strehl ratio is an important indicator for evaluating the imaging quality. It is the ratio of the PSF peak of the actual optical system to the PSF peak of the ideal optical system. The resolution and imaging quality indicators can be comprehensively evaluated by these two parameters. Edge detection is performed on the multispectral image data to obtain an edge sharpness map. By detecting the sharp changes in pixel brightness in the image, the details and edge information in the image can be highlighted. Commonly used edge detection algorithms include the Sobel operator, the Canny operator, etc. The edge sharpness map can reflect the edge clarity of the image by fusing the edge detection results of the multispectral image. Based on the edge sharpness map, the local contrast and global contrast are calculated to obtain the image clarity index. The local contrast refers to the contrast in a local area in the image, while the global contrast is the contrast of the entire image. The resolution and imaging quality indicators, as well as the image clarity indicators, are combined into an image quality assessment vector and input into a pre-trained generative adversarial network. The generative adversarial network consists of a generator and a discriminator. Through adversarial training, the generator is able to produce high-quality images that are close to reality, while the discriminator is used to evaluate image quality. The image quality assessment vector is input into the discriminator, and the image quality loss gradient is obtained through back propagation. The image quality loss gradient reflects the gap between the current image quality and the ideal image quality. Based on the image quality loss gradient and the output of the end-to-end deep convolutional neural network, a global optimization loss function is constructed. The purpose of the global optimization loss function is to comprehensively consider various quality indicators of the image, and optimize the weight parameters of the deep convolutional neural network by minimizing this loss function. The stochastic gradient descent algorithm is a commonly used optimization method that iteratively updates the weight parameters so that the loss function is gradually reduced. The specific formula is:

[0083] ;

[0084] in, It is The weight of the iteration, is the learning rate, is the gradient of the loss function.

[0085] In a specific embodiment, the process of executing step 105 may specifically include the following steps:

[0086] (1) Acquire multiple frames of continuous light field data, and perform time series sampling on the multiple frames of continuous light field data to obtain a light field data sequence;

[0087] (2) Feature decomposition is performed on adjacent frames in the light field data sequence to obtain a feature map sequence, and brightness constancy constraints and gradient constrains are calculated based on the feature map sequence to obtain the optical flow equation;

[0088] (3) Perform variational solution on the optical flow equation to obtain the optical flow field at the pixel level, and perform motion vector clustering based on the optical flow field at the pixel level to obtain the moving target in the scene;

[0089] (4) Perform Kalman filter tracking on the moving target in the scene to obtain the target motion trajectory, and build a light field propagation model and generate a light field propagation matrix based on the target motion trajectory and the current frame light field data;

[0090] (5) The light field propagation matrix is ​​used to perform time extrapolation on the light field data of the current frame to obtain the light field prediction for the next moment. At the same time, the three-dimensional illumination distribution is reconstructed based on the light field prediction for the next moment and the corresponding illumination change prediction data is calculated.

[0091] Specifically, multiple frames of continuous light field data are obtained. These data can be captured by a light field camera or other multi-view imaging devices, and contain the lighting and geometric information of the scene at different times. Through time series sampling, a representative light field data sequence is extracted from multiple frames of continuous light field data. Adjacent frames in the light field data sequence are feature decomposed to extract key features in the light field data, such as edges, textures, and lighting changes. Through feature decomposition, complex light field data is simplified into a more easily processable feature map sequence. According to the feature map sequence, the brightness constancy constraint and the gradient constancy constraint are calculated to establish the optical flow equation. The brightness constancy constraint assumes that the surface brightness of objects in the scene remains unchanged during motion, while the gradient constancy constraint assumes that the gradient of the image remains unchanged during motion. The optical flow equation can be expressed as:

[0092] ;

[0093] in, is the time derivative of the image, is the optical flow vector, is the spatial gradient of the image. The optical flow equation is solved by variational method to obtain the optical flow field at the pixel level. Variational solution is a technique for solving the optical flow equation by optimization method, and the optical flow vector is solved by minimizing the energy function. The obtained optical flow field reflects the motion information of each pixel in the scene. Motion vector clustering is performed according to the pixel-level optical flow field, and pixels with similar motion characteristics are clustered together to identify the moving targets in the scene. This is achieved through clustering algorithms such as K-means or DBSCAN, and the clustering results are used for further target tracking. Kalman filtering is performed on the moving targets in the scene to obtain the target motion trajectory. Kalman filtering is a recursive least squares algorithm that can estimate the state of a dynamic system in the presence of noise and uncertainty. Kalman filtering is iteratively performed through two steps of prediction and update, and finally a smooth trajectory of the target motion is obtained. The specific prediction and update formulas of Kalman filtering are:

[0094] ;

[0095] ;

[0096] ;

[0097] ;

[0098] ;

[0099] in, is the predicted state, is the update status, is the prediction covariance, is the updated covariance, is the Kalman gain, is the state transition matrix, is the control matrix, is the control vector, is the process noise covariance, is the observation matrix, is the observation noise covariance, is the observation vector. According to the target motion trajectory and the current frame light field data, a light field propagation model is constructed and a light field propagation matrix is ​​generated. The light field propagation model is used to describe the changing law of light field data in a time series, and the light field propagation matrix is ​​calculated based on the model and used to perform time extrapolation on the current frame light field data. The light field propagation matrix can be used to predict the light field data at the next moment. The specific formula is:

[0100] ;

[0101] in, is the light field data at the next moment, is the light field data at the current moment, is the light field propagation matrix. The light field propagation matrix is ​​used to perform time extrapolation on the light field data of the current frame to obtain the light field prediction for the next moment. Time extrapolation can predict the light field data at future moments, thereby providing more timely lighting information. Based on the light field prediction for the next moment, the three-dimensional lighting distribution is reconstructed and the corresponding lighting change prediction data is calculated. The reconstructed three-dimensional lighting distribution can accurately reflect the lighting conditions of the scene at future moments, and the lighting change prediction data is used as the decision basis for automatic lens switching.

[0102] In a specific embodiment, the process of executing step 106 may specifically include the following steps:

[0103] (1) Extract features from the illumination change prediction data to obtain an illumination change feature vector, and input the illumination change feature vector into a globally optimized end-to-end model to obtain a shot switching prediction result;

[0104] (2) Perform probability distribution inference on the shot switching prediction results to obtain the shot switching probability distribution, and calculate the expected risk based on the shot switching probability distribution to obtain the optimal shot switching decision;

[0105] (3) Discretizing the optimal lens switching decision to obtain a discrete lens parameter adjustment instruction, and generating a lens parameter adjustment strategy based on the discrete lens parameter adjustment instruction;

[0106] (4) Perform dynamic constraint analysis on the lens parameter adjustment strategy, generate a pulse sequence for the lens drive motor, and perform closed-loop feedback control on the pulse sequence to obtain a real-time lens position feedback signal;

[0107] (5) Generate the corresponding lens automatic switching result based on the real-time lens position feedback signal.

[0108] Specifically, feature extraction is performed on the illumination change prediction data. The illumination change prediction data may include information such as illumination intensity, illumination direction, and illumination change rate. Through feature extraction methods, key illumination change feature vectors are extracted from these data. Illumination change feature vectors can be extracted by a variety of methods, such as using convolutional neural networks to extract image features, or using principal component analysis to reduce dimensions to obtain the most representative feature vectors. The extracted illumination change feature vectors are input into a globally optimized end-to-end model for shot switching prediction. The end-to-end model can be a deep learning model, such as a long short-term memory network or a transformer model, which outputs a shot switching prediction result by learning the relationship between illumination change features and shot switching. After obtaining the shot switching prediction result, probability distribution inference is performed on it. By analyzing the probability distribution of the prediction results, the probability of each possible switching operation is calculated. Assume that the probability distribution of the shot switching is ,in Indicates different possible operations for camera switching. is the illumination change feature vector. According to the probability distribution of lens switching, the expected risk can be calculated. The expected risk reflects the potential risk of each switching operation. The formula is:

[0109] ;

[0110] in, is the expected risk, It is a switching operation Risk function. By minimizing the expected risk, the optimal lens switching decision is obtained. The optimal lens switching decision is discretized to obtain a discretized lens parameter adjustment instruction. The continuous optimal decision is converted into a specific operation instruction so that it can be actually applied to the lens control system. Assuming that the discretized lens parameter adjustment instruction is p, the lens parameter adjustment strategy needs to be generated next. The lens parameter adjustment strategy is based on these adjustment instructions and specifies how the lens is adjusted to achieve switching. Perform dynamic constraint analysis on the lens parameter adjustment strategy. Dynamic constraint analysis is to ensure that the adjustment strategy is feasible in actual operation and does not exceed the capabilities of the lens system by analyzing the physical and kinematic characteristics of the lens system. Move according to the dynamic adjustment strategy. The specific pulse sequence can be calculated by the following formula:

[0111] ;

[0112] in, is the control signal, is the reference signal, is the output signal, , , They are proportional, differential and integral gains respectively. Through closed-loop feedback control, the pulse sequence is adjusted in real time to ensure that the lens can move accurately according to the predetermined strategy. The basic principle of closed-loop feedback control is to monitor the position and state of the lens in real time, compare the actual output with the reference input, and adjust the control signal according to the deviation. Using a PID controller, the stability and response speed of the control system are ensured by adjusting the proportional, integral and differential gains. The real-time lens position feedback signal is obtained from the encoder or other position sensor of the lens drive motor. These feedback signals reflect the current state of the lens. According to the real-time lens position feedback signal, the corresponding lens automatic switching result is generated. The lens automatic switching result is the final output control command, which guides the lens to switch operations in different scenarios.

[0113] The above describes the automatic lens switching method based on intelligent light perception test in the embodiment of the present application. The following describes the automatic lens switching system based on intelligent light perception test in the embodiment of the present application. Figure 2 In the embodiment of the present application, an embodiment of the automatic lens switching system based on intelligent light perception test includes:

[0114] The acquisition module 201 is used to acquire raw light field data through a three-dimensional light field sensor array and generate a three-dimensional illumination distribution map of the scene;

[0115] A calculation module 202 is used to input the three-dimensional illumination distribution map of the scene into a preset end-to-end deep convolutional neural network for forward propagation calculation to obtain a spatial transformation matrix and an optical parameter vector;

[0116] An adjustment module 203 is used to adjust mirror parameters and optimize lens parameters of the initial lens optical system according to the space transformation matrix and the optical parameter vector to obtain a target lens optical system;

[0117] An adjustment module 204 is used to collect a target optical image through a target lens optical system, and calculate a global optimization loss function to adjust weight parameters of an end-to-end deep convolutional neural network to obtain a globally optimized end-to-end model;

[0118] A generation module 205 is used to obtain multiple frames of continuous light field data and calculate the motion vectors between adjacent frames by using an optical flow method to generate illumination change prediction data;

[0119] The switching module 206 is used to input the illumination change prediction data into the globally optimized end-to-end model to perform shot switching prediction, obtain the shot switching prediction result and generate a shot parameter adjustment strategy to achieve automatic shot switching.

[0120] Through the collaboration of the above components, the three-dimensional light field sensor array is used to collect raw light field data, which can more accurately capture and represent the three-dimensional illumination distribution of the scene. The end-to-end deep convolutional neural network is used for forward propagation calculation. Through the spatial transformation prediction sub-network and the optical parameter prediction sub-network, the joint optimization of the spatial transformation matrix and the optical parameter vector is realized, and the accuracy and flexibility of the lens parameter adjustment are improved. The introduction of multi-objective optimization and parameter coupling relationship model to adjust the mirror parameters and optimize the lens parameters of the initial lens optical system can better balance various optical performance indicators and obtain a better target lens optical system. Combined with the generative adversarial network and the global optimization loss function, the weight parameters of the end-to-end deep convolutional neural network are adjusted to achieve closed-loop optimization of lens switching and image quality evaluation. The motion vector between adjacent frames is calculated by the optical flow method. Combined with the light field propagation model, the three-dimensional illumination distribution at the next moment can be effectively predicted, which improves the robustness and reliability of the switching decision. The real-time and accuracy of the lens parameter adjustment strategy are ensured, and the automatic lens switching is realized, thereby improving the quality of the collected image.

[0121] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0122] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0123] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A lens automatic switching method based on intelligent light perception test, characterized in that: The method comprises: Collecting raw light field data through a three-dimensional light field sensor array and generating a three-dimensional illumination distribution map of the scene; Inputting the three-dimensional illumination distribution map of the scene into a preset end-to-end deep convolutional neural network for forward propagation calculation to obtain a spatial transformation matrix and an optical parameter vector; According to the space transformation matrix and the optical parameter vector, mirror parameter adjustment and lens parameter optimization are performed on the initial lens optical system to obtain a target lens optical system; The target optical image is collected through the target lens optical system, and a global optimization loss function is calculated to adjust the weight parameters of the end-to-end deep convolutional neural network to obtain a globally optimized end-to-end model; Acquire multiple frames of continuous light field data and calculate the motion vectors between adjacent frames using the optical flow method to generate illumination change prediction data; The illumination change prediction data is input into the globally optimized end-to-end model to perform lens switching prediction, obtain a lens switching prediction result and generate a lens parameter adjustment strategy to achieve automatic lens switching.

2. The automatic lens switching method based on intelligent light perception test according to claim 1, characterized in that: The method of collecting raw light field data through a three-dimensional light field sensor array and generating a three-dimensional illumination distribution map of the scene includes: Calibrate each light field sensor in the three-dimensional light field sensor array to obtain an internal parameter matrix and a distortion coefficient of each light field sensor, and collect raw light field data through the three-dimensional light field sensor array; Performing distortion correction on the original light field data according to the intrinsic parameter matrix and the distortion coefficient to obtain corrected light field data; Performing four-dimensional light field decoding on the corrected light field data to obtain a four-dimensional light field function, and calculating a scene depth map according to the four-dimensional light field function to obtain an initial depth estimation result; Performing multi-scale edge-preserving filtering on the initial depth estimation result to obtain an optimized depth map, and performing ray tracing according to the optimized depth map and the four-dimensional light field function to obtain a three-dimensional spatial light distribution; voxelize the three-dimensional spatial light distribution to obtain a discretized three-dimensional light intensity field, and perform three-dimensional convolutional neural network processing on the three-dimensional light intensity field to obtain a continuous three-dimensional light distribution; The illumination gradient field is calculated according to the continuous three-dimensional illumination distribution to obtain an illumination change feature map, and the illumination change feature map is feature-fused with the continuous three-dimensional illumination distribution to obtain a scene three-dimensional illumination distribution map.

3. The automatic lens switching method based on intelligent light perception test according to claim 1, characterized in that: The step of adjusting mirror parameters and optimizing lens parameters of the initial lens optical system according to the spatial transformation matrix and the optical parameter vector to obtain a target lens optical system comprises: Performing geometric transformation on the optical path of the initial lens optical system according to the spatial transformation matrix to obtain a transformed optical path model; Performing ray tracing simulation on the transformed optical path model to obtain the light propagation path and the light spot distribution on the imaging surface, and calculating the aberration function according to the light propagation path and the light spot distribution on the imaging surface to obtain the system aberration evaluation index; Performing multi-objective optimization on the initial lens optical system according to the system aberration evaluation index to obtain a mirror adjustment coefficient; Performing parameter mapping on the lens assembly according to the optical parameter vector to obtain lens reference parameters, and performing sensitivity analysis on the lens reference parameters to construct a parameter coupling relationship model; The parameters of the lens assembly are adjusted according to the mirror adjustment coefficient and the parameter coupling relationship model to obtain a target lens optical system.

4. The automatic lens switching method based on intelligent light perception test according to claim 1, characterized in that: The target optical image is collected by the target lens optical system, and the global optimization loss function is calculated to adjust the weight parameters of the end-to-end deep convolutional neural network to obtain a globally optimized end-to-end model, including: Performing multispectral imaging on the target lens optical system to obtain multispectral image data, and calculating the lens modulation transfer function according to the multispectral image data; Performing Fourier transformation on the modulation transfer function of the lens to obtain a point spread function, and calculating the Rayleigh criterion and Strehl ratio of the lens according to the point spread function to obtain resolution and imaging quality indicators; Performing edge detection on the multispectral image data to obtain an edge sharpness map, and calculating local contrast and global contrast according to the edge sharpness map to obtain an image clarity index; The resolution and imaging quality index and the image clarity index are combined into an image quality assessment vector, which is input into a pre-trained generative adversarial network, and back-propagated through the discriminator of the generative adversarial network to obtain an image quality loss gradient; According to the image quality loss gradient and the output of the end-to-end deep convolutional neural network, a global optimization loss function is constructed, and the global optimization loss function is minimized by a stochastic gradient descent algorithm, and the weight parameters of the end-to-end deep convolutional neural network are updated to obtain a globally optimized end-to-end model.

5. The automatic lens switching method based on intelligent light perception test according to claim 1, characterized in that: The method of acquiring multiple frames of continuous light field data and calculating motion vectors between adjacent frames by an optical flow method to generate illumination change prediction data includes: Acquire a plurality of continuous frames of light field data, and perform time series sampling on the plurality of continuous frames of light field data to obtain a light field data sequence; Performing feature decomposition on adjacent frames in the light field data sequence to obtain a feature map sequence, and calculating brightness constancy constraints and gradient constancy constraints according to the feature map sequence to obtain an optical flow equation; Performing variational solution on the optical flow equation to obtain an optical flow field at the pixel level, and performing motion vector clustering according to the optical flow field at the pixel level to obtain a moving target in the scene; Performing Kalman filtering tracking on a moving target in the scene to obtain a target motion trajectory, and constructing a light field propagation model and generating a light field propagation matrix based on the target motion trajectory and current frame light field data; The light field propagation matrix is ​​used to perform time extrapolation on the light field data of the current frame to obtain a light field prediction for the next moment. At the same time, three-dimensional illumination distribution is reconstructed according to the light field prediction for the next moment and corresponding illumination change prediction data is calculated.

6. The automatic lens switching method based on intelligent light perception test according to claim 1, characterized in that: The step of inputting the illumination change prediction data into the globally optimized end-to-end model to perform shot switching prediction, obtaining a shot switching prediction result and generating a shot parameter adjustment strategy to achieve automatic shot switching includes: Extracting features from the illumination change prediction data to obtain an illumination change feature vector, and inputting the illumination change feature vector into a globally optimized end-to-end model to obtain a shot switching prediction result; Performing probability distribution reasoning on the shot switching prediction result to obtain a shot switching probability distribution, and calculating the expected risk according to the shot switching probability distribution to obtain an optimal shot switching decision; Discretizing the optimal lens switching decision to obtain a discretized lens parameter adjustment instruction, and generating a lens parameter adjustment strategy according to the discretized lens parameter adjustment instruction; Performing dynamic constraint analysis on the lens parameter adjustment strategy, generating a pulse sequence of the lens drive motor, and performing closed-loop feedback control on the pulse sequence to obtain a real-time lens position feedback signal; A corresponding automatic lens switching result is generated according to the real-time lens position feedback signal.

7. An automatic lens switching system based on intelligent light perception test, used to execute the automatic lens switching method based on intelligent light perception test as claimed in any one of claims 1 to 6, characterized in that: The system comprises: An acquisition module, used to acquire raw light field data through a three-dimensional light field sensor array and generate a three-dimensional illumination distribution map of the scene; A calculation module, used for inputting the three-dimensional illumination distribution map of the scene into a preset end-to-end deep convolutional neural network for forward propagation calculation to obtain a spatial transformation matrix and an optical parameter vector; An adjustment module, used for adjusting mirror parameters and optimizing lens parameters of an initial lens optical system according to the space transformation matrix and the optical parameter vector, so as to obtain a target lens optical system; An adjustment module, used to collect a target optical image through the target lens optical system, and calculate a global optimization loss function to adjust weight parameters of the end-to-end deep convolutional neural network to obtain a globally optimized end-to-end model; A generation module is used to obtain continuous multi-frame light field data and calculate the motion vectors between adjacent frames through the optical flow method to generate illumination change prediction data; The switching module is used to input the illumination change prediction data into the globally optimized end-to-end model to perform lens switching prediction, obtain the lens switching prediction result and generate a lens parameter adjustment strategy to achieve automatic lens switching.

Images