Image acquisition parameter prediction model training method and image acquisition parameter adjusting method

Abstract

The invention discloses an image acquisition parameter prediction model training method and an image acquisition parameter adjusting method, which are applied to the field of industrial visual inspection, and the method comprises the following steps: acquiring a sample image set acquired from different detection targets under multiple groups of sample parameters including hardware acquisition parameters and software preprocessing parameters; obtaining a detection performance index of the sample image set under each group of sample parameters based on the defect detection model, and determining the sample parameter with the optimal detection performance index as the optimal sample parameter; taking the sample parameter and a sample image set collected under the sample parameter as a training sample, and constructing a training sample label based on the offset between the sample parameter and the optimal sample parameter; and training a visual Transform prediction model based on the training samples and the training sample labels to obtain a trained image acquisition parameter prediction model. Through joint prediction and collaborative optimization of hardware acquisition parameters and software preprocessing parameters, the detection accuracy under different defect detection tasks is improved.

Description

Technical Field

[0001] This invention relates to the field of industrial visual inspection, and in particular to an image acquisition parameter prediction model training method, an image acquisition parameter adjustment method, an image acquisition parameter prediction model training device, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Industrial vision systems are commonly used for the automatic identification of product appearance defects on production lines. Image quality directly determines detection accuracy; therefore, it is necessary to adjust the hardware parameters of the image acquisition equipment, while simultaneously using software algorithms for image enhancement to improve image details. Existing systems typically optimize image brightness without considering optimal detection performance. Furthermore, hardware and software parameter adjustments are handled by different modules, resulting in high manual debugging costs and poor generalization. Traditional parameter adjustments generally employ a uniform brightness adjustment strategy, treating all defect detection tasks the same and lacking adaptability to different detection schemes and specific defect types. Ultimately, this leads to poor image quality for model detection and low defect detection accuracy. Summary of the Invention

[0003] The purpose of this invention is to provide an image acquisition parameter prediction model training method, an image acquisition parameter adjustment method, an image acquisition parameter prediction model training device, an electronic device, and a computer-readable storage medium, which are applied in the field of industrial visual inspection. This method proposes an adaptive adjustment scheme for image acquisition parameters based on a visual Transformer network to achieve joint prediction and collaborative optimization of hardware acquisition parameters and software preprocessing parameters, thereby improving the detection accuracy under different defect detection tasks.

[0004] To address the aforementioned technical problems, this invention provides a method for training an image acquisition parameter prediction model, comprising:

[0005] Sample images acquired under multiple sets of sample parameters for different detection targets are obtained to construct a sample image set for the sample parameters; the sample parameters include hardware acquisition parameters and software preprocessing parameters of the images.

[0006] Based on the defect detection model, the detection performance index of the sample image set under each set of sample parameters is obtained, and the sample parameter with the largest detection performance index is determined as the optimal sample parameter.

[0007] The sample parameters and the set of sample images collected under the sample parameters are used as training samples, and training sample labels are constructed based on the offset between the sample parameters and the optimal sample parameters.

[0008] A visual Transformer prediction model is trained based on the training samples and the training sample labels to obtain a trained image acquisition parameter prediction model.

[0009] Optionally, a visual Transformer prediction model is trained based on the training samples and the training sample labels to obtain a trained image acquisition parameter prediction model, including:

[0010] The training samples and their labels are input into the visual Transformer prediction model to obtain the prediction offset and prediction performance index of the model output.

[0011] The loss term is determined based on the predicted offset and the predicted performance index; the loss term includes mean square error loss and weighted multi-task loss, the weighted multi-task loss being a weighted loss of hardware acquisition parameter loss and software preprocessing parameter loss;

[0012] The model loss value is determined based on the loss term and the regularization term, and the model parameters of the visual Transformer prediction model are updated based on the model loss value until the trained image acquisition parameter prediction model is obtained.

[0013] Optionally, constructing training sample labels based on the offset between the sample parameters and the optimal sample parameters includes:

[0014] The offset between the sample parameters and the optimal sample parameters is determined as the first training sample label of the training sample;

[0015] The detection performance index of the sample image set is determined as the second training sample label of the training samples.

[0016] Optionally, the method further includes:

[0017] When the manually input performance index of the sample image set is received, the manually input performance index is determined as the third training sample label of the training sample.

[0018] Alternatively, upon receiving the manually input performance index of the sample image set, the second training sample label of the training sample is determined based on the detection performance index of the sample image set and the manually input performance index.

[0019] Optionally, the method further includes:

[0020] When the incremental learning dataset is obtained, the model parameters of the image acquisition parameter prediction model are updated based on the incremental learning dataset to obtain the image acquisition parameter prediction model after the incremental learning is completed.

[0021] When the transfer learning dataset is obtained, the model parameters of the image acquisition parameter prediction model are updated based on the transfer learning dataset to obtain the image acquisition parameter prediction model after the transfer learning is completed.

[0022] Optionally, the detection performance index is the mAP index.

[0023] To solve the above technical problems, the present invention provides an image acquisition parameter adjustment method, comprising:

[0024] Acquire a set of target images acquired under target parameters; the target parameters include hardware acquisition parameters and software preprocessing parameters of the images;

[0025] The target parameters and the target image set are input into the image acquisition parameter prediction model to obtain the output prediction result;

[0026] Based on the prediction results, parameter adjustment values ​​are determined, and the hardware acquisition parameters of the image acquisition device and the software preprocessing parameters of the image preprocessing software are adjusted based on the parameter adjustment values.

[0027] The image acquisition parameter prediction model is the model trained by the image acquisition parameter prediction model training method described above.

[0028] To solve the above-mentioned technical problems, the present invention provides an image acquisition parameter prediction model training device, comprising:

[0029] The first module is used to acquire sample images of different detection targets under multiple sets of sample parameters, so as to construct a sample image set of the sample parameters; the sample parameters include hardware acquisition parameters and software preprocessing parameters of the images.

[0030] The second module is used to obtain the detection performance index of the sample image set under each set of sample parameters based on the defect detection model, and to determine the sample parameter with the largest detection performance index as the optimal sample parameter.

[0031] The third module is used to take the sample parameters and the set of sample images collected under the sample parameters as training samples, and to construct training sample labels based on the offset between the sample parameters and the optimal sample parameters.

[0032] The fourth module is used to train a visual Transformer prediction model based on the training samples and the training sample labels, so as to obtain a trained image acquisition parameter prediction model.

[0033] To solve the above-mentioned technical problems, the present invention provides an electronic device, comprising:

[0034] Memory, used to store computer programs;

[0035] A processor is configured to implement, when executing the computer program, the image acquisition parameter prediction model training method as described above, or the image acquisition parameter adjustment method as described above.

[0036] To address the aforementioned technical problems, the present invention provides a computer-readable storage medium storing computer-executable instructions. When executed by a processor, the computer-executable instructions implement the image acquisition parameter prediction model training method or the image acquisition parameter adjustment method as described above.

[0037] As can be seen, this invention constructs a sample image set of sample parameters by acquiring sample images of different detection targets under multiple sets of sample parameters; the sample parameters include hardware acquisition parameters and software preprocessing parameters of the images; based on the defect detection model, the detection performance index of the sample image set under each set of sample parameters is obtained, and the sample parameters with the best detection performance index are determined as the optimal sample parameters; the sample parameters and the sample image set acquired under the sample parameters are used as training samples, and training sample labels are constructed based on the offset between the sample parameters and the optimal sample parameters; a visual Transformer prediction model is trained based on the training samples and training sample labels to obtain the trained image acquisition parameter prediction model.

[0038] This invention proposes an adaptive adjustment scheme for image acquisition parameters based on a visual Transformer network. This scheme enables joint prediction and collaborative optimization of hardware acquisition parameters and software preprocessing parameters, thereby improving detection accuracy under different defect detection tasks. The target image set is acquired under specific target parameters, which include hardware acquisition parameters and software preprocessing parameters. The target parameters and image set are input into a trained image acquisition parameter prediction model to obtain the output prediction results. Based on the prediction results, parameter adjustment values ​​are determined, and the hardware acquisition parameters of the image acquisition device and the software preprocessing parameters are adjusted accordingly. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0040] Figure 1 A flowchart of an image acquisition parameter prediction model training method provided in an embodiment of the present invention;

[0041] Figure 2 This is an example diagram illustrating the training process of a visual Transformer prediction model provided in an embodiment of the present invention.

[0042] Figure 3 This is an example of the architecture of a visual Transformer prediction model provided in an embodiment of the present invention;

[0043] Figure 4 This is a flowchart of an image acquisition parameter adjustment method provided in an embodiment of the present invention;

[0044] Figure 5 This is an application example diagram of an image acquisition parameter prediction model provided in an embodiment of the present invention;

[0045] Figure 6 This is a structural block diagram of an image acquisition parameter prediction model training device provided in an embodiment of the present invention;

[0046] Figure 7 This is a structural block diagram of an image acquisition parameter adjustment device provided in an embodiment of the present invention. Detailed Implementation

[0047] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0048] In industrial vision inspection systems, image quality directly determines inspection accuracy. Therefore, it is usually necessary to adjust hardware parameters such as camera exposure, light source brightness, and white balance, while also using software image enhancement (such as brightness, contrast, highlights, and shadows) to improve image details. However, existing technologies have the following technical shortcomings:

[0049] (1) Existing automatic exposure and automatic white balance algorithms are based solely on overall visual brightness or color balance adjustment, pursuing the "optimal brightness" or "color balance" effect of the image in terms of visual perception. Although this adjustment method can make the image look "visually comfortable" as a whole, it ignores the detail visibility and contrast requirements of key areas in industrial defect detection. Therefore, key defect areas may not be effectively identified due to overexposure, shadow occlusion, or insufficient contrast, leading to misjudgment or missed detection.

[0050] (2) Traditional automatic exposure systems adopt a uniform brightness adjustment strategy, treating all defect detection tasks the same, and lack the ability to adapt to different detection algorithms and specific defect types. Its parameter adjustment is unrelated to the specific detection task, and it cannot identify and prioritize the detail quality of key defect areas in the image, resulting in unstable detection results.

[0051] (3) Image processing software relies on manual experience parameters and lacks automatic adaptability. The software image enhancement parameters need to be manually adjusted, which is difficult to adapt to diverse product materials and complex environmental lighting conditions. This not only increases the complexity of operation and debugging time, but also reduces the efficiency and reliability of real-time detection on the production line.

[0052] (4) The hardware and software parameter adjustment modules are separated, lacking a unified and collaborative optimization control mechanism. Hardware parameters (exposure, light source brightness, white balance) are controlled by an independent system, while software image processing (brightness, highlights, shadows, etc.) is adjusted by another system. The two lack linkage decision-making based on image content. Simply using software to brighten the overall image to compensate for insufficient exposure will significantly amplify noise in dark areas, reduce the image signal-to-noise ratio, and thus affect the accuracy and robustness of subsequent defect detection algorithms. Manual debugging is costly and difficult to generalize parameters to different products and environments.

[0053] In summary, the existing technology directly results in industrial vision inspection systems being unable to automatically obtain the "optimal inspection effect" image quality under complex product types and varying lighting conditions, affecting the accuracy of defect detection and system stability.

[0054] The following combination Figure 1 , Figure 1 A flowchart of an image acquisition parameter prediction model training method provided in an embodiment of the present invention, the method may include:

[0055] S101: Acquire sample images of different detection targets under multiple sets of sample parameters to construct a sample image set of sample parameters; the sample parameters include the hardware acquisition parameters and software preprocessing parameters of the images.

[0056] This embodiment first acquires sample images of different detection targets under multiple sets of sample parameters to construct a sample image set for the sample parameters. To avoid training misguidance caused by individual image fluctuations and to improve parameter stability and generalization, this embodiment can acquire sample images of multiple categories of detection targets under each set of sample parameters. This embodiment does not limit the number of categories of detection targets and can be set based on actual applications.

[0057] This embodiment does not limit the number of images acquired for the same detection target under each set of sample parameters; these can be set based on actual applications. This embodiment also does not limit the total number of sample images in the sample image set; these can be set based on actual applications, but generally the total number of images in each sample image set should not be less than 50.

[0058] Each set of sample images can be used as batch data for model input. Each set of sample images corresponds to a set of sample parameters. Each set of sample parameters includes hardware acquisition parameters of the hardware device used to acquire images and software preprocessing parameters used to optimize the acquired images using software algorithms.

[0059] This embodiment does not limit the specific types of hardware devices and software algorithms used, nor does it limit the specific types of hardware acquisition parameters and software preprocessing parameters. They can be set based on actual applications.

[0060] In this embodiment, the image acquisition device may include an industrial camera, a light source, and related hardware devices, responsible for image acquisition. Hardware acquisition parameters may include exposure time, light source brightness, white balance, and gain. These parameters take effect before the image acquisition device acquires an image or during image generation; these parameters must be set through the interface controlling the camera or light source device and cannot be changed after acquisition.

[0061] Exposure time controls the amount of light entering the camera, affecting brightness, and is generally set through the industrial camera hardware interface; light source brightness controls the intensity of external lighting, and is generally adjusted through an external lighting control system; white balance adjusts the three-channel gain and controls color shift, and is generally set through the camera's ISP (Image Signal Processor) module or sensor driver.

[0062] Image acquisition equipment can utilize mature camera and light source control technologies from existing industrial vision systems. Industrial cameras, such as those from Basler and Daheng, can be connected via GigE (Gigabit Ethernet) or USB 3.0 (Universal Serial Bus 3.0) interfaces. They support SDK (Software Development Kit) / API (Application Programming Interface) control methods, such as Pylon and Dahua SDKs (control interfaces for industrial cameras), to allow real-time setting of parameters such as exposure time, gain, and white balance. The light source controller can communicate with a host computer via RS232 / 485 (serial communication standard) or digital I / O interfaces to adjust the light source voltage / brightness.

[0063] This embodiment can encapsulate the parameter adjustment interface of the image acquisition device into a unified control interface for the aforementioned hardware device, through which the parameters can be adjusted in a unified manner.

[0064] Typical software algorithms may include software image enhancement algorithms, supporting image quality optimization processing such as local highlight, shadow, and contrast adjustments based on brightness range. These software image enhancement algorithms can employ standard image enhancement techniques, such as OpenCV (Open Source Computer Vision Library) and Halcon (a machine vision software), supporting local brightness partitioning and enhancement operations based on image histogram analysis. Specific functions include: automatic brightness partition (bright / dark area) recognition, highlight compression, shadow brightening, and contrast stretching enhancement algorithms.

[0065] Specific software preprocessing parameters may include: overall image brightness adjustment, highlight / shadow / white / darkness adjustment, contrast, and color balance or saturation. These parameters only affect the acquired image data and do not change the physical sampling behavior of the image itself. Overall image brightness adjustment is used to increase or decrease the overall image brightness to achieve linear gain or gamma correction; highlight / shadow / white / darkness adjustment is used to enhance the visibility of local areas of the image, adjusting based on brightness distribution; contrast is used to differentiate grayscale distribution, and this parameter affects the sharpness of edges and defects; color balance or saturation is used to adjust the vibrancy of colors, mainly affecting visual perception and not necessarily beneficial for detection.

[0066] This embodiment does not limit the specific format of the sample images. RAW format images are a common feature of industrial cameras, as they retain the most complete original sensor information. During the training phase, acquiring a single RAW image and reconstructing it using software algorithms can simulate images with multiple different hardware acquisition parameters. When there is insufficient time to acquire training data, multiple sample images can be obtained by software reconstruction of RAW images to reduce sampling workload, improve parameter coverage, and help train a more robust Transformer model.

[0067] This embodiment does not limit the types of hardware acquisition parameters that can be simulated through software algorithm reconstruction. Generally, exposure, white balance, and gain can be simulated. Different exposures can be simulated by adjusting the overall gain of the RAW image, but it does not have the dynamic range expansion effect brought by real long exposure; white balance can be simulated by adjusting the three-channel gain, but the simulation result is slightly different from the real color temperature; the simulated gain can be adjusted, but it cannot truly reflect the hardware noise amplification characteristics.

[0068] This embodiment does not limit the total number of sample parameters, which can generally be set based on the actual application. After constructing the sample image set for each set of sample parameters, defects can be labeled on the images in the sample image set to determine the detection performance based on the detection results of the defect detection model.

[0069] S102: Based on the defect detection model, obtain the detection performance index of the sample image set under each set of sample parameters, and determine the sample parameters with the best detection performance index as the optimal sample parameters.

[0070] This embodiment can evaluate the detection performance of sample image sets under various sample parameters using a defect detection model.

[0071] This embodiment does not limit the specific architecture and training method of the defect detection model. It can be set according to the actual application. In this embodiment, the defect detection model can be a commonly used target detection model such as the YOLO model or the Unet model.

[0072] Specifically, each set of sample images can be detected using a defect detection model to obtain defect detection results. Based on the defect detection results, the detection performance index of each set of sample images, i.e., the detection accuracy, can be determined.

[0073] This embodiment does not limit the specific type of detection performance index. The detection performance index can adopt the standardized index mAP (mean Average Precision), such as mAP@0.5, mAP@0.5:0.95, and perform statistical analysis on each batch of sample images to evaluate the comprehensive performance of each group of sample parameters in terms of overall detection accuracy, recall, and positioning accuracy.

[0074] This embodiment can determine the optimal detection performance index from the detection performance indexes of each sample image set. For example, the sample image set with the largest mAP can be determined as the optimal sample parameter.

[0075] S103: Use the sample parameters and the set of sample images collected under the sample parameters as training samples, and construct training sample labels based on the offset between the sample parameters and the optimal sample parameters.

[0076] This embodiment can construct a training dataset for a visual Transformer prediction model based on sample parameters, sample image sets, and optimal sample parameters.

[0077] Specifically, in this embodiment, sample parameters and a set of sample images collected under those parameters can be used as training samples. Each training sample contains a set of sample parameters and a set of sample images collected under those parameters. In this embodiment, to enhance the model's ability to perceive different environments and historical adjustment effects, and to assist in more accurate parameter adjustments, ambient lighting data when the sample dataset was collected under the current sample parameters can also be added to the training samples.

[0078] Furthermore, this embodiment can construct sample labels for each training sample. This embodiment does not limit the specific method of constructing sample labels. Generally, training sample labels can be constructed based on the offset between the sample parameters and the optimal sample parameters. Specifically, the offset between the sample parameters in each training sample and the optimal sample parameters can be determined as the label of the training sample. This offset can represent the value of the sample parameters that need to be optimized and improved. For example, if the value of a certain type of parameter in the optimal sample parameters is A, and the value in the sample parameters in the training samples is B, then the offset is (AB). It can be understood that if this type of parameter is to reach the optimal value, the parameter value in the sample parameters needs to be adjusted based on the offset, for example, B + (AB) = A.

[0079] In this embodiment, the parameter offset is used as the model output target. The model learns the offset value relative to the "0-point reference parameter" rather than the absolute parameter, which enables the model to have the ability to generalize across detection targets and scenes, effectively avoiding overfitting to specific detection target types.

[0080] In this embodiment, from image batches with multiple different parameter combinations, the set of parameters with the best detection performance index is selected as the label for "optimal detection effect" in that scenario (i.e., the adjustment value required for this parameter combination is set to 0). Simultaneously, based on this benchmark, the adjustment values ​​required for the parameters of the remaining samples are calculated and used as image label inputs as the supervised training target for the visual Transformer model. This supervision mechanism uses detection performance as the core evaluation criterion and possesses high task specificity, reproducibility, and noise robustness.

[0081] To enhance the accuracy of supervised labels, this embodiment can combine detection performance metrics to construct training labels. For example, the offset between the sample parameters and the optimal sample parameters can be determined as the first training sample label of the training sample; and the detection performance metrics of the sample image set can be determined as the second training sample label of the training sample.

[0082] Furthermore, this embodiment can also introduce manual review, setting performance indicators for manual review to correct issues such as blurred areas, missing details, and overexposure / shadowing in a few images. Specifically, when receiving the manual review performance indicators of a manually input sample image set, the manual review performance indicators are determined as the third training sample label for the training samples; or, when receiving the manual review performance indicators of a manually input sample image set, the second training sample label for the training samples is determined based on the detection performance indicators of the sample image set and the manual review performance indicators.

[0083] In this embodiment, the manual review performance index can be used as a separate label, or the manual review performance index can be fused with the detection performance index to construct a second training sample label.

[0084] After the labels are constructed, this embodiment can divide the training dataset to construct training and test sets, and then train the visual Transformer prediction model using the training and test sets.

[0085] In this embodiment, the confidence level or classification loss of the defect detection model can be used as the detection performance index of the sample image set; alternatively, an image quality assessment network, such as NIMA (Neural Image Assessment) or BRISQUE (Blind / Referenceless Image Spatial Quality Evaluator), can be used to generate a no-reference image quality score to assist in scoring; or the statistical information of historical detection results of existing targets can be used to determine the quality of sample parameters through unsupervised clustering; or reinforcement learning methods can be introduced to learn parameter adjustment strategies using detection performance as a reward signal.

[0086] S104: Train a visual Transformer prediction model based on training samples and training sample labels to obtain a trained image acquisition parameter prediction model.

[0087] This embodiment can train a visual Transformer prediction model based on training samples and training sample labels to obtain a trained image acquisition parameter prediction model.

[0088] In this embodiment, the overall training process of the visual Transformer prediction model can be exemplified as follows: Figure 2As shown, a set of sample images is obtained under multiple sets of sample parameters, acquired by an image acquisition device and processed by a software preprocessing algorithm. The sample image set is input into a defect detection model for defect detection. Training sample labels are constructed based on the detection performance index obtained from the defect detection results and the performance index obtained from manual review. The optimal sample parameters are determined through the performance index, and training sample labels are constructed based on the offset between the sample parameters and the optimal sample parameters. The sample parameters and the sample image set are used as training samples. The training samples and training sample labels are input into a visual Transformer prediction model for model training, and the model parameters are updated. After training, the trained visual Transformer prediction model is determined as the image acquisition parameter prediction model.

[0089] The visual Transformer prediction model is a core component, primarily used to extract key region information from the original image. Combining sample parameters and the input image, it regresses and predicts a set of optimal adjustment values ​​for hardware acquisition parameters and software image processing parameters to improve the overall performance of subsequent defect detection. The model does not directly perform defect detection; instead, it uses performance feedback from the defect detection model as a supervisory signal, guiding the model to learn the mapping relationship between parameters and detection performance. The model possesses image semantic understanding and task adaptation capabilities, which are crucial for achieving optimal image quality in terms of detection performance.

[0090] This embodiment does not limit the specific architecture of the visual Transformer prediction model; generally, it can be as follows: Figure 3 As shown, after the sample images and sample parameters are input into the model, image encoding and parameter encoding are performed respectively. The image-encoded and parameter-encoded data are then input into the Transformer encoder, fusion layer and output regression head connected in sequence to finally obtain the output prediction results, such as prediction offset and prediction detection performance indicators.

[0091] Specifically, in the actual training process, the input sample image can be divided into pixel blocks of a preset size, such as a 16×16 pixel patch, through image embedding and segmentation, and mapped to a fixed-dimensional embedding space through linear projection; positional encoding is added to the patch embedding to preserve spatial information; the Transformer encoder consists of multiple layers of self-attention and feedforward networks, used to extract global and local features of the image and understand image content and lighting details; the fusion layer fuses the sample parameter features with the image features output by the Transformer, enhancing the model's ability to perceive the acquisition context; the regression head consists of several fully connected layers, used to regress and predict the offset (i.e., adjustment value) of the parameters of each group of samples.

[0092] The model incorporates an attention-weighted mechanism to enhance its sensitivity to features of potential defect regions in the image. Although the model does not directly output the defect category or location, it improves the capture of detailed information through feature weighting, thereby more accurately adjusting the acquisition and preprocessing parameters. The model output not only predicts the offset of hardware acquisition parameters but also predicts the adjustment values ​​of software preprocessing parameters based on the local brightness range of the image, achieving joint optimization of hardware and software parameters.

[0093] Model training is a multi-task joint regression prediction problem, and the output includes a multi-dimensional continuous parameter vector. Therefore, mean squared error loss and weighted multi-task loss are used in combination, and L2 regularization is added to the loss function to prevent overfitting.

[0094] Specifically, the training samples and their labels are input into the visual Transformer prediction model to obtain the model's output prediction offset and prediction performance metrics.

[0095] The loss term is determined based on the predicted offset and prediction performance indicators. The loss term includes mean square error loss and weighted multi-task loss. The weighted multi-task loss is a weighted loss of hardware acquisition parameter loss and software preprocessing parameter loss. This embodiment does not limit the weighting coefficients of hardware acquisition parameter loss and software preprocessing parameter loss, and can be set according to actual application.

[0096] The model loss value is determined based on the loss term and the regularization term. The model parameters of the visual Transformer prediction model are updated based on the model loss value until the trained image acquisition parameter prediction model is obtained.

[0097] In this embodiment, the optimization algorithm for the general model can use the AdamW optimizer in the PyTorch framework (a deep learning framework). The AdamW optimizer is an adaptive learning rate optimization algorithm that uses weight decay to more effectively control regularization. The parameter configuration can be: initial learning rate: 1×10 −4 The momentum decay rates β1 = 0.9 and β2 = 0.999, and the numerical stability term ϵ = 10. −8 The learning rate scheduler can be set to cosine annealing, and a warm-up strategy can be used in the early stages of training to gradually increase the learning rate to the preset value.

[0098] This embodiment allows for batch training of sample images, using a batch size that adjusts memory and computing resources based on image size, typically between 16 and 64. The training cycle is generally no less than 50 rounds, and overfitting is avoided by monitoring validation set performance.

[0099] Set up a validation set and periodically evaluate the model's loss and key metrics on the validation set, such as prediction parameter error and corresponding defect detection performance metrics mAP. Employ an early stopping mechanism to prevent overfitting.

[0100] Furthermore, during the training phase, appropriate data augmentation is performed on the input images, such as random cropping, rotation, and color jitter, to improve the model's generalization ability.

[0101] After the model training is completed, the weights of the best-performing model can be saved and exported in a format suitable for deployment, such as ONNX (Open Neural Network Exchange) or TensorRT (Tensor Runtime, a toolkit for optimizing deep learning model inference), to facilitate subsequent deployment in industrial settings.

[0102] Furthermore, in this embodiment, the model can be continuously optimized and transferred to other models. For example, during subsequent defect detection, the collected parameters, image data, and detection performance can be continuously recorded to update the Transformer model weights, improve the model's generalization ability, adapt to different products and complex scenarios, and achieve continuous learning and transfer.

[0103] Specifically, when the incremental learning dataset is obtained, the model parameters of the image acquisition parameter prediction model are updated based on the incremental learning dataset to obtain the image acquisition parameter prediction model after incremental learning is completed.

[0104] When the transfer learning dataset is obtained, the model parameters of the image acquisition parameter prediction model are updated based on the transfer learning dataset to obtain the image acquisition parameter prediction model after the transfer learning is completed.

[0105] Furthermore, in this embodiment, incremental learning and transfer learning can also be performed on the defect detection model.

[0106] In this embodiment, the visual Transformer prediction model can be replaced with other deep learning architectures that have image semantic modeling capabilities, such as multi-task learning models based on convolutional neural networks; structures that combine convolutional neural networks with attention mechanisms, such as CBAM (Convolutional Block Attention Module) and SE-ResNet (Squeeze-and-Excitation ResNet); or lightweight models, such as MobileNet (lightweight convolutional neural network) and EfficientNet (efficient convolutional neural network), can be used to adapt to edge deployment scenarios.

[0107] Based on the above embodiments, this invention proposes an adaptive adjustment scheme for image acquisition parameters based on a visual Transformer network, which realizes joint prediction and collaborative optimization of hardware acquisition parameters and software preprocessing parameters, thereby improving the detection accuracy under different defect detection tasks.

[0108] The following combination Figure 4 , Figure 4 A flowchart of an image acquisition parameter adjustment method provided in an embodiment of the present invention, the method may include:

[0109] S201: Acquire the target image set acquired under the target parameters; the target parameters include the hardware acquisition parameters and software preprocessing parameters of the images;

[0110] S201: Input the target parameters and target image set into the image acquisition parameter prediction model to obtain the output prediction result;

[0111] S201: Determine the parameter adjustment value based on the prediction result, and adjust the hardware acquisition parameters of the image acquisition device and the software preprocessing parameters of the image preprocessing software based on the parameter adjustment value; wherein, the image acquisition parameter prediction model is a model trained according to the image acquisition parameter prediction model training method.

[0112] In this embodiment, after training the image acquisition parameter prediction model, the model can be used to adjust the hardware and software parameters of the acquired images, such as... Figure 5 As shown, target parameters are set to acquire the target image set. The target parameters include target hardware acquisition parameters and software preprocessing parameters.

[0113] The target parameters and target image set are input into the image acquisition parameter prediction model to obtain the prediction results output by the model. The prediction results include hardware acquisition parameter offsets and software preprocessing parameter offsets. The hardware acquisition parameter offsets are used to adjust the hardware acquisition parameters in the image acquisition device, and the software preprocessing parameter offsets are used to adjust the software preprocessing parameters in the image preprocessing algorithm.

[0114] This embodiment does not limit the specific method of parameter adjustment. Generally, hardware instructions can be generated based on hardware acquisition parameter offsets, and software instructions can be generated based on software preprocessing parameter offsets. The hardware and software parameters can then be adjusted through a unified control interface based on the hardware and software instructions. For example, hardware instructions are sent to the industrial camera / light source control system to dynamically adjust acquisition conditions such as exposure time, light source brightness, and color temperature; software instructions are input into the image preprocessing algorithm to perform local highlight / shadow partition enhancement (such as local adjustment based on brightness partitions) to compensate for detail information.

[0115] After parameter adjustment is completed, the adjusted parameters can be verified. Specifically, in this embodiment, an optimized image set after parameter conditions are set can be collected, and the optimized image set can be input into the defect detection model. The detection performance index can be determined based on the detection results of the defect detection model.

[0116] If the detection performance indicators of the optimized parameters cannot meet the preset target, it is necessary to re-predict the parameter adjustment values ​​in the input image acquisition parameter prediction model by using the optimized parameters as target parameters.

[0117] In this embodiment, if the detection performance index still fails to meet the preset target after adjusting the target number of times, the image acquisition parameter prediction model can be retrained.

[0118] In this embodiment, during the parameter adjustment process, a rule engine based on a state machine or expert system can be used to drive the adjustment of control parameters after segmenting and quantizing the prediction results; or a traditional PID (proportional-integral-derivative) controller can be introduced to adjust the acquisition parameters in a closed loop according to the image brightness error and regional contrast error; or all image processing operations can be concentrated at the image algorithm end to perform simulated illumination enhancement, thereby reducing the dependence on hardware.

[0119] This embodiment utilizes a deep learning model to automatically understand image content and detection task requirements, intelligently predicting and adjusting image acquisition and processing parameters. Through hardware and software linkage control, it prioritizes optical parameters to ensure overall brightness, supplemented by software-based local detail adjustment, avoiding noise amplification caused by simple software brightening and improving the image signal-to-noise ratio. It achieves adaptive adjustment of image acquisition parameters for different product types and complex ambient lighting conditions, significantly reducing manual intervention and debugging costs, thereby ensuring that the acquired images have clear details and reasonable contrast, achieving optimal image quality for "detection effect," and improving the accuracy and robustness of subsequent defect detection. The parameter adaptive adjustment method based on the visual Transformer prediction model has high specificity. By creating training labels through the inference output of the actual defect detection algorithm, it can dynamically adjust image acquisition and preprocessing parameters for different detection tasks, ensuring that the details and contrast of key defect areas in the image are optimal, thereby significantly improving detection accuracy and robustness.

[0120] Based on the above embodiments, this invention proposes an adaptive adjustment scheme for image acquisition parameters based on a visual Transformer network, which realizes joint prediction and collaborative optimization of hardware acquisition parameters and software preprocessing parameters, thereby improving the detection accuracy under different defect detection tasks.

[0121] The following combination Figure 6 , Figure 6 This is a structural block diagram of an image acquisition parameter prediction model training device provided in an embodiment of the present invention. The device may include:

[0122] The first module 100 is used to acquire sample images of different detection targets under multiple sets of sample parameters, so as to construct a sample image set of sample parameters; the sample parameters include the hardware acquisition parameters and software preprocessing parameters of the images.

[0123] The second module 200 is used to obtain the detection performance index of the sample image set under each set of sample parameters based on the defect detection model, and to determine the sample parameters with the best detection performance index as the optimal sample parameters.

[0124] The third module 300 is used to take the sample parameters and the set of sample images collected under the sample parameters as training samples, and to construct training sample labels based on the offset between the sample parameters and the optimal sample parameters.

[0125] The fourth module 400 is used to train a visual Transformer prediction model based on training samples and training sample labels to obtain a trained image acquisition parameter prediction model.

[0126] Based on the above embodiments, this invention proposes an adaptive adjustment scheme for image acquisition parameters based on a visual Transformer network, which realizes joint prediction and collaborative optimization of hardware acquisition parameters and software preprocessing parameters, thereby improving the detection accuracy under different defect detection tasks.

[0127] Based on the above embodiments, the fourth module 400 may include:

[0128] The first unit is used to input training samples and training sample labels into the visual Transformer prediction model to obtain the prediction offset and prediction performance index of the model output.

[0129] The second unit is used to determine the loss term based on the predicted offset and prediction performance indicators. The loss term includes mean square error loss and weighted multi-task loss. The weighted multi-task loss is a weighted loss of hardware acquisition parameter loss and software preprocessing parameter loss.

[0130] The third unit is used to determine the model loss value based on the loss term and the regularization term, and to update the model parameters of the visual Transformer prediction model based on the model loss value until the trained image acquisition parameter prediction model is obtained.

[0131] Based on the above embodiments, the third module 300 may include:

[0132] The fourth unit is used to determine the first training sample label of the training sample by the offset between the sample parameters and the optimal sample parameters.

[0133] The fifth unit is used to determine the detection performance index of the sample image set as the second training sample label for the training samples.

[0134] Based on the above embodiments, the third module 300 may further include:

[0135] The sixth unit is used to determine the manual review performance index as the third training sample label when a manual review performance index of a sample image set input by humans is received.

[0136] The seventh unit is used to determine the second training sample label based on the detection performance index and the manual review performance index of the sample image set when receiving the manually input sample image set.

[0137] Based on the above embodiments, the device may further include:

[0138] The fifth module is used to update the model parameters of the image acquisition parameter prediction model based on the incremental learning dataset when the incremental learning dataset is obtained, so as to obtain the image acquisition parameter prediction model after the incremental learning is completed.

[0139] The sixth module is used to update the model parameters of the image acquisition parameter prediction model based on the transfer learning dataset when the transfer learning dataset is obtained, so as to obtain the image acquisition parameter prediction model after the transfer learning is completed.

[0140] Based on the above embodiments, the detection performance index is the mAP index.

[0141] The following combination Figure 7 , Figure 7 This is a structural block diagram of an image acquisition parameter adjustment device provided in an embodiment of the present invention. The device may include:

[0142] Module 700 is used to acquire the target image set obtained under the target parameters; the target parameters include the hardware acquisition parameters and software preprocessing parameters of the images.

[0143] The eighth module 800 is used to input the target parameters and target image set into the image acquisition parameter prediction model and obtain the output prediction result;

[0144] Module 900 is used to determine parameter adjustment values ​​based on prediction results, and to adjust the hardware acquisition parameters of the image acquisition device and the software preprocessing parameters of the image preprocessing software based on the parameter adjustment values.

[0145] Among them, the image acquisition parameter prediction model is a model trained by the image acquisition parameter prediction model training device.

[0146] Based on the above embodiments, this invention proposes an adaptive adjustment scheme for image acquisition parameters based on a visual Transformer network, which realizes joint prediction and collaborative optimization of hardware acquisition parameters and software preprocessing parameters, thereby improving the detection accuracy under different defect detection tasks.

[0147] Based on the above embodiments, the present invention also provides an electronic device, which may include a memory and a processor. The memory stores a computer program, and when the processor calls the computer program in the memory, it can implement the steps provided in the above embodiments. Of course, the device may also include various necessary network interfaces, a power supply, and other components.

[0148] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by an execution terminal or processor, can implement the method provided in the embodiments of the present invention; the storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0149] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A method for training an image acquisition parameter prediction model, characterized in that, include: Acquire sample images of different detection targets under multiple sets of sample parameters to construct a sample image set of the sample parameters; The sample parameters include the hardware acquisition parameters and software preprocessing parameters of the image; Based on the defect detection model, the detection performance index of the sample image set under each set of sample parameters is obtained, and the sample parameter with the largest detection performance index is determined as the optimal sample parameter. The sample parameters and the set of sample images collected under the sample parameters are used as training samples, and training sample labels are constructed based on the offset between the sample parameters and the optimal sample parameters. A visual Transformer prediction model is trained based on the training samples and the training sample labels to obtain a trained image acquisition parameter prediction model.

2. The image acquisition parameter prediction model training method according to claim 1, characterized in that, A visual Transformer prediction model is trained based on the training samples and the training sample labels to obtain a trained image acquisition parameter prediction model, including: The training samples and their labels are input into the visual Transformer prediction model to obtain the prediction offset and prediction performance index of the model output. The loss term is determined based on the predicted offset and the predicted performance index; the loss term includes mean square error loss and weighted multi-task loss, the weighted multi-task loss being a weighted loss of hardware acquisition parameter loss and software preprocessing parameter loss; The model loss value is determined based on the loss term and the regularization term, and the model parameters of the visual Transformer prediction model are updated based on the model loss value until the trained image acquisition parameter prediction model is obtained.

3. The image acquisition parameter prediction model training method according to claim 1, characterized in that, Constructing training sample labels based on the offset between the sample parameters and the optimal sample parameters includes: The offset between the sample parameters and the optimal sample parameters is determined as the first training sample label of the training sample; The detection performance index of the sample image set is determined as the second training sample label of the training samples.

4. The image acquisition parameter prediction model training method according to claim 3, characterized in that, Also includes: When the manually input performance index of the sample image set is received, the manually input performance index is determined as the third training sample label of the training sample. Alternatively, upon receiving the manually input performance index of the sample image set, the second training sample label of the training sample is determined based on the detection performance index of the sample image set and the manually input performance index.

5. The image acquisition parameter prediction model training method according to claim 1, characterized in that, Also includes: When the incremental learning dataset is obtained, the model parameters of the image acquisition parameter prediction model are updated based on the incremental learning dataset to obtain the image acquisition parameter prediction model after the incremental learning is completed. When the transfer learning dataset is obtained, the model parameters of the image acquisition parameter prediction model are updated based on the transfer learning dataset to obtain the image acquisition parameter prediction model after the transfer learning is completed.

6. The image acquisition parameter prediction model training method according to claim 1, characterized in that, The detection performance index is the mAP index.

7. A method for adjusting image acquisition parameters, characterized in that, include: Obtain the target image set acquired under the target parameters; The target parameters include the hardware acquisition parameters and software preprocessing parameters of the image; The target parameters and the target image set are input into the image acquisition parameter prediction model to obtain the output prediction result; Based on the prediction results, parameter adjustment values ​​are determined, and the hardware acquisition parameters of the image acquisition device and the software preprocessing parameters of the image preprocessing software are adjusted based on the parameter adjustment values. The image acquisition parameter prediction model is a model trained by the image acquisition parameter prediction model training method according to any one of claims 1 to 6.

8. A training device for an image acquisition parameter prediction model, characterized in that, include: The first module is used to acquire sample images collected from different detection targets under multiple sets of sample parameters, so as to construct a sample image set of the sample parameters; The sample parameters include the hardware acquisition parameters and software preprocessing parameters of the image; The second module is used to obtain the detection performance index of the sample image set under each set of sample parameters based on the defect detection model, and to determine the sample parameter with the largest detection performance index as the optimal sample parameter. The third module is used to take the sample parameters and the set of sample images collected under the sample parameters as training samples, and to construct training sample labels based on the offset between the sample parameters and the optimal sample parameters. The fourth module is used to train a visual Transformer prediction model based on the training samples and the training sample labels, so as to obtain a trained image acquisition parameter prediction model.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the image acquisition parameter prediction model training method as described in any one of claims 1 to 6, or the image acquisition parameter adjustment method as described in claim 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the image acquisition parameter prediction model training method as described in any one of claims 1 to 6, or the image acquisition parameter adjustment method as described in claim 7.

Images