An imaging parameter differentiable and computing power adaptive visual inspection system and method

Abstract

The present application relates to the technical field of industrial visual inspection, and specifically relates to a visual inspection system and method with differentiable imaging parameters and adaptive computing power, which mathematically differentiates the physical camera imaging process into a differentiable computation graph node, constructs an end-to-end optimization link of "differentiable imaging-inference", combines multi-modal closed-loop control based on a three-dimensional state space of "speed-temperature-confidence", realizes adaptive allocation of computing power, and compensates for light distribution through ROI partition and adaptive light control; the visual inspection system and method with differentiable imaging parameters and adaptive computing power can automatically search for optimal imaging parameters for a detection task, solve the defects of traditional visual system imaging and algorithm fragmentation and rigid computing power allocation, significantly improve the defect detection rate under complex light scenes, reduce the overkill rate and equipment operating temperature, balance the detection real-time performance and accuracy, and adapt to the dynamic working condition requirements of industrial production lines.

Description

Technical Field

[0001] This invention relates to the field of industrial visual inspection technology, specifically a visual inspection system and method with minimizable imaging parameters and adaptive computing power. Background Technology

[0002] Currently, in the field of industrial vision inspection, conventional industrial vision systems typically follow a sequential working mode of "imaging first, then processing." Optical imaging parameters such as camera exposure time, analog gain, and light source brightness are manually preset by on-site technicians or configured through the automatic adjustment function of the image signal processor (ISP). After the camera acquires and generates digital images after parameter configuration, the generated digital images are then sent to the defect detection AI model to complete inference tasks such as defect identification and target detection. This working mode is the mainstream implementation method in current industrial vision inspection scenarios.

[0003] Existing industrial vision systems suffer from several unresolved technical shortcomings when applied in real-world industrial scenarios: First, there is a serious disconnect between the imaging stage and the algorithm reasoning stage. Images with high clarity and uniform brightness in human visual perception are not the images from which defect detection AI models can most easily extract effective features and complete accurate identification. When the ambient light in the production site fluctuates, causing overexposure or underexposure in the acquired images, the detection performance of AI detection models with fixed parameter configurations will drop sharply. Secondly, there is the problem of rigid computing power allocation. After the traditional AI detection model is deployed on the edge hardware, its computing power consumption remains basically fixed and cannot be dynamically adjusted according to the on-site working conditions. When the production line speed fluctuates or the edge NPU processing chip overheats, it is impossible to dynamically adjust the inference scale and computing power occupation of the model. This easily leads to problems such as inference delay accumulation when the production line is running at high speed and serious waste of computing power resources when the production line is running at low speed. Meanwhile, existing technologies lack an end-to-end optimization link between imaging and inference. Existing neural architecture search (NAS) technology only optimizes the structure of neural networks and does not include the physical imaging parameters at the front end in the optimization search space. It cannot achieve coordinated optimization between the imaging and inference stages, and it is difficult to fundamentally solve the performance bottleneck of industrial vision inspection under complex working conditions. Therefore, in view of the above situation, there is an urgent need to develop a visual inspection system and method with minimizable imaging parameters and adaptive computing power to overcome the shortcomings in current practical applications. Summary of the Invention

[0004] The purpose of this invention is to provide a visual inspection system and method with minimizable imaging parameters and adaptive computing power, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A vision inspection system with minimizable imaging parameters and adaptive computing power is applied to industrial vision inspection scenarios. It includes an image acquisition unit, a light source unit, a processor, and a defect detection AI model. It also includes a differentiable imaging optimization module, a multimodal closed-loop control module, and an ROI illumination control module. The differentiable imaging optimization module is used to mathematize the imaging process of the physical camera into differentiable computational graph nodes, incorporate imaging physical parameters into the optimization closed loop of the defect detection AI model, and construct an end-to-end joint optimization link of differentiable imaging-inference. The multimodal closed-loop control module is used to dynamically schedule a pre-trained shared-weight multi-branch inference network based on a three-dimensional state space constructed from production line operating speed, processor core temperature, and model detection result confidence, thereby achieving adaptive allocation of computing power. The ROI illumination control module is used to perform partitioned closed-loop control of the partitioned programmable LED light source array of the light source unit based on the activation intensity of the intermediate layer features of the defect detection AI model, and to adaptively compensate for the illumination distribution of the detection area.

[0006] As a further aspect of the present invention: the differentiable imaging optimization module constructs a virtual ISP physical mathematical model, which is used to convert scene light radiation into a digital image. Its input parameters include exposure time, analog gain, noise model and white balance coefficient, and the output is a digital image. For the truncation function in the model, a pass-through estimator or a soft truncation function is used for differentiable approximation during backpropagation; The differentiable imaging optimization module defines a total loss function during the training phase. The total loss function includes the detection task loss and the hardware constraint regularization term. By calculating the partial derivative of the total loss function with respect to the imaging physical parameters, the optimal combination of imaging parameters that fits the detection task is searched using the gradient descent method to maximize the signal-to-noise ratio of the defect features.

[0007] As a further aspect of the present invention: the multimodal closed-loop control module is pre-trained with a multi-branch inference network with shared weights, the multi-branch inference network including at least a lightweight inference branch and a high-precision inference branch, and each branch achieves millisecond-level seamless switching through weight sharing.

[0008] As a further aspect of the present invention: the multimodal closed-loop control module collects three state variables in real time at a preset control cycle: production line operating speed, processor core temperature and model detection result confidence level, and executes corresponding branch switching rules based on the state variables. The switching rules include at least anti-ghosting mode, thermal protection mode and resampling verification mode. The anti-smear mode is used to switch to the lightweight inference branch, lock the upper limit of the exposure time, and compensate for the image brightness when the production line running speed exceeds the preset threshold; The thermal protection mode is used to trigger a frequency reduction strategy, switch to the lightweight inference branch, and adjust the image input resolution or the acquisition frame rate when the processor core temperature exceeds the preset threshold; The resampling confirmation mode is used to trigger an external hardware trigger to collect a second frame of image, switch to the high dynamic range imaging parameters and the high-precision inference branch for secondary confirmation when the confidence level of the detection result is within the preset hesitation interval during the operation of the lightweight inference branch and the processor temperature is within the normal range.

[0009] As a further solution of the present invention: the partition-programmable LED light source array supporting the ROI illumination control module adopts a multi-partition matrix structure, and the brightness of each partition is independently controlled; The ROI illumination control module extracts the intermediate layer feature map of the defect detection AI model, divides the feature map into grid partitions corresponding to the light source array in the spatial dimension, and calculates the average activation intensity of each grid partition.

[0010] As a further solution of the present invention: when the average activation intensity of a certain grid partition of the ROI illumination control module is less than the preset threshold, the brightness of the corresponding light source partition is increased during the acquisition of the next frame of image, and the illumination self-adaptation of the detected workpiece is completed through iterative illumination compensation.

[0011] A vision detection method with differentiable imaging parameters and adaptive computing power, applied to industrial vision detection scenarios, includes image acquisition, defect detection inference and result output steps, and also includes the following steps: S1. Construct an end-to-end joint optimization link for differentiable imaging-inference, mathematize the imaging process of the physical camera into a differentiable computational graph node, incorporate the imaging physical parameters into the optimization closed-loop of the defect detection AI model, and complete the pre-optimization of the imaging parameters and the pre-training of the shared-weight multi-branch inference network; S2. Real-time collect three state variables: the production line running speed, the processor core temperature, and the confidence level of the model detection result; S3. Based on the collected state variables, dynamically schedule the pre-trained shared-weight multi-branch inference network, adjust the imaging parameters, and achieve adaptive allocation of computing power; S4. Based on the intermediate layer feature activation intensity of the defect detection AI model, perform partition closed-loop control on the partition-programmable LED light source array, and adaptively compensate the illumination distribution of the detection area; S5. Output the final defect detection conclusion based on the output result of the current inference branch.

[0012] As a further aspect of the present invention: In step S1, a virtual ISP physical mathematical model is constructed to realize the differentiable processing of the imaging process. The virtual ISP physical mathematical model is used to convert the scene light radiation into a digital image. Its input parameters include exposure time, analog gain, noise model and white balance coefficient, and the output is a digital image. For the truncation function in the model, a pass-through estimator or a soft truncation function is used for differentiable approximation during backpropagation; During the training phase, a total loss function is defined, which includes the detection task loss and the hardware constraint regularization term. By calculating the partial derivative of the total loss function with respect to the imaging physical parameters, the optimal combination of imaging parameters that fits the detection task is searched using the gradient descent method to maximize the signal-to-noise ratio of the defect features.

[0013] As a further aspect of the present invention: in step S3, the state variables are collected in real time with a preset control cycle, and the corresponding branch switching rules are executed based on the state variables. The switching rules include at least anti-ghosting mode, thermal protection mode and resampling verification mode. The anti-ghosting mode is to switch to the lightweight inference branch when the production line speed exceeds a preset threshold, lock the upper limit of the exposure time and compensate for the image brightness. The thermal protection mode is to trigger a frequency reduction strategy when the processor core temperature exceeds a preset threshold, switch to a lightweight inference branch, and adjust the image input resolution or acquisition frame rate. The resampling verification mode is as follows: when the lightweight inference branch is running and the processor temperature is within the normal range, if the confidence level of the detection result is within the preset hesitation range, the hardware external trigger is triggered to acquire a second frame image, and the high dynamic range imaging parameters and high precision inference branch are switched to perform secondary verification. The final detection result is based on the output of the high precision inference branch.

[0014] As a further aspect of the present invention: in step S4, the partitioned programmable LED light source array adopts a multi-partition matrix structure, and the brightness of each partition is independently controlled; Extract the intermediate layer feature map of the defect detection AI model, divide the feature map into grid partitions corresponding to the light source array in the spatial dimension, and calculate the average activation intensity of each grid partition; When the average activation intensity of a certain grid partition is less than a preset threshold, the brightness of the corresponding light source partition is increased during the acquisition of the next frame image. The illumination adaptation of the workpiece is completed through iterative illumination compensation.

[0015] Compared with the prior art, the beneficial effects of the present invention are: This invention breaks through the physical imaging bottleneck of traditional industrial vision systems. By mathematizing physical imaging parameters into differentiable computational graph nodes and incorporating them into the optimization loop of a neural network, it constructs an end-to-end optimization link of "differentiable imaging-inference," solving the problem of physical layer information loss that traditional algorithms cannot handle, such as the disappearance of defect textures caused by overexposure and underexposure. It can automatically search for the optimal combination of imaging parameters adapted to specific detection tasks, maximizing the signal-to-noise ratio of defect features. At the same time, combined with ROI partitioning and adaptive lighting control, it can specifically eliminate problems such as workpiece surface reflection and local texture blurring, significantly improving the defect detection rate in complex lighting scenarios. Compared with traditional solutions, the defect detection rate can be improved by 5.3%. This invention achieves optimal energy efficiency for edge deployment. By using multimodal closed-loop control based on a three-dimensional state space of "speed-temperature-confidence" and combining it with a pre-trained shared weight multi-branch supernet, it achieves dynamic adaptive allocation of computing power. This allows the device to operate in a low-power, lightweight inference mode for 90% of its runtime, triggering high-precision resampling inference only when there is uncertainty in the detection results. While ensuring detection accuracy, it reduces the over-detection rate to 0.4%, lowering the false alarm rate by 87.5% compared to traditional solutions. It also significantly reduces the average inference latency and improves detection efficiency by about 3 times compared to traditional solutions. It balances the requirements of real-time detection and accuracy and is highly adaptable to the deployment requirements of edge embedded hardware. This invention effectively extends the service life of end-side detection equipment. Through thermal sensing and dynamic computing power scheduling mechanism, it can effectively prevent the NPU processing chip from being in an overheated state for a long time. Compared with traditional solutions, it can reduce the operating temperature of the equipment by 13°C, avoid hardware damage caused by long-term overheating of the chip, significantly improve the mean time between failures (MTBF) of the equipment, and greatly enhance the stability of long-term continuous operation in industrial sites. Attached Figure Description

[0016] Figure 1 This is a diagram illustrating the forward / backward propagation structure of a differentiable ISP and a neural network in an embodiment of the present invention; it shows how physical imaging parameters are incorporated into the neural network computation graph and optimized through backpropagation, wherein: 101 represents the scene light radiation R, 102 represents the virtual ISP physical and mathematical model, 103 represents the optimal parameter combination output unit, 201 represents the defect detection AI model, 202 represents the task loss calculation, 203 represents the hardware constraint regularization term, 204 represents the total loss function, and 205 represents the STE pass-through estimator / Soft Tanh.

[0017] Figure 2 This is a flowchart of the multimodal closed-loop state machine switching in an embodiment of the present invention; it illustrates the three-dimensional hardware scheduling logic based on "speed-temperature-confidence".

[0018] Figure 3This is a mapping diagram of the ROI illumination feedback control array in an embodiment of the present invention; it shows the 4x4 matrix mapping relationship between the network layer feature map and the physical light source, as well as the closed loop of illumination compensation calculation. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.

[0021] Please see Figures 1-3 This invention provides a vision inspection system with differentiable imaging parameters and adaptive computing power. The system constructs an end-to-end optimized link of "differentiable imaging-inference," and combines multimodal state perception to achieve coordinated hardware and software adaptation. This solves the technical defects of existing industrial vision systems, such as the separation of imaging and algorithms, rigid allocation of computing power, and lack of end-to-end optimization. Those skilled in the art will understand that this system can be widely applied to various industrial vision scenarios, including online defect detection, dimensional measurement, and target recognition on production lines, as well as the beverage bottle cap defect detection scenario described in subsequent embodiments.

[0022] The vision inspection system includes three core functional modules: a micro-imaging optimization module, a multi-modal closed-loop control module, and a ROI illumination control module. Each module interacts with its corresponding hardware unit via an industrial bus. The hardware unit includes at least a global shutter image sensor, a programmable ISP unit, a multi-core NPU processor, a partitioned programmable LED light source array, a production line speed acquisition unit, and a temperature acquisition unit. All of the above hardware units are conventional hardware configurations for industrial vision systems in this field.

[0023] 1. Microscopic Imaging Optimization Module The core of this module lies in mathematizing the imaging process of a physical camera into a differentiable computational graph node, enabling it to participate in the backpropagation of a neural network and achieve end-to-end joint optimization of imaging physical parameters and AI detection models.

[0024] 1.1 Differentiable ISP Modeling This module constructs a virtual ISP layer. ISP This refers to the virtual ISP physical mathematical model 102, whose input is the simulated scene light radiation R (Radiance) 101, and whose output is the digital image I. The model definition formula is as follows: I=Clip((R t g+N) WB,0,255); Where t is the exposure time; g is the analog gain; N is the noise model, which uses a Gaussian-Poisson mixed distribution to simulate sensor noise; WB is the white balance coefficient; and Clip is the cutoff function. To ensure differentiability, the STE pass-through estimator / Soft Tanh 205 approximation is used during backpropagation to solve the problem of non-differentiability of the Clip function.

[0025] The STE (Straight Through Estimator) and Soft Tanh (Soft Truncation) mentioned above are both differentiable approximation methods known in the art, which can solve the gradient vanishing problem of the truncated function at the boundary and ensure the normal progress of the backpropagation process.

[0026] By using the aforementioned differentiable ISP modeling, the physical imaging process, which was originally independent of the neural network, can be incorporated into the computational graph of the neural network. This breaks the traditional serial and fragmented mode of "imaging first, then processing" in industrial vision systems, allowing the imaging physical parameters to be gradient optimized according to the needs of the detection task, rather than just meeting the requirements of human vision for clarity and brightness balance. This avoids the problem of loss of defect features caused by overexposure and underexposure from the source of physical imaging, and improves the robustness of the detection model in complex lighting scenarios.

[0027] 1.2 Joint Parameter Optimization Algorithm During the training phase, this module defines the total loss function L. total 204, the formula is as follows: L total =L task (M(I),Y)+λ L reg (t,g); Where M is the defect detection AI model 201, and Y is the ground truth label for the detection task; L task For task loss calculation 202, the task loss function can be selected according to the type of detection task, choosing a conventional loss function in the field, such as cross-entropy loss for classification tasks and CIoU loss for detection tasks; L reg 203 is a hardware constraint regularization term used to constrain t and g within the range allowed by the physical hardware (e.g., t is between 10μs and 20ms); λ is the weight coefficient of the regularization term.

[0028] By calculating the partial derivatives of the total loss function 204 with respect to the exposure time t and the analog gain g, the system can automatically search for the optimal parameter combination output unit 103 for a specific detection target (such as a fine scratch on a reflective metal surface) using the gradient descent method: optimal exposure time t. *Optimal gain g * This maximizes the signal-to-noise ratio (SNR) of defect features in the image, rather than achieving the most balanced brightness across the entire image.

[0029] Through the above parameter joint optimization algorithm, the system can automatically optimize imaging parameters according to the feature extraction requirements of specific detection tasks, effectively solving the problem of difficulty in identifying subtle defects in reflective and low-contrast scenes, and greatly improving the detection rate and anti-interference ability of the detection model.

[0030] 2. Multimodal closed-loop control module This module uses a three-dimensional state space of "speed-temperature-confidence" to dynamically schedule model structure and hardware resources, thereby achieving adaptive allocation of computing power and safe and stable operation of hardware.

[0031] 2.1 SuperNet Architecture Design This module pre-trains a multi-branch network with shared weights as the base model for dynamic scheduling. The multi-branch network contains at least the following two inference branches: Branch A (Lite mode): A lightweight CNN with a depth of 8 layers, an input resolution of 320x320, and a single-frame inference time of less than 5ms; Branch B (Heavy mode): A 50-layer Transformer hybrid architecture with an input resolution of 1024x1024 and a single-frame inference time of approximately 45ms.

[0032] The shared weight multi-branch supernet is a well-known technology in the field of neural architecture search (NAS). It achieves unified training and fast switching between different branches through weight sharing, without the need to reload model weights when switching branches, thus avoiding detection interruptions and delays during the switching process.

[0033] Through the above supernet architecture design, it is possible to achieve millisecond-level seamless switching between models with different computing power requirements and different detection accuracy, adapt to the dynamic changes in the operating speed and hardware status of the production line, and solve the defects of the traditional fixed model deployment, which has fixed computing power consumption and cannot adapt to dynamic scenarios.

[0034] 2.2 State Machine Transition Logic This module sets the control cycle of the state machine to 10ms, and collects three state variables in real time during each control cycle: production line speed V. line NPU core temperature (Temp) npu The confidence level (Conf) of the model output. Based on the above three state variables, the following switching rules are executed: Rule 1 (Anti-ghosting mode) If the production line speed V line Greater than the preset threshold V thresh(e.g., 2m / s), force a switch to branch A for inference, and lock the exposure time t at t. max =Δx / V line (Δx is the allowable pixel equivalent of motion blur), while automatically increasing the analog gain g to compensate for the reduced image brightness caused by the shortened exposure time.

[0035] The aforementioned anti-ghosting mode completely avoids the problem of workpiece imaging ghosting under high-speed movement. At the same time, it ensures effective image brightness through automatic analog gain compensation, balancing imaging clarity and real-time detection in high-speed production line scenarios, and avoiding the problem of missed detection caused by the accumulation of detection delays.

[0036] Rule 2 (Thermal Protection Mode) If the NPU core temperature Temp npu Greater than 85 The frequency reduction strategy is triggered: first, the resolution of the input image is reduced to 70% of the original image, and branch A is forced to be called for inference; if the temperature continues to rise, the camera's acquisition frame rate (FPS) is further reduced.

[0037] The above thermal protection mode can effectively prevent the chip from overheating for a long time, reduce the operating temperature of the device, and increase the mean time between failures (MTBF) of the device. At the same time, it maximizes the use of computing resources under the premise of controllable temperature, avoiding the waste of computing power and the risk of overheating damage in the fixed computing power mode.

[0038] Rule 3 (Resampling Confirmation Mode) Running on branch A and with NPU core temperature Temp npu If the model output is "defective" and the confidence level (Conf) is in the "hesitation interval" of 0.5 to 0.8 when the result is within the normal range, perform the following two steps: Step 1: Immediately trigger the hardware trigger signal to acquire the second frame image at time t+Δt; Step 2: The second frame image is automatically switched to the high exposure parameter (High Dynamic Range), and branch B is called for inference. The final detection result is based on the output of branch B.

[0039] The aforementioned hardware trigger signal is a well-known external trigger acquisition mode in the field of industrial cameras, which can achieve precise synchronization between camera acquisition and production line movement and model inference, ensuring the accuracy of the acquisition timing of resampled frames and the position of the target workpiece.

[0040] The above-mentioned resampling verification mode can maintain lightweight inference with low power consumption and low latency in most normal scenarios. High-precision resampling inference is triggered only when there is uncertainty in the detection result. While ensuring an extremely low overkill rate, it also takes into account the overall detection efficiency, thus solving the core contradiction of "high precision and low latency cannot be achieved at the same time" in traditional fixed models.

[0041] 3. ROI Illumination Control Module This module uses the activation intensity of intermediate layer features in the AI ​​detection model to achieve partitioned closed-loop control of the programmable light source array, adaptively compensate for the illumination distribution in the detection area, and eliminate feature loss problems caused by surface reflection and local underexposure.

[0042] 3.1 Hardware Configuration The hardware accompanying this module uses a 16-zone (4x4 matrix) programmable LED light source array, with each zone independently controlling the PWM duty cycle (0-100%) to achieve independent brightness adjustment of each zone.

[0043] By employing a zone-independently controlled light source array, differentiated illumination compensation can be achieved for different areas of the workpiece being inspected. Compared to traditional global light sources, this approach can specifically address the issues of localized reflections or insufficient illumination caused by the curved surfaces and uneven structures of the workpiece, providing better illumination conditions for feature extraction from the model.

[0044] 3.2 Feature Feedback Mapping This module extracts the feature map (F) of the third convolutional layer in the AI ​​detection model; it then divides the feature map F into a 4x4 grid corresponding to the light source array, and calculates the average activation intensity A of each grid region k. k .

[0045] By mapping the intermediate feature maps of the AI ​​model to physical light source partitions in a one-to-one spatial correspondence, the illumination can be adjusted directly based on the feature extraction effect feedback of the model, rather than based on the image brightness observed by the human eye. This allows the illumination adjustment to fully serve the feature extraction needs of the detection task, achieving deep collaboration between illumination control and detection algorithms.

[0046] 3.3 Illumination Compensation Strategy If the activation intensity A of a certain grid region k is... k If the brightness is less than a preset threshold (indicating that the texture features in that area are blurry and the model has not extracted effective information), then the brightness L of the corresponding physical location LED partition will be increased during the next frame image acquisition. k The brightness update formula is as follows: L k_next =L k_curr +α (Threshold-A k); Among them, L k_next For the target brightness of this partition in the next frame, L k_curr The value is the brightness of the current frame for that partition, α is the iteration step size coefficient, and Threshold is the preset threshold for activation intensity.

[0047] Through the aforementioned iterative illumination compensation strategy, the system can complete the adaptive illumination adaptation of the workpiece surface geometry within an iteration cycle of 3-5 frames, effectively eliminating problems such as surface reflection and local texture blurring, improving the model's feature extraction capability for subtle defects, and further enhancing the detection rate and operational stability.

[0048] 4. Specific Application Examples The application scenario of this embodiment is the detection of defects in the internal sealing gasket of a high-speed beverage bottle cap. The core characteristics of this scenario are: the inner surface of the bottle cap has strong reflectivity, the production line operating speed fluctuates greatly, and there are extremely high requirements for the real-time performance, detection rate and overkill rate of the detection.

[0049] 4.1 Hardware parameter configuration of this embodiment Processor: Rockchip RK3588 (NPU with integrated 6 TOPS computing power) Image sensor: Sony IMX296 global shutter sensor Light source: Customized 16-zone coaxial light source, matched with the ROI lighting control module mentioned above. Supporting production line: Adjustable speed beverage filling line, with a speed fluctuation range of 0.5m / s-3m / s. 4.2 Comparison of the effect data between this embodiment and the traditional solution (1) Defect detection rate (Recall): Traditional approach (fixed imaging parameters + ResNet50 detection model): 94.5%, severely affected by reflections from the inner surface of the bottle cap, and prone to missing minor defects such as small rubber pads and air bubbles; The solution of this invention (adaptive ISP optimization + SuperNet dynamic scheduling): 99.8%, through differentiable imaging optimization and regional illumination adaptive control, automatically optimizes illumination to eliminate reflection interference and effectively extracts subtle defect features; Results: The defect detection rate increased by 5.3%.

[0050] (2) False Alarm Rate: Traditional approach: 3.2%, prone to misjudging reflectivity and stains as defects; The solution of this invention: 0.4%, through a resampling verification mechanism, performs high-precision secondary verification on results with questionable confidence, which greatly reduces the probability of misjudgment; Results: The false alarm rate was reduced by 87.5%.

[0051] (3) Average processing latency: Traditional solution: 35ms (fixed inference time, independent of production line speed); The solution of this invention is: 12ms (when the production line is running at low speed) / 6ms (when the production line is running at high speed), by dynamically switching the supernet branches, the inference time is adjusted to adapt to the production line speed. Results: Detection efficiency is improved by approximately 3 times.

[0052] (4) Equipment operating surface temperature: Traditional solution: 55℃, NPU operates at full load for a long time, and the temperature remains consistently high; The solution of this invention: At 42℃, through thermal sensing and dynamic scheduling of computing power, the device operates in low-power mode for 90% of the time, effectively controlling the operating temperature; Results: The equipment operating temperature decreased by 13 degrees Celsius, effectively extending the equipment's service life and long-term operational stability.

[0053] This embodiment also provides a visual inspection method with scalable imaging parameters and adaptive computing power. The method is implemented based on the above-mentioned visual inspection system and includes the following steps: S1. Model Training and Imaging Parameter Pre-optimization: Construct an end-to-end joint optimization link between the virtual ISP physical and mathematical model 102 and the defect detection AI model 201, using the total loss function L. total 204 optimizations yielded initial imaging parameters suitable for the detection task, while pre-training of the shared-weight multi-branch supernet was completed. S2. Real-time online status acquisition: The production line speed V is acquired in real time with a control cycle of 10ms. line NPU core temperature (Temp) npu The model outputs three state variables: confidence level (Conf). S3. Adaptive scheduling of computing power and imaging parameters: Based on the acquired state variables, execute the corresponding state machine switching rules, dynamically select the inference branch of the supernet, and adjust the imaging parameters and camera acquisition frame rate. S4, ROI Partition Illumination Adaptive Control: Extract the intermediate layer feature map of the AI ​​detection model, calculate the feature activation intensity of each partition, iteratively adjust the brightness of the corresponding LED partition, and complete the illumination closed-loop compensation. S5. Defect Detection Result Output: Based on the output of the current inference branch, output the final defect detection conclusion. For results with confidence in the hesitation interval, trigger the resampling confirmation process and output the final conclusion.

[0054] It should be noted that, in this invention, although the specification describes the embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A vision inspection system with minimizable imaging parameters and adaptive computing power, applied to industrial vision inspection scenarios, comprising an image acquisition unit, a light source unit, a processor, and a defect detection AI model, characterized in that, It also includes a micro-imaging optimization module, a multimodal closed-loop control module, and a ROI illumination control module; The differentiable imaging optimization module is used to mathematize the imaging process of the physical camera into differentiable computational graph nodes, incorporate imaging physical parameters into the optimization closed loop of the defect detection AI model, and construct an end-to-end joint optimization link of differentiable imaging-inference. The multimodal closed-loop control module is used to dynamically schedule a pre-trained shared-weight multi-branch inference network based on a three-dimensional state space constructed from production line operating speed, processor core temperature, and model detection result confidence, thereby achieving adaptive allocation of computing power. The ROI illumination control module is used to perform partitioned closed-loop control of the partitioned programmable LED light source array of the light source unit based on the activation intensity of the intermediate layer features of the defect detection AI model, and to adaptively compensate for the illumination distribution of the detection area.

2. The vision inspection system with minimizable imaging parameters and adaptive computing power according to claim 1, characterized in that, The differentiable imaging optimization module constructs a virtual ISP physical mathematical model, which is used to convert scene light radiation into digital images. Its input parameters include exposure time, analog gain, noise model and white balance coefficient, and the output is a digital image. For the truncation function in the model, a pass-through estimator or a soft truncation function is used for differentiable approximation during backpropagation; The differentiable imaging optimization module defines a total loss function during the training phase. The total loss function includes the detection task loss and the hardware constraint regularization term. By calculating the partial derivative of the total loss function with respect to the imaging physical parameters, the optimal combination of imaging parameters that fits the detection task is searched using the gradient descent method to maximize the signal-to-noise ratio of the defect features.

3. The vision inspection system with minimizable imaging parameters and adaptive computing power according to claim 1, characterized in that, The multimodal closed-loop control module is pre-trained with a multi-branch inference network with shared weights. The multi-branch inference network includes at least a lightweight inference branch and a high-precision inference branch. Each branch achieves seamless switching at the millisecond level through weight sharing.

4. The vision inspection system with minimizable imaging parameters and adaptive computing power according to claim 3, characterized in that, The multimodal closed-loop control module collects three state variables in real time: production line operating speed, processor core temperature, and model detection result confidence level, according to a preset control cycle. Based on the state variables, it executes the corresponding branch switching rules, which include at least anti-ghosting mode, thermal protection mode, and resampling confirmation mode. The anti-ghosting mode is used to switch to the lightweight inference branch when the production line speed exceeds a preset threshold, lock the upper limit of the exposure time and compensate for the image brightness. The thermal protection mode is used to trigger a frequency reduction strategy when the processor core temperature exceeds a preset threshold, switch to a lightweight inference branch, and adjust the image input resolution or acquisition frame rate. The resampling verification mode is used to trigger the acquisition of a second frame image by external hardware trigger when the detection result confidence is within a preset hesitation range, and switch to high dynamic range imaging parameters and high precision inference branch for secondary verification, while the lightweight inference branch is running and the processor temperature is within the normal range.

5. The vision inspection system with minimizable imaging parameters and adaptive computing power according to claim 1, characterized in that, The ROI illumination control module is equipped with a partitioned programmable LED light source array, which adopts a multi-partition matrix structure, and the brightness of each partition is controlled independently. The ROI illumination control module extracts the intermediate layer feature map of the defect detection AI model, divides the feature map into grid partitions corresponding to the light source array in the spatial dimension, and calculates the average activation intensity of each grid partition.

6. The vision inspection system with minimizable imaging parameters and adaptive computing power according to claim 5, characterized in that, When the average activation intensity of a certain grid partition is less than a preset threshold, the ROI illumination control module increases the brightness of the corresponding light source partition during the next frame image acquisition, and completes the adaptive illumination of the workpiece through iterative illumination compensation.

7. A visual inspection method with scalable imaging parameters and adaptive computing power, applied to industrial visual inspection scenarios, comprising image acquisition, defect detection inference, and result output steps, characterized in that, It also includes the following steps: S1. Construct an end-to-end joint optimization link for differentiable imaging-inference, mathematize the imaging process of the physical camera into differentiable computational graph nodes, incorporate imaging physical parameters into the optimization closed loop of the defect detection AI model, and complete the pre-optimization of imaging parameters and the pre-training of the shared weight multi-branch inference network. S2. Real-time acquisition of three state variables: production line operating speed, processor core temperature, and model detection result confidence level; S3. Based on the collected state variables, dynamically schedule the pre-trained shared weight multi-branch inference network, adjust the imaging parameters, and achieve adaptive allocation of computing power; S4. Based on the activation intensity of the intermediate layer features of the defect detection AI model, partitioned closed-loop control is performed on the partitioned programmable LED light source array to adaptively compensate for the illumination distribution in the detection area. S5. Based on the output of the current inference branch, output the final defect detection conclusion.

8. The visual detection method with minimizable imaging parameters and adaptive computing power according to claim 7, characterized in that, In step S1, a virtual ISP physical mathematical model is constructed to realize the differentiable processing of the imaging process. The virtual ISP physical mathematical model is used to convert the scene light radiation into a digital image. Its input parameters include exposure time, analog gain, noise model and white balance coefficient, and the output is a digital image. For the truncation function in the model, a pass-through estimator or a soft truncation function is used for differentiable approximation during backpropagation; During the training phase, a total loss function is defined, which includes the detection task loss and the hardware constraint regularization term. By calculating the partial derivative of the total loss function with respect to the imaging physical parameters, the optimal combination of imaging parameters that fits the detection task is searched using the gradient descent method to maximize the signal-to-noise ratio of the defect features.

9. The visual detection method with minimizable imaging parameters and adaptive computing power according to claim 7, characterized in that, In step S3, the state variables are collected in real time with a preset control cycle, and the corresponding branch switching rules are executed based on the state variables. The switching rules include at least anti-ghosting mode, thermal protection mode and resampling verification mode. The anti-ghosting mode is to switch to the lightweight inference branch when the production line speed exceeds a preset threshold, lock the upper limit of the exposure time and compensate for the image brightness. The thermal protection mode is to trigger a frequency reduction strategy when the processor core temperature exceeds a preset threshold, switch to a lightweight inference branch, and adjust the image input resolution or acquisition frame rate. The resampling verification mode is as follows: when the lightweight inference branch is running and the processor temperature is within the normal range, if the confidence level of the detection result is within the preset hesitation range, the hardware external trigger is triggered to acquire a second frame image, and the high dynamic range imaging parameters and high precision inference branch are switched to perform secondary verification. The final detection result is based on the output of the high precision inference branch.

10. The visual detection method with minimizable imaging parameters and adaptive computing power according to claim 7, characterized in that, In step S4, the partitioned programmable LED light source array adopts a multi-partition matrix structure, and the brightness of each partition is controlled independently; Extract the intermediate layer feature map of the defect detection AI model, divide the feature map into grid partitions corresponding to the light source array in the spatial dimension, and calculate the average activation intensity of each grid partition; When the average activation intensity of a certain grid partition is less than a preset threshold, the brightness of the corresponding light source partition is increased during the acquisition of the next frame image. The illumination adaptation of the workpiece is completed through iterative illumination compensation.

Images