A Multi-Task Industrial Vision Collaborative Inspection Method Based on VisionMaster

By using the multi-task collaborative detection method of the VisionMaster platform, feature extraction is performed by sharing the convolutional backbone network and channel attention module, and OCR and object detection are performed in parallel. This solves the speed and accuracy problems of visual inspection tasks on the SMT assembly line and achieves efficient and low-cost multi-task collaborative detection.

CN123118616APending Publication Date: 2026-11-17XIAN UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611587381.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-14
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies cannot meet the cycle time requirements of high-precision, high-speed SMT assembly lines. Traditional methods result in excessively long calculations for repetitive feature extraction, making it impossible to simultaneously complete number identification and defect detection. Furthermore, hardware deployment costs are high or accuracy is compromised.

Method used

A multi-task collaborative detection method based on the VisionMaster platform is adopted. Feature extraction is performed by sharing a convolutional backbone network and a channel attention module. OCR recognition and target detection are performed in parallel. Combined with learnable gating vectors and a two-way verification mechanism, the synchronous output of serial number recognition and solder joint defect detection is achieved.

Benefits of technology

A single image acquisition can complete multiple detection tasks, improving inference speed, reducing hardware costs, meeting the real-time and accuracy requirements of high-speed production lines, and enhancing the model's adaptability to different scenarios and the reliability of output results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118616A_ABST
    Figure CN123118616A_ABST
Patent Text Reader

Abstract

This invention relates to a multi-task industrial vision collaborative inspection method based on VisionMaster, specifically in the fields of image processing and pattern recognition. By sharing a convolutional backbone network and a channel attention module, it achieves simultaneous serial number recognition and weld defect detection in a single image acquisition, avoiding redundant feature extraction and significantly improving inference speed. The learnable gating vector dynamically balances the gradients of the two tasks, enhancing the model's adaptability to different scenarios. The bidirectional verification mechanism utilizes spatial relationships to mutually correct each other, improving the reliability of the output results. The entire process is completed within the VisionMaster platform, requiring no additional hardware, reducing deployment costs, and meeting the dual requirements of real-time performance and accuracy for high-speed production lines.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of image processing and pattern recognition, and more specifically, to a multi-task industrial vision collaborative detection method based on VisionMaster. Background Technology

[0002] As the consumer electronics manufacturing industry develops towards higher precision and higher speed, the production cycle of SMT (Surface Mount Technology) assembly lines has generally increased to 0.5 seconds per piece. Taking mobile phone motherboards and chip modules as examples, two visual inspection tasks often need to be completed simultaneously at the same workstation: one is to read the serial number on the surface of the printed circuit board through optical character recognition technology to achieve product traceability, and the other is to use target detection algorithms to determine whether there are defects such as cold solder joints, misalignment, or missing components to ensure soldering quality. Currently, these production lines widely use VisionMaster as the host computer vision platform, in conjunction with a single high-resolution industrial camera to complete image acquisition and analysis. Since the camera's frame rate is fixed at 30 frames per second, each frame of image must complete all processing steps within a specified time to keep up with the production line's cycle time.

[0003] In practical applications, the traditional approach is to separate number identification and defect classification into two independent VisionMaster toolchain processes, each calling its own deep learning model; or to execute the forward inference of the two models sequentially within the same process. Regardless of the method, each frame of image must undergo two complete neural network forward propagation processes, namely feature extraction, classification, or identification. This repetitive feature extraction calculation often results in the total processing time for a single frame of image exceeding 0.5 seconds, which cannot meet the production line cycle time requirements. The reason for this is that the two tasks independently extract low-level features from the original image, consuming a large amount of computing resources from the graphics processing unit or central processing unit, while also causing memory bandwidth competition, resulting in serious computational redundancy.

[0004] To address the aforementioned issues, existing technologies offer several alternative solutions. One approach is to use a high-performance graphics processor to run two independent models in parallel, such as multi-stream inference based on TensorRT. However, this method is limited by the memory capacity of edge industrial control computers, making it difficult to deploy on low-cost hardware. Another approach is to reduce model size through model compression techniques such as pruning or quantization, but this often sacrifices the detection accuracy of individual tasks. Yet another approach is to construct a single multi-task neural network, such as fusing the YOLO object detection network with an OCR recognition network. However, such customized models are difficult to deeply integrate with the existing toolchain of the VisionMaster platform, failing to fully utilize the platform's existing mature modules such as image localization, camera calibration, and PLC communication. Therefore, how to design a multi-task collaborative inference architecture that can simultaneously produce number recognition and defect classification results from a single image acquisition without excessively increasing hardware investment, leveraging the programmability and toolchain orchestration capabilities of the VisionMaster platform, has become a pressing technical challenge. Summary of the Invention

[0005] This invention addresses the technical problems existing in the prior art by providing a multi-task industrial vision collaborative detection method based on VisionMaster, thereby solving the problems mentioned in the background art.

[0006] The technical solution of this invention to solve the above-mentioned technical problems is as follows: specifically, it includes the following steps: Step S1: Configure the high-resolution industrial camera and receive the trigger signal through the VisionMaster workflow editor, acquire PCB board images, use the template matching module built into VisionMaster to automatically identify and crop the region of interest containing the serial number area and solder joint area, perform distortion correction on the region of interest and perform multi-scale normalization processing to obtain the normalized tensor. Step S2: Feed the standardized tensor into the shared convolutional backbone network for feature extraction, and embed a channel attention module at the end of the shared convolutional backbone network. Learn the channel weights through global average pooling and two fully connected networks, and perform channel-wise weighting on the feature map output by the shared convolutional backbone network to output an enhanced feature map. Step S3: The enhanced feature map is simultaneously fed into the OCR recognition head and the target detection head for parallel decoding. The OCR recognition head compresses the height through convolution and outputs a character probability matrix and a first confidence score through a bidirectional LSTM network. The character probability matrix is ​​decoded to obtain a sequence number string. The target detection head performs multi-scale fusion through a feature pyramid network and outputs a defect list and a second confidence score through a lightweight detection head. At the same time, a learnable gating vector is introduced to dynamically scale the backpropagation gradient corresponding to the OCR recognition head and the backpropagation gradient corresponding to the target detection head to achieve adaptive balance. Step S4: Synchronize the serial number string output by the OCR recognition head and the defect list output by the target detection head through the VisionMaster global variable interface. Perform bidirectional verification based on the preset first confidence threshold and the preset second confidence threshold. After mutual correction using spatial relationships, call the VisionMaster PLC communication module through the VisionMaster event triggering mechanism to package the results and send them to the production line control system, completing the multi-task collaborative output of a single forward inference. In a preferred embodiment, in step S1, the working parameters of the high-resolution industrial camera are configured through the VisionMaster workflow editor, including setting the exposure time to adaptive mode or a fixed millisecond value, setting the gain to automatic gain control or a fixed decibel value, and setting the trigger mode to hardware rising edge trigger or continuous acquisition mode. A high-resolution industrial camera receives trigger signals from the production line control system and accurately captures an image of the PCB board when it arrives at the designated station. Using the template matching module built into VisionMaster, a pre-stored PCB board reference template is loaded. A normalized cross-correlation algorithm is used to search for the position in the captured PCB board image that best matches the PCB board reference template, and the center coordinates and rotation angle of the matching area are returned. Based on the center coordinates and rotation angle of the matching area, the region of interest, which includes the serial number area and the solder joint area, is automatically cropped.

[0007] In a preferred embodiment, the specific operation of performing distortion correction and multi-scale normalization on the region of interest is as follows: First, using the radial and tangential distortion coefficients obtained during the high-resolution industrial camera calibration phase, the coordinates of each pixel within the region of interest (ROI) are mapped to the coordinates under an ideal pinhole model using a bilinear interpolation algorithm, resulting in a corrected ROI. Then, the corrected ROI is scaled to a fixed target height and width using bilinear interpolation, yielding a scaled ROI. Next, Z-score standardization is applied to each pixel value within the scaled ROI. This involves subtracting the average pixel value from the average pixel value of all pixels in the scaled ROI and then dividing by the standard deviation of all pixels in the scaled ROI, ensuring that the standardized pixel values ​​follow a standard normal distribution with a mean of 0 and a variance of 1. Specifically, the standardized pixel value is equal to the quotient obtained by subtracting the average pixel value from the average pixel value and then dividing by the standard deviation, thus obtaining the standardized ROI corresponding to the serial number region and the solder joint region.

[0008] In a preferred embodiment, the specific process of obtaining the standardized tensor is as follows: The normalized region of interest corresponding to the serial number region and the normalized region of interest corresponding to the solder joint region are spliced ​​along the channel dimension to form a normalized tensor with 6 channels. The height of the normalized tensor is equal to the target height and the width is equal to the target width. Each channel carries the normalized pixel values ​​of the normalized region of interest corresponding to the serial number region or the normalized region of interest corresponding to the solder joint region in the red, green and blue color channels.

[0009] In a preferred embodiment, in step S2, the shared convolutional backbone network adopts the MobileNetV3-Large structure. The MobileNetV3-Large structure is composed of multiple depthwise separable convolutional blocks and linear bottleneck layers stacked end to end. The whole structure includes 5 downsampling stages. In the first 4 downsampling stages, each downsampling stage sets a separable convolution with a stride of 2 at the beginning to reduce the spatial resolution and increase the number of channels. The 5th downsampling stage keeps the resolution unchanged to extract high-dimensional features. After 5 stages of processing, the spatial size of the feature map is reduced to 1 / 32 of the input normalized tensor, and the number of channels is expanded to 960 dimensions, thereby extracting the feature map output by the shared convolutional backbone network containing global and local features of the sequence number region and the solder joint region.

[0010] In a preferred embodiment, the channel attention module performs a channel-dimensional global average pooling operation on the feature map output by the shared convolutional backbone network, compressing the two-dimensional spatial features of each channel into a scalar value, thereby generating a feature vector containing 960 scalar elements, each element of the feature vector representing the global response intensity of the corresponding channel; then the feature vector is input into the first fully connected network, and the dimension is linearly compressed from 960 dimensions to 240 dimensions through matrix multiplication, and a ReLU nonlinear transformation is applied to the compressed value, the ReLU nonlinear transformation expression is f(x)=max(0,x), where x is any scalar output by the first fully connected network; Next, the numerical values ​​after ReLU nonlinear transformation are input into the second fully connected network. The dimensions are then linearly restored from 240 to 960 dimensions through matrix multiplication. The restored values ​​are then subjected to probability normalization using the Sigmoid function, expressed as f(x)=1 / (1+e^{-x}), which limits the values ​​to between 0 and 1. This outputs the channel weights, which contain 960 values, each representing the importance of a channel in the feature map output by the shared convolutional backbone network. Finally, the channel weights are multiplied one channel at a time with the feature map output by the shared convolutional backbone network. This involves multiplying the two-dimensional feature matrix of each channel in the feature map by the corresponding value in the channel weight, thereby generating the enhanced feature map.

[0011] In a preferred embodiment, in step S3, the OCR recognition head performs a convolution operation with a stride of 2 on the enhanced feature map, compressing the height dimension of the enhanced feature map to 1 while keeping the width dimension unchanged, to obtain a sequence feature map. The sequence feature map is unfolded into a time step sequence along the width direction, with each time step corresponding to a feature vector at a location. The time step sequence is then input into a bidirectional LSTM network, which consists of forward LSTM units and backward LSTM units. These units extract the sequence context dependencies from front to back and from back to front, respectively, and output the hidden state sequence. Each hidden state is mapped to the character category space through a fully connected layer and normalized using the Softmax function to obtain a character probability matrix. The number of rows in the character probability matrix equals the width dimension, and the number of columns equals the total number of character categories. The first confidence level is defined as the geometric mean of the maximum probability values ​​at all time steps in the character probability matrix. The calculation process is as follows: first, obtain the maximum probability value corresponding to each time step in the character probability matrix, calculate the natural logarithm of the maximum probability value, sum the natural logarithm results of all time steps, divide the sum by the total number of time steps, and take the exponentiation of the division result. The resulting value is the first confidence level. The CTC greedy decoding algorithm is used on the character probability matrix to remove whitespace and duplicate characters to obtain the sequence number string.

[0012] In a preferred embodiment, the target detection head applies a feature pyramid network to the enhanced feature map to perform multi-scale feature fusion. The feature pyramid network generates multi-layer pyramid features through top-down paths and lateral connections. Each layer of pyramid features has a different spatial resolution and the same number of channels. Each pyramid feature layer is followed by a lightweight detection head, which includes a classification sub-network and a regression sub-network. The classification sub-network outputs the class probability of each anchor box, and the regression sub-network outputs the bounding box offset. After non-maximum suppression of the detection results of all layers, the detection boxes with a confidence level higher than a preset second confidence threshold are retained to form a defect list. Each element in the defect list includes the defect category, bounding box coordinates, and second confidence level. The learnable gating vector is generated by a gating network. The gating network takes the concatenated vector obtained by processing the enhanced feature map with global average pooling and global max pooling operations as input, maps it to a two-dimensional vector through a fully connected layer, and then normalizes it through the Softmax function to obtain two components. The normalization process of the Softmax function is as follows: The first element of the two-dimensional vector output by the fully connected layer is subtracted from the maximum value of all elements in the two-dimensional vector. The difference is multiplied by a preset temperature coefficient and then exponentially calculated to obtain the first intermediate value. The second element of the two-dimensional vector output by the fully connected layer is subtracted from the maximum value of all elements in the two-dimensional vector. The difference is multiplied by a preset temperature coefficient and then exponentially calculated to obtain the second intermediate value. The first intermediate value is divided by the sum of the two intermediate values ​​to obtain the first component. The second intermediate value is divided by the sum of the two intermediate values ​​to obtain the second component. The sum of the two components is 1. The first component corresponds to the scaling factor of the backpropagation gradient of the OCR recognition head, and the second component corresponds to the scaling factor of the backpropagation gradient of the object detection head, thus achieving a dynamic balance between the gradients of the two tasks.

[0013] In a preferred embodiment, step S4 includes a first confidence verification branch and a second confidence verification branch. In the first confidence verification branch, when the first confidence output by the OCR recognition head is less than the preset first confidence threshold, the bounding box coordinates of the solder joint area are extracted from the defect list. Based on the relative positional relationship between the solder joint area and the serial number area determined in step S1, the inferred position of the serial number area is calculated. OCR recognition is re-executed on the image area corresponding to the inferred position. The serial number string obtained by re-recognition is used as the final serial number string, and the confidence after re-recognition is updated to the first confidence. In the second confidence verification branch, when the second confidence output by the target detection head is less than the preset second confidence threshold, the position of the sequence number region determined by the center coordinates and rotation angle of the matching region in step S1 is used as a spatial prior. Each detection box in the defect list is traversed, and detection boxes with an overlap area with the sequence number region exceeding the preset intersection-union ratio threshold are deleted. The remaining detection boxes are retained to form an updated defect list, and the maximum value of the second confidence of all detection boxes in the updated defect list is used as the updated second confidence. If the first confidence level after re-identification is still lower than the preset first confidence level threshold, the overall confidence level will be marked as pending manual review; if the updated second confidence level is still lower than the preset second confidence level threshold, the overall confidence level will also be marked as pending manual review.

[0014] In a preferred embodiment, after mutual correction using spatial relationships, the sequence number string and the defect list are packaged into a structured data packet. The structured data packet includes a sequence number string field, a defect list field, and an overall confidence flag field. The value of the overall confidence flag field is determined based on the first confidence level and the second confidence level after bidirectional verification. When the first confidence level is greater than or equal to a preset first confidence level threshold and the second confidence level is greater than or equal to a preset second confidence level threshold, the overall confidence flag is set to normal; otherwise, it is set to await manual review. Through the VisionMaster event triggering mechanism, in the next clock cycle after the completion of a single forward inference, the VisionMaster's PLC communication module is called to encapsulate the structured data packet according to the Modbus TCP protocol format and send it to the production line control system. After sending, the global variables storing the sequence number string and the defect list are cleared to prepare for the processing of the next frame image. The structured data packet also includes a timestamp field to record the time when this inference is completed.

[0015] The beneficial effects of this invention are as follows: by sharing the convolutional backbone network and the channel attention module, it achieves simultaneous serial number recognition and solder joint defect detection in a single image acquisition, avoiding repeated feature extraction and significantly improving inference speed; the learnable gating vector dynamically balances the gradients of the two tasks, enhancing the model's adaptability to different scenarios; the bidirectional verification mechanism uses spatial relationships to mutually correct each other, improving the reliability of the output results; the entire process is completed within the VisionMaster platform, requiring no additional hardware, reducing deployment costs, and meeting the dual requirements of real-time performance and accuracy for high-speed production lines. Attached Figure Description

[0016] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0019] In the description of this application, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0020] Example 1: This example provides the following... Figure 1 The method for multi-task industrial vision collaborative inspection based on VisionMaster is shown, and specifically includes the following steps: Step S1: Configure the high-resolution industrial camera and receive the trigger signal through the VisionMaster workflow editor, acquire PCB board images, use the template matching module built into VisionMaster to automatically identify and crop the region of interest containing the serial number area and solder joint area, perform distortion correction on the region of interest and perform multi-scale normalization processing to obtain the normalized tensor. Step S2: Feed the standardized tensor into the shared convolutional backbone network for feature extraction, and embed a channel attention module at the end of the shared convolutional backbone network. Learn the channel weights through global average pooling and two fully connected networks, and perform channel-wise weighting on the feature map output by the shared convolutional backbone network to output an enhanced feature map. Step S3: The enhanced feature map is simultaneously fed into the OCR recognition head and the target detection head for parallel decoding. The OCR recognition head compresses the height through convolution and outputs a character probability matrix and a first confidence score through a bidirectional LSTM network. The character probability matrix is ​​decoded to obtain a sequence number string. The target detection head performs multi-scale fusion through a feature pyramid network and outputs a defect list and a second confidence score through a lightweight detection head. At the same time, a learnable gating vector is introduced to dynamically scale the backpropagation gradient corresponding to the OCR recognition head and the backpropagation gradient corresponding to the target detection head to achieve adaptive balance. Step S4: Synchronize the serial number string output by the OCR recognition head and the defect list output by the target detection head through the VisionMaster global variable interface. Perform bidirectional verification based on the preset first confidence threshold and the preset second confidence threshold. After mutual correction using spatial relationships, call the VisionMaster PLC communication module through the VisionMaster event triggering mechanism to package and send the results to the production line control system, completing the multi-task collaborative output of a single forward inference.

[0021] In this embodiment, it is specifically necessary to explain that in step S1, the working parameters of the high-resolution industrial camera are configured through the VisionMaster workflow editor, including setting the exposure time to adaptive mode or fixed millisecond value, setting the gain to automatic gain control or fixed decibel value, and setting the trigger mode to hardware rising edge trigger or continuous acquisition mode. A high-resolution industrial camera receives a trigger signal from the production line control system and accurately captures an image of the PCB board when it arrives at the designated workstation. Using the template matching module built into VisionMaster, a pre-stored PCB board reference template is loaded. A normalized cross-correlation algorithm is used to search for the position in the captured PCB board image that best matches the reference template, returning the center coordinates and rotation angle of the matching area. Based on the center coordinates and rotation angle of the matching area, the region of interest, including the serial number area and solder joint area, is automatically cropped. In step S1, when the exposure time of the high-resolution industrial camera is set to adaptive mode, the camera automatically adjusts the exposure time according to the ambient light intensity, typically ranging from 0.1 milliseconds to 10 milliseconds. Gain setting... When automatic gain control is enabled, the gain is automatically adjusted between 1x and 16x. When the trigger mode is set to hardware rising edge trigger, the camera immediately acquires an image after receiving the rising edge pulse signal from the production line control system. The pulse width is not less than 1 microsecond to ensure reliable triggering. The PCB reference template is a pre-stored grayscale image with a size of 128 pixels by 128 pixels. The normalized cross-correlation threshold for template matching is set to 0.85. Positioning is considered successful only when the matching score is greater than 0.85; otherwise, an alarm is triggered and manual intervention is required. The center coordinates and rotation angle of the matching area are used to accurately crop the region of interest. The minimum bounding rectangle size of the region of interest is 256 pixels by 512 pixels to ensure complete coverage of the serial number area and solder joint area. The specific steps for performing distortion correction and multi-scale normalization on the region of interest are as follows: First, using the radial and tangential distortion coefficients obtained during the high-resolution industrial camera calibration phase, a bilinear interpolation algorithm is used to map the coordinates of each pixel within the region of interest (ROI) to the coordinates under an ideal pinhole model, eliminating geometric distortion caused by lens distortion and obtaining a corrected ROI. Then, the corrected ROI is scaled to a fixed target height and width, using bilinear interpolation to avoid losing high-frequency information, resulting in a scaled ROI. Next, Z-score standardization is performed on each pixel value within the scaled ROI. This involves subtracting the average pixel value of all pixels in the scaled ROI from each pixel value and then dividing by the standard deviation of all pixels in the scaled ROI, ensuring that the standardized pixel values ​​follow a standard normal distribution with a mean of 0 and a variance of 1. This eliminates brightness shifts caused by lighting differences between different batches of PCB images. Specifically, the standardized pixel value is equal to the quotient obtained by subtracting the average pixel value from the pixel value and dividing by the standard deviation, yielding the standardized ROI corresponding to the serial number area and the solder joint area. Distortion correction is then performed on the ROI. In this process, the radial distortion coefficients k1 and k2 and the tangential distortion coefficients p1 and p2 are pre-obtained using a checkerboard calibration method, with typical values ​​of k1=-0.25, k2=0.08, p1=0.001, and p2=-0.002, respectively. The bilinear interpolation algorithm calculates the mapped pixel value using the weighted average of the four nearest neighbor pixels, with the weights determined by the reciprocal of the distance. The corrected region of interest is scaled to a target height of 256 pixels and a target width of 512 pixels, a size that balances the resolution requirements of subsequent feature extraction with inference speed. During Z-score normalization, the average... Both pixel values ​​and standard deviations are calculated based on all pixel values ​​within the scaled region of interest. The average pixel value typically ranges from 80 to 150 (for an 8-bit grayscale image), and the standard deviation ranges from 40 to 70. The standardized pixel values ​​are truncated to between -3 and 3 to prevent extreme values ​​from affecting the stability of subsequent neural network training. Through the above processing, the standardized regions of interest corresponding to the sequence number region and the standardized regions of interest corresponding to the solder joint region have the same size and pixel distribution characteristics, providing a consistent data distribution input for the shared backbone network. The specific process of obtaining the standardized tensor is as follows: The normalized region of interest (ROI) corresponding to the sequence number region and the normalized ROI corresponding to the solder joint region are concatenated along the channel dimension to form a normalized tensor with 6 channels. The height and width of the normalized tensor are equal to the target height and target width, respectively. Each channel carries the normalized pixel values ​​of the normalized ROI corresponding to the sequence number region or the normalized ROI corresponding to the solder joint region in the red, green, and blue color channels, which serve as the sole input to the shared convolutional backbone network in step S2, while preserving the spatial relationship and photometric consistency between the sequence number region and the solder joint region. When concatenating to obtain the normalized tensor, the normalized ROI corresponding to the sequence number region and the normalized ROI corresponding to the solder joint region are concatenated along the channel dimension to form a normalized tensor with 6 channels. The regions are all three-channel color images (red, green, and blue), so they are stitched together along the channel dimension to form a 6-channel tensor. The spatial dimension of this 6-channel tensor is 256 pixels by 512 pixels, and the channel order is as follows: the first three channels are the red, green, and blue channels of the sequence number region, and the last three channels are the red, green, and blue channels of the solder joint region. The normalized tensor serves as the sole input to the shared convolutional backbone network in step S2, and its data range is between -3 and 3, with a data type of 32-bit floating-point numbers. By fusing the information of the two regions into one tensor, the shared backbone network can simultaneously perceive the texture features of the sequence number and the geometric features of the solder joint during the same feature extraction process, avoiding the feature isolation problem caused by processing the two regions separately, thereby improving the efficiency of multi-task collaborative detection.

[0022] In this embodiment, it is particularly important to note that in step S2, the shared convolutional backbone network adopts the MobileNetV3-Large structure. This MobileNetV3-Large structure is composed of multiple depthwise separable convolutional blocks and linear bottleneck layers stacked end to end. It includes five downsampling stages. In the first four downsampling stages, each stage begins with a separable convolution with a stride of 2 to reduce spatial resolution and increase the number of channels. The fifth downsampling stage maintains the resolution to extract high-dimensional features. After these five stages, the spatial size of the feature map is reduced to 1 / 32 of the input normalized tensor, and the number of channels expands to 960 dimensions, thereby extracting the feature map output by the shared convolutional backbone network, which contains global and local features of the sequence number region and solder joint region. In this MobileNetV3-Large structure, each depthwise separable convolutional block contains one depthwise convolution and one pointwise convolution. The kernel size of the depthwise convolution is 3. 3. The kernel size of pointwise convolution is 1. 1. Each linear bottleneck layer contains one dilated convolution, one depthwise convolution, and one projective convolution. The dilation factor is set to 6, meaning the input channel number is first expanded by a factor of 6, then features are extracted through depthwise convolution, and finally compressed back to the original channel number through projective convolution. The specific channel number configurations for the five downsampling stages are as follows: the first downsampling stage outputs 16 dimensions, the second downsampling stage outputs 24 dimensions, the third downsampling stage outputs 40 dimensions, the fourth downsampling stage outputs 112 dimensions, and the fifth downsampling stage outputs 960 dimensions. Each convolution operation is followed by a batch normalization layer and a Hard-Swish activation function to accelerate training convergence and improve feature representation capabilities. The design purpose of this MobileNetV3-Large structure is to extract sufficiently rich multi-scale features while maintaining low computational cost, so as to meet the needs of subsequent OCR recognition heads and object detection heads for information of different granularities. The channel attention module performs channel-dimensional global average pooling on the feature maps output by the shared convolutional backbone network, compressing the two-dimensional spatial features of each channel into a scalar value, thereby generating a feature vector containing 960 scalar elements. Each element of the feature vector represents the global response intensity of the corresponding channel. The feature vector is then input into the first fully connected layer, where matrix multiplication linearly compresses the dimension from 960 to 240. A ReLU nonlinear transformation is then applied to the compressed value, expressed as f(x) = max(0,x), where x is any scalar output from the first fully connected layer, resulting in a 240-dimensional nonlinear feature. The first-layer weight matrix has a dimension of 960. 240, the bias vector length is 240; Next, the numerical values ​​after ReLU nonlinear transformation are input into the second fully connected network. The dimensions are then linearly restored from 240 to 960 dimensions through matrix multiplication. The restored values ​​are then subjected to probability normalization using the Sigmoid function, expressed as f(x)=1 / (1+e^{-x}), which limits the values ​​to between 0 and 1. This outputs the channel weights, which contain 960 values, each representing the importance of a channel in the feature map output by the shared convolutional backbone network. Finally, the channel weights are multiplied channel-by-channel with the feature map output by the shared convolutional backbone network. This involves multiplying the two-dimensional feature matrix of each channel in the feature map by the corresponding value in the channel weight, thus generating an enhanced feature map. In this process, the enhanced feature map completely retains the original spatial structure of the feature map output by the shared convolutional backbone network in the channel dimension. The relative importance of each channel is recalibrated through the channel weights. Feature channels relevant to both sequence number recognition and solder joint detection are amplified, while task-irrelevant or noisy channels are suppressed, resulting in the direct output of the enhanced feature map. Specifically, the global average pooling operation compresses the two-dimensional feature map of each channel (spatial size of eight pixels by sixteen pixels) into a scalar, resulting in a one-dimensional vector of length 960. The weight matrix of the first fully connected network has a dimension of 960 rows. The second layer of the fully connected network has 240 columns, a bias vector length of 240, and uses the ReLU function for nonlinear transformation, with the expression f(x) = max(0,x). The weight matrix of the second layer has 240 rows. The feature map has 960 columns, a bias vector length of 960, and uses the Sigmoid function for probability normalization, expressed as f(x) = 1 / (1 + e^{-x}). The reduction ratio r is set to 4, meaning the intermediate layer dimension is 1 / 4 of the input dimension. Through the aforementioned channel attention mechanism, the enhanced feature map retains the original spatial structure in the channel dimension, but the response values ​​of each channel are reweighted. This results in higher weights for channels that contribute to both downstream tasks (such as character edges, solder joint contours, etc.), while the weights of background noise or irrelevant texture channels are reduced, thereby improving the accuracy and robustness of subsequent parallel decoding. The spatial size of the enhanced feature map remains 8 pixels. 16 pixels, with 960 channels, serves as the common input for both the OCR recognition head and the target detection head in step S3.

[0023] In this embodiment, it is specifically noted that in step S3, the OCR recognition head performs a convolution operation with a stride of 2 on the enhanced feature map, compressing the height dimension of the enhanced feature map to 1 while keeping the width dimension unchanged, to obtain the sequence feature map. The sequence feature map is unfolded into a time step sequence along the width direction, with each time step corresponding to a feature vector at a location. The time step sequence is then input into a bidirectional LSTM network, which consists of forward LSTM units and backward LSTM units. These units extract the sequence context dependencies from front to back and from back to front, respectively, and output the hidden state sequence. Each hidden state is mapped to the character category space through a fully connected layer and normalized using the Softmax function to obtain a character probability matrix. The number of rows in the character probability matrix equals the width dimension, and the number of columns equals the total number of character categories. The first confidence level is defined as the geometric mean of the maximum probability values ​​at all time steps in the character probability matrix. The calculation process is as follows: first, obtain the maximum probability value corresponding to each time step in the character probability matrix, calculate the natural logarithm of the maximum probability value, sum the natural logarithm results of all time steps, divide the sum by the total number of time steps, and take the exponentiation of the division result. The resulting value is the first confidence level. The closer the first confidence level is to 1, the higher the overall reliability of OCR recognition. The CTC greedy decoding algorithm is used on the character probability matrix to remove whitespace and duplicate characters to obtain the sequence number string. In this OCR recognition head, the hidden layer dimension of the bidirectional LSTM network is set to 128, meaning that the forward LSTM unit and the backward LSTM unit each output 128-dimensional hidden states, which are concatenated to obtain a 256-dimensional hidden state sequence. The output dimension of the fully connected layer is equal to the total number of character categories, which includes digits 0 to 9, English letters A to Z, and whitespace, totaling 37 categories. The CTC greedy decoding algorithm selects the character with the highest probability at each time step, then merges adjacent identical characters and removes whitespace to obtain the final sequence number string. In the calculation of the first confidence level, the total number of time steps is equal to the width dimension value of 16, so the first confidence level is the geometric mean of the 16 maximum probabilities, with a value between 0 and 1. A typical value above 0.95 indicates reliable recognition. The target detection head applies a feature pyramid network to the enhanced feature map for multi-scale feature fusion. The feature pyramid network generates multi-layer pyramid features through top-down paths and lateral connections. Each layer of pyramid features has a different spatial resolution and the same number of channels. Each pyramid feature layer is followed by a lightweight detection head, which includes a classification sub-network and a regression sub-network. The classification sub-network outputs the class probability of each anchor box, and the regression sub-network outputs the bounding box offset. After non-maximum suppression of the detection results of all layers, the detection boxes with a confidence level higher than a preset second confidence threshold are retained to form a defect list. Each element in the defect list includes the defect category, bounding box coordinates, and second confidence level. The learnable gating vector is generated by a gating network. The gating network takes the concatenated vector obtained by processing the enhanced feature map with global average pooling and global max pooling operations as input, maps it to a two-dimensional vector through a fully connected layer, and then normalizes it through the Softmax function to obtain two components. The normalization process of the Softmax function is as follows: The first element of the two-dimensional vector output by the fully connected layer is subtracted from the maximum value of all elements in the two-dimensional vector. The difference is multiplied by a preset temperature coefficient and then exponentially calculated to obtain the first intermediate value. The second element of the two-dimensional vector output by the fully connected layer is subtracted from the maximum value of all elements in the two-dimensional vector. The difference is multiplied by a preset temperature coefficient and then exponentially calculated to obtain the second intermediate value. The first intermediate value is divided by the sum of the two intermediate values ​​to obtain the first component. The second intermediate value is divided by the sum of the two intermediate values ​​to obtain the second component. The sum of the two components is 1. The first component corresponds to the scaling factor of the backpropagation gradient of the OCR recognition head, and the second component corresponds to the scaling factor of the backpropagation gradient of the object detection head, thus achieving a dynamic balance between the gradients of the two tasks. In this target detection head, the feature pyramid network generates three layers of pyramid features with a spatial resolution of 8 pixels each. 16 pixels, 4 pixels 8 pixels, 2 pixels 4 pixels, 256 channels per layer; the classification and regression subnetworks of the lightweight detector head are both composed of 3 stacked convolutional layers, with each convolutional layer having a kernel size of 3. 3. The padding is 1, the stride is 1, and the number of output channels is related to the number of anchor boxes. Nine prior boxes of different scales and aspect ratios are preset for each feature point. The preset second confidence threshold is set to 0.5, meaning only detection boxes with a classification score greater than 0.5 are retained. The crossover ratio threshold for non-maximum suppression is set to 0.45 to remove overlapping boxes. The fully connected layer in the gated network has an input dimension of 1920 (composed of 960-dimensional global average pooling and 960-dimensional global max pooling results) and an output dimension of 2. The preset temperature coefficient is set to 1 to control the smoothness of the Softmax distribution. A larger temperature coefficient indicates a closer balance between the two components, while a smaller temperature coefficient indicates a more pronounced difference between the components. In the early stages of training, the temperature coefficient can be set to a larger value (e.g., 1.5) to encourage exploration, and then gradually reduced to 0.5 to enhance task differentiation. Through this gating mechanism, the model can automatically adjust the gradient ratio of the two tasks based on the input image features, avoiding one task dominating training and improving the stability of multi-task collaboration.

[0024] In this embodiment, it is specifically necessary to explain that in step S4, the bidirectional verification includes a first confidence verification branch and a second confidence verification branch. In the first confidence verification branch, when the first confidence output by the OCR recognition head is less than the preset first confidence threshold, the bounding box coordinates of the solder joint area are extracted from the defect list. Based on the relative positional relationship between the solder joint area and the serial number area determined in step S1, the inferred position of the serial number area is calculated. OCR recognition is re-executed on the image area corresponding to the inferred position. The serial number string obtained by re-recognition is used as the final serial number string, and the confidence after re-recognition is updated to the first confidence. In the second confidence verification branch, when the second confidence output by the target detection head is less than the preset second confidence threshold, the position of the sequence number region determined by the center coordinates and rotation angle of the matching region in step S1 is used as a spatial prior. Each detection box in the defect list is traversed, and detection boxes with an overlap area with the sequence number region exceeding the preset intersection-union ratio threshold are deleted. The remaining detection boxes are retained to form an updated defect list, and the maximum value of the second confidence of all detection boxes in the updated defect list is used as the updated second confidence. If the first confidence level after re-identification is still lower than the preset first confidence level threshold, the overall confidence level is marked as pending manual review; if the updated second confidence level is still lower than the preset second confidence level threshold, the overall confidence level is also marked as pending manual review. In this bidirectional verification process, the preset first confidence threshold is set to 0.85, the preset second confidence threshold is set to 0.75, and the preset intersection-over-union (IoU) threshold is set to 0.3. The relative positional relationship between the solder joint area and the serial number area is pre-calculated and stored based on the difference between the center coordinates of the two regions of interest determined by template matching in step S1. This difference is a fixed value in both the horizontal and vertical directions, and the unit is pixels. When it is necessary to re-perform OCR recognition on the image area corresponding to the inferred position, the system captures the image data of that local area and scales it. The OCR head input is set to the same size as in step S3, i.e., 32 pixels high and 128 pixels wide. Then, the trained OCR head from step S3 is reused for forward inference to obtain a new character probability matrix and a new first confidence level. In the second confidence level verification branch, when traversing each detection box in the defect list, the ratio of the intersection area to the union area of ​​the detection box and the bounding rectangle of the sequence number region is calculated. If the ratio is greater than the preset intersection-union ratio threshold of 0.3, it is determined that the detection box overlaps too much with the sequence number region and is deleted. After mutual correction using spatial relationships, the sequence number string and defect list are packaged into a structured data packet. The structured data packet includes a sequence number string field, a defect list field, and an overall confidence flag field. The value of the overall confidence flag field is determined based on the first and second confidence levels after bidirectional verification. When the first confidence level is greater than or equal to a preset first confidence threshold and the second confidence level is greater than or equal to a preset second confidence threshold, the overall confidence flag is set to normal; otherwise, it is set to await manual review. Through the VisionMaster event triggering mechanism, in the next clock cycle after the completion of a single forward inference, the VisionMaster PLC communication module is called to encapsulate the structured data packet according to the ModbusTCP protocol format and send it to the production line control system. After sending, the global variables storing the sequence number string and defect list are cleared to prepare for the processing of the next frame image. The structured data packet also includes a timestamp field to record the time when this inference was completed. During the data packaging and transmission process, the structured data packets are serialized using JSON format for easy parsing by the production line control system; the serial number string field is a variable-length string with a maximum length of 32 characters; the defect list field is an array, with each array element containing an enumeration value for the defect category (e.g., zero for cold solder joints, one for offset, and two for missing parts), the bounding box coordinates (in pixels, including the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, width, and height), and the second confidence level of the detection box; the overall confidence flag field is a boolean value, corresponding to true for normal operation and false for operation requiring manual review; the timestamp field is a Unix timestamp, accurate to milliseconds. The VisionMaster event triggering mechanism is implemented by registering a callback function. After step S3 is completed and the result is output, the callback function is immediately triggered to perform data packaging and sending operations. When encapsulating the Modbus TCP protocol, the JSON string is converted into a byte stream, and a Modbus application protocol header is added to the message header. The function code is set to hexadecimal 03 (read holding register) or 06 (write to a single register), which is configured according to the requirements of the production line control system. After the transmission is completed, the global variables Str_ID and Defect_List are cleared, and the status register is reset, waiting for the next trigger signal to arrive.

[0025] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0026] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0027] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0028] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0029] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0030] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0031] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A multi-task industrial vision collaborative inspection method based on VisionMaster, characterized in that, Specifically, the steps include the following: Step S1: Configure the high-resolution industrial camera and receive the trigger signal through the VisionMaster workflow editor, acquire PCB board images, use the template matching module built into VisionMaster to automatically identify and crop the region of interest containing the serial number area and solder joint area, perform distortion correction on the region of interest and perform multi-scale normalization processing to obtain the normalized tensor. Step S2: Feed the standardized tensor into the shared convolutional backbone network for feature extraction, and embed a channel attention module at the end of the shared convolutional backbone network. Learn the channel weights through global average pooling and two fully connected networks, and perform channel-wise weighting on the feature map output by the shared convolutional backbone network to output an enhanced feature map. Step S3: The enhanced feature map is simultaneously fed into the OCR recognition head and the target detection head for parallel decoding. The OCR recognition head compresses the height through convolution and outputs a character probability matrix and a first confidence score through a bidirectional LSTM network. The character probability matrix is ​​decoded to obtain a sequence number string. The target detection head performs multi-scale fusion through a feature pyramid network and outputs a defect list and a second confidence score through a lightweight detection head. At the same time, a learnable gating vector is introduced to dynamically scale the backpropagation gradient corresponding to the OCR recognition head and the backpropagation gradient corresponding to the target detection head to achieve adaptive balance. Step S4: Synchronize the serial number string output by the OCR recognition head and the defect list output by the target detection head through the VisionMaster global variable interface. Perform bidirectional verification based on the preset first confidence threshold and the preset second confidence threshold. After mutual correction using spatial relationships, call the VisionMaster PLC communication module through the VisionMaster event triggering mechanism to package and send the results to the production line control system, completing the multi-task collaborative output of a single forward inference.

2. The multi-task industrial vision collaborative inspection method based on VisionMaster according to claim 1, characterized in that: In step S1, the working parameters of the high-resolution industrial camera are configured through the VisionMaster workflow editor, including setting the exposure time to adaptive mode or a fixed millisecond value, setting the gain to automatic gain control or a fixed decibel value, and setting the trigger mode to hardware rising edge trigger or continuous acquisition mode. A high-resolution industrial camera receives trigger signals from the production line control system and accurately captures an image of the PCB board when it arrives at the designated station. Using the template matching module built into VisionMaster, a pre-stored PCB board reference template is loaded. A normalized cross-correlation algorithm is used to search for the position in the captured PCB board image that best matches the PCB board reference template, and the center coordinates and rotation angle of the matching area are returned. Based on the center coordinates and rotation angle of the matching area, the region of interest, which includes the serial number area and the solder joint area, is automatically cropped.

3. The multi-task industrial vision collaborative inspection method based on VisionMaster according to claim 2, characterized in that: The specific operations for performing distortion correction and multi-scale normalization on the region of interest are as follows: First, using the radial and tangential distortion coefficients obtained during the high-resolution industrial camera calibration phase, the coordinates of each pixel within the region of interest (ROI) are mapped to the coordinates under an ideal pinhole model using a bilinear interpolation algorithm, resulting in a corrected ROI. Then, the corrected ROI is scaled to a fixed target height and width using bilinear interpolation, yielding a scaled ROI. Next, Z-score standardization is applied to each pixel value within the scaled ROI. This involves subtracting the average pixel value from the average pixel value of all pixels in the scaled ROI and then dividing by the standard deviation of all pixels in the scaled ROI, ensuring that the standardized pixel values ​​follow a standard normal distribution with a mean of 0 and a variance of 1. Specifically, the standardized pixel value is equal to the quotient obtained by subtracting the average pixel value from the average pixel value and then dividing by the standard deviation, thus obtaining the standardized ROI corresponding to the serial number region and the solder joint region.

4. The multi-task industrial vision collaborative inspection method based on VisionMaster according to claim 3, characterized in that: The specific process for obtaining the standardized tensor is as follows: The normalized region of interest corresponding to the serial number region and the normalized region of interest corresponding to the solder joint region are spliced ​​along the channel dimension to form a normalized tensor with 6 channels. The height of the normalized tensor is equal to the target height and the width is equal to the target width. Each channel carries the normalized pixel values ​​of the normalized region of interest corresponding to the serial number region or the normalized region of interest corresponding to the solder joint region in the red, green and blue color channels.

5. The multi-task industrial vision collaborative inspection method based on VisionMaster according to claim 4, characterized in that: In step S2, the shared convolutional backbone network adopts the MobileNetV3-Large structure. This MobileNetV3-Large structure is composed of multiple depthwise separable convolutional blocks and linear bottleneck layers stacked end to end. The whole structure includes 5 downsampling stages. In the first 4 downsampling stages, each downsampling stage sets a separable convolution with a stride of 2 at the beginning to reduce the spatial resolution and increase the number of channels. The 5th downsampling stage keeps the resolution unchanged to extract high-dimensional features. After 5 stages of processing, the spatial size of the feature map is reduced to 1 / 32 of the input normalized tensor, and the number of channels is expanded to 960 dimensions, thereby extracting the feature map output by the shared convolutional backbone network containing global and local features of the sequence number region and the solder joint region.

6. The multi-task industrial vision collaborative inspection method based on VisionMaster according to claim 5, characterized in that: The channel attention module performs a channel-dimensional global average pooling operation on the feature map output by the shared convolutional backbone network, compressing the two-dimensional spatial features of each channel into a scalar value, thereby generating a feature vector containing 960 scalar elements. Each element of the feature vector represents the global response intensity of the corresponding channel. The feature vector is then input into the first fully connected network layer, and the dimension is linearly compressed from 960 dimensions to 240 dimensions through matrix multiplication. A ReLU nonlinear transformation is then applied to the compressed value. The ReLU nonlinear transformation expression is f(x)=max(0,x), where x is any scalar output by the first fully connected network layer. Next, the numerical values ​​after ReLU nonlinear transformation are input into the second fully connected network. The dimensions are then linearly restored from 240 to 960 dimensions through matrix multiplication. The restored values ​​are then subjected to probability normalization using the Sigmoid function, expressed as f(x)=1 / (1+e^{-x}), which limits the values ​​to between 0 and 1. This outputs the channel weights, which contain 960 values, each representing the importance of a channel in the feature map output by the shared convolutional backbone network. Finally, the channel weights are multiplied one channel at a time with the feature map output by the shared convolutional backbone network. This involves multiplying the two-dimensional feature matrix of each channel in the feature map by the corresponding value in the channel weight, thereby generating the enhanced feature map.

7. The multi-task industrial vision collaborative inspection method based on VisionMaster according to claim 6, characterized in that: In step S3, the OCR recognition head performs a convolution operation with a stride of 2 on the enhanced feature map, compressing the height dimension of the enhanced feature map to 1 while keeping the width dimension unchanged, to obtain a sequence feature map. The sequence feature map is unfolded into a time step sequence along the width direction, with each time step corresponding to a feature vector at a location. The time step sequence is then input into a bidirectional LSTM network, which consists of forward LSTM units and backward LSTM units. These units extract the sequence context dependencies from front to back and from back to front, respectively, and output the hidden state sequence. Each hidden state is mapped to the character category space through a fully connected layer and normalized using the Softmax function to obtain a character probability matrix. The number of rows in the character probability matrix equals the width dimension, and the number of columns equals the total number of character categories. The first confidence level is defined as the geometric mean of the maximum probability values ​​at all time steps in the character probability matrix. The calculation process is as follows: first, obtain the maximum probability value corresponding to each time step in the character probability matrix, calculate the natural logarithm of the maximum probability value, sum the natural logarithm results of all time steps, divide the sum by the total number of time steps, and take the exponentiation of the division result. The resulting value is the first confidence level. The CTC greedy decoding algorithm is used on the character probability matrix to remove whitespace and duplicate characters to obtain the sequence number string.

8. The multi-task industrial vision collaborative inspection method based on VisionMaster according to claim 7, characterized in that: The target detection head applies a feature pyramid network to the enhanced feature map for multi-scale feature fusion. The feature pyramid network generates multi-layer pyramid features through top-down paths and lateral connections. Each layer of pyramid features has a different spatial resolution and the same number of channels. Each pyramid feature layer is followed by a lightweight detection head, which contains a classification subnetwork and a regression subnetwork. The classification subnetwork outputs the class probability of each anchor box, and the regression subnetwork outputs the bounding box offset. After performing non-maximum suppression on the detection results of all levels, the detection boxes with confidence scores higher than the preset second confidence threshold are retained to form a defect list. Each element in the defect list contains the defect category, bounding box coordinates, and second confidence score. The learnable gating vector is generated by a gating network. The gating network takes the concatenated vector obtained by processing the enhanced feature map with global average pooling and global max pooling operations as input, maps it to a two-dimensional vector through a fully connected layer, and then normalizes it through the Softmax function to obtain two components. The normalization process of the Softmax function is as follows: The first element of the two-dimensional vector output by the fully connected layer is subtracted from the maximum value of all elements in the two-dimensional vector. The difference is multiplied by a preset temperature coefficient and then exponentially calculated to obtain the first intermediate value. The second element of the two-dimensional vector output by the fully connected layer is subtracted from the maximum value of all elements in the two-dimensional vector. The difference is multiplied by a preset temperature coefficient and then exponentially calculated to obtain the second intermediate value. The first intermediate value is divided by the sum of the two intermediate values ​​to obtain the first component. The second intermediate value is divided by the sum of the two intermediate values ​​to obtain the second component. The sum of the two components is 1. The first component corresponds to the scaling factor of the backpropagation gradient of the OCR recognition head, and the second component corresponds to the scaling factor of the backpropagation gradient of the object detection head, thus achieving a dynamic balance between the gradients of the two tasks.

9. The multi-task industrial vision collaborative inspection method based on VisionMaster according to claim 8, characterized in that: In step S4, the bidirectional verification includes a first confidence verification branch and a second confidence verification branch. In the first confidence verification branch, when the first confidence output by the OCR recognition head is less than the preset first confidence threshold, the bounding box coordinates of the solder joint area are extracted from the defect list. Based on the relative positional relationship between the solder joint area and the serial number area determined in step S1, the inferred position of the serial number area is calculated. OCR recognition is re-executed on the image area corresponding to the inferred position. The serial number string obtained by re-recognition is used as the final serial number string, and the confidence after re-recognition is updated to the first confidence. In the second confidence verification branch, when the second confidence output by the target detection head is less than the preset second confidence threshold, the position of the sequence number region determined by the center coordinates and rotation angle of the matching region in step S1 is used as a spatial prior. Each detection box in the defect list is traversed, and detection boxes with an overlap area with the sequence number region exceeding the preset intersection-union ratio threshold are deleted. The remaining detection boxes are retained to form an updated defect list, and the maximum value of the second confidence of all detection boxes in the updated defect list is used as the updated second confidence. If the first confidence level after re-identification is still lower than the preset first confidence level threshold, the overall confidence level will be marked as requiring manual review. If the updated second confidence level is still lower than the preset second confidence level threshold, the overall confidence level will also be marked as pending manual review.

10. The multi-task industrial vision collaborative inspection method based on VisionMaster according to claim 9, characterized in that: After mutual correction using spatial relationships, the sequence number string and the defect list are packaged into a structured data packet. The structured data packet includes a sequence number string field, a defect list field, and an overall credibility flag field. The value of the overall confidence flag field is determined based on the first and second confidence levels after bidirectional verification. When the first confidence level is greater than or equal to the preset first confidence threshold and the second confidence level is greater than or equal to the preset second confidence threshold, the overall confidence flag is set to normal; otherwise, it is set to await manual review. Through the VisionMaster event triggering mechanism, in the next clock cycle after the completion of a single forward inference, the VisionMaster's PLC communication module is called to encapsulate the structured data packet according to the ModbusTCP protocol format and send it to the production line control system. After the transmission is completed, the global variables storing the sequence number string and the defect list are cleared to prepare for the processing of the next frame image. The structured data packet also contains a timestamp field to record the time when this inference is completed.