Industrial multi-station vision detection positioning method and program product based on pixel-level coding mask

Abstract

The application discloses an industrial multi-station visual detection positioning method and program product based on a pixel-level coding mask, belongs to the technical field of visual detection positioning, and comprises a labeling stage, a deployment stage, an inference stage and a verification stage. The coding mask image is applied to blacken the ingot area of a to-be-detected image to obtain a shielding image, an abnormality detection frame is obtained, a two-stage sampling strategy is used for positioning to obtain a station physical number set; real-time frame input is carried out, and the station physical number set is sent to a continuous frame verifier; when and only when the same station physical number is hit in N continuous frames, it is confirmed that the station hit in the N continuous frames is abnormal and an alarm is output. When the station layout is adjusted or the model of the ingot (the color of the ingot body is changed) is replaced, only the ellipse needs to be redrawn in the labeling tool and a new mask is exported, and the deep learning model does not need to be retrained or the detection code does not need to be modified.

Description

Technical Field

[0001] This invention discloses an industrial multi-station visual inspection and positioning method and program product based on pixel-level coded masks, belonging to the field of visual inspection and positioning technology. Background Technology

[0002] In industrial production lines, the field of view of a single industrial camera typically covers dozens or even hundreds of workstations (e.g., in a spinning line, a single camera covers 30-80 spinning stations). Deep learning object detection models can effectively detect abnormal areas in images, but there are two key technical bottlenecks. Bottleneck one is station-level precise positioning. The detection model can only output "there is an anomaly at a certain location," but cannot directly indicate "which workstation experienced the anomaly." Bottleneck two is the drastic variation in the color of the spindle (circular guide) itself due to differences in model / batch. In a spinning line, the yarn bundle passes through a circular guide device called a "spindle." The field of view of an industrial camera mainly captures the spindle and its surrounding area. Different models or batches of spindles may have drastically different colors—sometimes black, sometimes white, and sometimes other colors (such as the original metallic color, colored coating, etc.). The defect (broken yarn) that the model needs to detect occurs within the circular area of ​​the spindle, and the color variation of the spindle makes the same target appear against drastically different backgrounds. When a deep learning model is trained on ingots of one color, its detection accuracy often drops significantly or even fails completely when inference on ingots of another color. Traditional solutions either train a separate model for each ingot color (which is extremely costly to maintain) or rely on a large amount of multi-color labeled data (which is extremely costly to collect).

[0003] In summary, the following solutions are mainly used in existing technologies to achieve precise positioning at the workstation level: 1. After dividing the ROI by workstation, train the classification model separately: Deploy a separate classifier for each workstation. The number of models is proportional to the number of workstations. The deployment and maintenance costs are extremely high. All models need to be retrained after the workstation layout is adjusted. 2. Nearest neighbor matching between the detection frame coordinates and the workstation coordinate table: The center coordinates and range of each workstation are pre-measured, and the detection frame center is matched with the coordinate table during inference. This method has a drastic drop in matching accuracy when the camera undergoes slight displacement (such as vibration or thermal expansion and contraction), and requires complex geometric region definitions for irregularly arranged workstations. 3. Use a binary mask to limit the detection area: Use a 0 / 1 binary mask to block non-ROI areas to reduce false detections, but the mask only indicates "whether the area is of interest" and cannot distinguish between different workstations.

[0004] None of the above solutions can achieve low-cost, robust, and precise station-level positioning under the premise of a single detection model. Summary of the Invention

[0005] This invention discloses an industrial multi-station visual inspection and positioning method and program product based on pixel-level coded mask, which overcomes the problems of existing technologies, such as the need to train a model separately for each spindle color or collect a large amount of multi-color annotation data, resulting in extremely high costs, and the defects of station mapping schemes, such as high deployment costs, poor robustness, and inability to maintain ID consistency across frames.

[0006] An industrial multi-station visual inspection and localization method based on pixel-level coded masks includes: S1, annotation stage: acquire the reference image corresponding to the field of view of the industrial camera, annotate each station with an interactive ellipse on the reference image, the station being an automated detection point, automatically assign a global ID to the elliptical region of each interactive ellipse, and export the coded mask image. S2, Deployment Phase: The exported coded mask image is placed in the mask directory of the industrial inspection system. When the system starts, the MaskManager module automatically scans and loads the coded mask image; the image to be inspected is acquired in real time by the industrial camera. When the size of the coded mask image is inconsistent with the size of the image to be inspected, nearest neighbor interpolation is used for automatic alignment. After the industrial inspection system is ready, it waits for real-time frame input. S3, In the inference stage, the coded mask image is used to black out the ingot region of the image to be detected to obtain the masked image. See below. Figure 5 , Figure 6 and Figure 7 The masked image is fed into the OpenVINO or YOLO inference engine to obtain anomaly detection boxes. The model outputs the detection boxes as follows: Figure 8 A two-level sampling strategy was used for location analysis to obtain a set of physical workstation numbers; the two-level sampling strategy is described in [link to strategy]. Figure 4 ; S4, verification phase: Real-time frame input is performed, and the set of physical station numbers is sent to the continuous frame verifier. If and only if the same physical station number is hit in N consecutive frames, it is confirmed that the station that is hit in N consecutive frames has an anomaly and an alarm is output.

[0007] During the annotation phase, the coded mask image directly carries the physical number of the workstation using grayscale values. The coded mask image is a single-channel grayscale image of the same size as the image to be detected.

[0008] Generating the coded mask image includes loading reference images, initializing a global counter to 1, and sequentially labeling each reference image with interactive elliptical markers according to the natural order of the images. For each elliptical region in the reference image, an incremental global ID is assigned sequentially according to the labeling order, with the global ID starting from 1 and incrementing continuously. The pixel value of non-station regions is 0. The current global ID is used as the pixel value to fill all pixels of the corresponding elliptical region on the canvas. The coded mask image is exported without loss. Multiple camera fields of view share a set of global ID systems, and the station numbers of different camera fields of view are not repeated.

[0009] During the deployment phase, compatibility processing is performed. If the pixel values ​​of the encoded mask image only contain 0 and 255, the encoded mask image is normalized so that the pixel values ​​are 0 or 1, thus obtaining the encoded mask image.

[0010] During the reasoning phase, the elliptical regions of each ingot are standardized to black before reasoning, and the structures outside the ingot regions are completely preserved as a stable reference background. The structures outside the ingot regions include the mechanical frame and rollers.

[0011] During the inference phase, a two-level sampling strategy is used for each anomaly detection box. First, the mask pixel value at the geometric center point coordinates of the anomaly detection box is read. If the mask pixel value is 0, the non-zero pixel value with the highest frequency within the boundary of the anomaly detection box is counted as the physical number of the workstation hit by the anomaly detection box.

[0012] Figure 9 For the center point sampling image, input the coordinates of the anomaly detection box. Calculate the center point coordinates using the encoded mask image. , Read the pixel value of the center point coordinates of the coded mask image. If the pixel value of the center point coordinates is not equal to 0, return the pixel value of the center point coordinates. If the pixel value of the center point coordinates is 0, perform a region mode vote, extract the mask sub-region within the anomaly detection box, use the unique function of the NumPy library to count the frequency of occurrence of each non-zero pixel value in the sub-region, and return the non-zero pixel value with the highest frequency.

[0013] During the verification phase, a preset cooling-off period and maximum interval are used to map the anomaly detection box to the workstation. If the mapping is successful, a global ID set of the hit workstations is obtained, and continuous frame confirmation is performed based on the global ID set of the hit workstations. If the mapping fails, the current frame is not mapped to a workstation, no global ID set of the hit workstations is obtained, the queue is cleared, and unconfirmed is returned. The inference phase and verification phase are then re-executed.

[0014] Continuous frame confirmation includes maintaining a double-ended queue for each camera, storing timestamp sets of the most recent frames and global ID sets of hit stations; if the timestamp sets in the double-ended queue are less than N frames, it returns unconfirmed, and the inference and verification phases are re-executed; if the timestamp sets in the double-ended queue are sufficient for N frames, expired records with timestamps exceeding a preset maximum interval are automatically cleaned up for each frame; the global ID sets of hit stations recorded in the most recent N frames are taken, the intersection is calculated, and only if the intersection is not empty and the cooling-off period condition is met is the wire breakage at the hit station confirmed; the double-ended queue is cleared to prevent the same event from being reported repeatedly. A program product is provided, which enables a processor to execute the aforementioned industrial multi-station visual inspection and positioning method based on pixel-level coded masks.

[0015] Compared to existing technologies, this invention has at least the following advantages: This invention masks ingot color differences, achieving cross-model generalization. By using an coded mask to standardize the elliptical regions of each ingot to black, it fundamentally eliminates background differences caused by variations in ingot color (e.g., black ingots / white ingots / metal ingots, etc.) across different models / batches. Regardless of the actual color of the ingot used on the production line, the visual features of the ingot region input into the model are always a uniform pure black, avoiding fluctuations in detection accuracy or even model failure due to changes in ingot color. The mapping from the detection box to the workstation ID requires only one pixel value read, eliminating the need for coordinate matching, database queries, or additional inference. When the workstation layout is adjusted or the ingot model is changed (ingot color change), only the ellipse needs to be redrawn in the annotation tool and a new mask exported, without retraining the deep learning model or modifying the detection code. Attached Figure Description

[0016] Figure 1 This is a technical flowchart of the present invention; Figure 2 It is a coded mask for pseudo-color display; Figure 3 This is a schematic diagram of a local pixel matrix; Figure 4 This is a flowchart of a two-level sampling strategy; Figure 5 These are comparison images of black ingots before and after processing; Figure 6 These are comparison images of white ingots before and after treatment; Figure 7 These are comparison images of metal ingots before and after processing; Figure 8 These are the detection boxes output by the model; Figure 9 It is a center point sampling image. Detailed Implementation

[0017] An industrial multi-station visual inspection and positioning method based on pixel-level coded masks. Figure 1The technical flowchart of the present invention includes: S1, annotation stage: acquire the reference image corresponding to the field of view of the industrial camera, annotate each station with an interactive ellipse on the reference image, the station being an automated detection point, automatically assign a global ID to the elliptical region of each interactive ellipse, and export the coded mask image. S2, Deployment Phase: The exported coded mask image is placed in the mask directory of the industrial inspection system. When the system starts, the MaskManager module automatically scans and loads the coded mask image; the image to be inspected is acquired in real time by the industrial camera. When the size of the coded mask image is inconsistent with the size of the image to be inspected, nearest neighbor interpolation is used for automatic alignment. After the industrial inspection system is ready, it waits for real-time frame input. S3, Inference stage, the coded mask image is used to blacken the spindle region of the image to be detected to obtain the masked image. The masked image is sent to the OpenVINO or YOLO inference engine to obtain the anomaly detection box. A two-level sampling strategy is used for localization to obtain the set of workstation physical numbers. S4, verification phase: Real-time frame input is performed, and the set of physical station numbers is sent to the continuous frame verifier. If and only if the same physical station number is hit in N consecutive frames, it is confirmed that the station that is hit in N consecutive frames has an anomaly and an alarm is output.

[0018] like Figure 2 and Figure 3 As shown, in the annotation stage, the coded mask image directly carries the physical number of the workstation using grayscale values. The coded mask image is a single-channel grayscale image of the same size as the image to be detected.

[0019] Generating the coded mask image includes loading reference images, initializing a global counter to 1, and sequentially labeling each reference image with interactive elliptical markers according to the natural order of the images. For each elliptical region in the reference image, an incremental global ID is assigned sequentially according to the labeling order, with the global ID starting from 1 and incrementing continuously. The pixel value of non-station regions is 0. The current global ID is used as the pixel value to fill all pixels of the corresponding elliptical region on the canvas. The coded mask image is exported without loss. Multiple camera fields of view share a set of global ID systems, and the station numbers of different camera fields of view are not repeated.

[0020] During the deployment phase, compatibility processing is performed. If the pixel values ​​of the encoded mask image only contain 0 and 255, the encoded mask image is normalized so that the pixel values ​​are 0 or 1, thus obtaining the encoded mask image.

[0021] During the reasoning phase, the elliptical regions of each ingot are standardized to black before reasoning, and the structures outside the ingot regions are completely preserved as a stable reference background. The structures outside the ingot regions include the mechanical frame and rollers.

[0022] During the inference phase, a two-level sampling strategy is used for each anomaly detection box. First, the mask pixel value at the geometric center point coordinates of the anomaly detection box is read. If the mask pixel value is 0, the non-zero pixel value with the highest frequency within the boundary of the anomaly detection box is counted as the physical number of the workstation hit by the anomaly detection box.

[0023] Input anomaly detection box Calculate the center point coordinates using the encoded mask image. , Read the pixel value of the center point coordinates of the coded mask image. If the pixel value of the center point coordinates is not equal to 0, return the pixel value of the center point coordinates. If the pixel value of the center point coordinates is 0, perform a region mode vote, extract the mask sub-region within the anomaly detection box, use the unique function of the NumPy library to count the frequency of occurrence of each non-zero pixel value in the sub-region, and return the non-zero pixel value with the highest frequency.

[0024] During the verification phase, a preset cooling-off period and maximum interval are used to map the anomaly detection box to the workstation. If the mapping is successful, a global ID set of the hit workstations is obtained, and continuous frame confirmation is performed based on the global ID set of the hit workstations. If the mapping fails, the current frame is not mapped to a workstation, no global ID set of the hit workstations is obtained, the queue is cleared, and unconfirmed is returned. The inference phase and verification phase are then re-executed.

[0025] Continuous frame confirmation includes maintaining a double-ended queue for each camera, storing timestamp sets of the most recent frames and global ID sets of hit stations; if the timestamp sets in the double-ended queue are less than N frames, it returns unconfirmed, and the inference and verification phases are re-executed; if the timestamp sets in the double-ended queue are sufficient for N frames, expired records with timestamps exceeding a preset maximum interval are automatically cleaned up for each frame; the global ID sets of hit stations recorded in the most recent N frames are taken, the intersection is calculated, and only if the intersection is not empty and the cooling-off period condition is met is the wire breakage at the hit station confirmed; the double-ended queue is cleared to prevent the same event from being reported repeatedly. A program product is provided, which enables a processor to execute the aforementioned industrial multi-station visual inspection and positioning method based on pixel-level coded masks.

[0026] In this invention, the mask image data structure directly encodes the physical number of the workstation using pixel values. Unlike traditional binary masks (where pixel values ​​are only 0 or 1), the mask image in this invention is a single-channel grayscale image of the same size as the image acquired by the camera. The data type is uint16 (supporting a maximum of 65535 workstations). The pixel area corresponding to each workstation is labeled with a globally unique physical number for that workstation (incrementing continuously from 1). The pixel value of non-workstation areas (background) is 0. Multiple camera fields of view share a global ID system, and the workstation numbers of different camera fields of view will not be repeated. The core effect of this data structure is that during the inference stage, for any detection box, the physical number of the workstation can be directly obtained by reading the pixel value of the corresponding coordinates of the mask image—a single O(1) array indexing operation, without the need for coordinate matching, database query, or additional model inference.

[0027] To efficiently generate the aforementioned coded mask, this invention provides an interactive ellipse annotation tool and automatic global ID assignment. A reference image is loaded, and the operator selects a representative production line image captured by an industrial camera as the annotation base map. The ellipse annotation is interactive; the operator draws ellipses on the image by pressing and dragging the mouse, with each ellipse corresponding to the spatial range of a workstation.

[0028] The tool offers the following interactive capabilities: the W key toggles the annotation mode (enabled / disabled) to prevent accidental operations during daily browsing; dragging with the left mouse button to draw an ellipse, and automatically calculating and recording the center coordinates of the ellipse when the mouse button is released. and major and minor axis radii The Z key undoes the previous annotation; the scroll wheel zooms and the middle mouse button drags to zoom, and the right mouse button drags to pan, supporting precise positioning in densely packed workstations; annotation data is persisted to a file in JSON format, with the key being the image filename (without the extension) and the value being the ellipse parameter list.

[0029] The global ID is automatically assigned sequentially and the mask is exported. The system processes the images according to the following rules: the global counter is initialized to 1; each reference image is processed sequentially according to the natural order of the image files; for the labeled ellipse in each image, the global ID is assigned sequentially in incrementing order according to the labeling order (or can be sorted by X coordinate); on the uint16 canvas, all pixels in the corresponding ellipse area are filled with the current global ID as the pixel value; the global counter is incremented by 1 after each ID is assigned.

[0030] The mask image is exported as a PNG without loss of quality. Since OpenCV's cv2.ellipse function does not support drawing filled ellipses directly on uint16 images, an indirect drawing method is adopted. This scheme ensures that the number of images is labeled, the number of workstations in each image, and that all workstations obtain globally unique and continuous physical numbers.

[0031] After the detection model outputs the anomaly detection box, the target box needs to be mapped to a specific workstation number. This invention proposes a two-level sampling strategy that balances efficiency and robustness. Center point sampling (O(1) time complexity) is based on the prior knowledge that wire breakage anomalies usually occur near the center of the workstation, with an O(1) time complexity. Region mode voting (triggered only when the first level returns 0) serves as a fallback mechanism to handle boundary conditions such as detection box offset, workstation edge anomalies, and small target detection.

[0032] Even after successfully mapping the detection frame to the workstation ID, a single frame result may still produce false alarms due to momentary interference (filament swaying, insects passing by, light flickering). This invention proposes a continuous frame confirmation based on the intersection of workstation IDs. A double-ended queue is maintained for each camera, storing records of (timestamp, set of hit workstation IDs) for the most recent frames. Expired records with timestamps exceeding a preset maximum interval are automatically cleared each frame. If no ID is hit in the current frame, the queue is cleared and "unconfirmed" is returned (a true broken wire will not disappear instantly). If the number of records in the queue is less than N frames, "unconfirmed" is returned (continuous observation). The key step is to take the set of workstation IDs recorded in the most recent N frames and calculate their intersection. A broken wire is confirmed at a workstation only when the intersection is not empty (i.e., the same workstation ID is hit in all N frames) and the cooling-off period condition is met. After confirmation, the queue is immediately cleared to prevent the same event from being reported repeatedly. The core ingenuity of this mechanism lies in the fact that it does not simply count "detected in N consecutive frames," but requires that "the same workstation ID is detected in N consecutive frames." This subtle but crucial difference fundamentally distinguishes sporadic interference (random location, random ID) from genuine wire breaks (fixed location, fixed ID).

[0033] The industrial inspection system of this invention is a single-spindle inspection system, which is built on both OpenVINO and YOLO.

[0034] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. An industrial multi-station visual inspection and positioning method based on pixel-level coded masks, characterized in that, include: S1, annotation stage: acquire the reference image corresponding to the field of view of the industrial camera, annotate each station with an interactive ellipse on the reference image, the station being an automated detection point, automatically assign a global ID to the elliptical region of each interactive ellipse, and export the coded mask image. S2, Deployment Phase: The exported coded mask image is placed in the mask directory of the industrial inspection system. When the system starts, the MaskManager module automatically scans and loads the coded mask image; the image to be inspected is acquired in real time by the industrial camera. When the size of the coded mask image is inconsistent with the size of the image to be inspected, nearest neighbor interpolation is used for automatic alignment. After the industrial inspection system is ready, it waits for real-time frame input. S3, Inference stage, the coded mask image is used to blacken the spindle region of the image to be detected to obtain the masked image. The masked image is sent to the OpenVINO or YOLO inference engine to obtain the anomaly detection box. A two-level sampling strategy is used for localization to obtain the set of workstation physical numbers. S4, verification phase: Real-time frame input is performed, and the set of physical station numbers is sent to the continuous frame verifier. If and only if the same physical station number is hit in N consecutive frames, it is confirmed that the station that is hit in N consecutive frames has an anomaly and an alarm is output.

2. The industrial multi-station visual inspection and positioning method based on pixel-level coded mask according to claim 1, characterized in that, During the annotation phase, the coded mask image directly carries the physical number of the workstation using grayscale values. The coded mask image is a single-channel grayscale image of the same size as the image to be detected.

3. The industrial multi-station visual inspection and positioning method based on pixel-level coded mask according to claim 2, characterized in that, Generating the coded mask image includes loading reference images, initializing a global counter to 1, and sequentially labeling each reference image with interactive elliptical markers according to the natural order of the images. For each elliptical region in the reference image, an incremental global ID is assigned sequentially according to the labeling order, with the global ID starting from 1 and incrementing continuously. The pixel value of non-station regions is 0. The current global ID is used as the pixel value to fill all pixels of the corresponding elliptical region on the canvas. The coded mask image is exported without loss. Multiple camera fields of view share a set of global ID systems, and the station numbers of different camera fields of view are not repeated.

4. The industrial multi-station visual inspection and positioning method based on pixel-level coded mask according to claim 1, characterized in that, During the deployment phase, compatibility processing is performed. If the pixel values ​​of the encoded mask image only contain 0 and 255, the encoded mask image is normalized so that the pixel values ​​are 0 or 1, thus obtaining the encoded mask image.

5. The industrial multi-station visual inspection and positioning method based on pixel-level coded mask according to claim 1, characterized in that, During the reasoning phase, the elliptical regions of each ingot are standardized to black before reasoning, and the structures outside the ingot regions are completely preserved as a stable reference background. The structures outside the ingot regions include the mechanical frame and rollers.

6. The industrial multi-station visual inspection and positioning method based on pixel-level coded mask according to claim 1, characterized in that, During the inference phase, a two-level sampling strategy is used for each anomaly detection box. First, the mask pixel value at the geometric center point coordinates of the anomaly detection box is read. If the mask pixel value is 0, the non-zero pixel value with the highest frequency within the boundary of the anomaly detection box is counted as the physical number of the workstation hit by the anomaly detection box.

7. The industrial multi-station visual inspection and positioning method based on pixel-level coded mask according to claim 6, characterized in that, Input the coordinates of the anomaly detection box Calculate the center point coordinates using the encoded mask image. , Read the pixel value of the center point coordinates of the coded mask image. If the pixel value of the center point coordinates is not equal to 0, return the pixel value of the center point coordinates. If the pixel value of the center point coordinates is 0, perform a region mode vote, extract the mask sub-region within the anomaly detection box, use the unique function of the NumPy library to count the frequency of each non-zero pixel value in the sub-region, and return the non-zero pixel value with the highest frequency.

8. The industrial multi-station visual inspection and positioning method based on pixel-level coded mask according to claim 1, characterized in that, During the verification phase, a preset cooling-off period and maximum interval are used to map the anomaly detection box to the workstation. If the mapping is successful, a global ID set of the hit workstations is obtained, and continuous frame confirmation is performed based on the global ID set of the hit workstations. If the mapping fails, the current frame is not mapped to a workstation, no global ID set of the hit workstations is obtained, the queue is cleared, and unconfirmed is returned. The inference phase and verification phase are then re-executed.

9. The industrial multi-station visual inspection and positioning method based on pixel-level coded mask according to claim 8, characterized in that, Continuous frame confirmation includes maintaining a double-ended queue for each camera, storing the timestamp set records of the most recent frames and the global ID set records of the hit station; if the timestamp set records in the double-ended queue are less than N frames, it returns unconfirmed and the inference and verification phases are re-executed; when the timestamp set records in the double-ended queue meet the N frames, expired records whose timestamps exceed the preset maximum interval are automatically cleaned up for each frame; the global ID set of the hit station records of the most recent N frames is taken, the intersection is calculated, and the wire breakage at the hit station is confirmed only if the intersection is not empty and the cooldown period condition is met; the double-ended queue is cleared to prevent the same event from being reported repeatedly.

10. A program product, characterized in that, The program product is used to enable the processor to execute an industrial multi-station visual inspection and positioning method based on pixel-level coded masks as described in any one of claims 1-9.

Images