A small-batch scene-oriented few-shot instance segmentation part detection method and system

Abstract

This invention discloses a method and system for few-sample instance segmentation and part detection in small-batch scenarios. The method includes: determining the detection area; acquiring a video of the object to be labeled and a reference background image from a fixed camera position, and assigning labels to the object to be labeled; discretizing the object video into multiple images, inputting them along with the reference image into a reference-based foreground segmentation model to obtain the Alpha mask of the target foreground, and generating instance labels by combining the sample labels; then forming a part training set using conventional instance segmentation data augmentation techniques; training a lightweight improved Mask R-CNN model using the part training set and a human key part training set to obtain the model weights for this batch; designing a human-machine collaborative detection process, performing frame extraction processing on the video input, and simultaneously detecting human key parts and parts from a fixed viewpoint to obtain instance information of the parts.

Description

A method and system for few-sample instance segmentation of parts detection for small-batch scenarios Technical Field

[0001] This invention relates to the field of parts inspection technology in small-batch scenarios, and in particular to a method and system for parts inspection using few-sample instance segmentation in small-batch scenarios. Background Technology

[0002] Part identification is a crucial foundation for component assembly and packaging, and an indispensable part of the product manufacturing process. In small-batch inspection scenarios, the number and variety of parts inspected in a batch are limited, the repetition rate of the same part is low, and sometimes it is necessary to flexibly adapt to the part types required for different batches. In such scenarios with high flexibility and low inspection repeatability, using automated equipment is costly, and a fully manual inspection method is generally adopted. Manual inspection is highly subjective; different inspectors may produce biased results, and the inspection process lacks traceability.

[0003] Part detection and identification methods include traditional machine vision methods, such as CN114549513A; and deep learning methods, such as patents CN114218692A and CN114972967A.

[0004] The specific patent prior art documents mentioned above are as follows:

[0005] 1) "Part Identification Method, Device, Quality Inspection Method, Electronic Equipment, and Storage Medium", Patent No. CN114549513A. This invention discloses a part identification method, device, quality inspection method, electronic equipment, and storage medium. It uses machine vision to acquire the outer contour information of the part to be tested and the outer contour information of multiple types of standard parts. According to a preset deflection interval, it acquires a first distance-to-diameter ratio set for the part to be tested and multiple second distance-to-diameter ratio sets corresponding to the multiple standard parts. The type of the part to be tested is determined based on the correlation and minimum distance between the first distance-to-diameter ratio set and the multiple second distance-to-diameter ratio sets. Sampling the first and second distance-to-diameter ratio sets through a preset deflection interval effectively reduces the amount of data to be analyzed. Furthermore, using the distance-to-diameter ratio for comparison and analysis blurs the specific dimensional parameters, effectively ensuring the accuracy of part identification without requiring scaling operations.

[0006] 2) "Similar Parts Recognition System, Medium, and Method Based on Deep Learning and Model Simulation," Patent No. CN114218692A. This invention discloses a similar parts recognition system, medium, and method based on deep learning and model simulation. It uses a 3D model of the part to generate a simulated image of the part, obtains the minimum bounding box coordinates of the part image, generates a training set for a similar parts recognition network, trains a YOLO4 neural network, constructs an optimal recognition viewpoint set based on the test set results, controls a camera to acquire on-site part images, and determines whether to adjust the pose of similar parts under the camera viewpoint using a motion turntable based on the confidence level of the recognition results, and then re-recognizes. Using a simulated image of a 3D model to generate the training set shortens the sample production time, and the use of a turntable solves the problem of decreased recognition accuracy caused by part occlusion under a single viewpoint.

[0007] 3) "A Method and System for Identifying and Counting Aircraft Parts," Patent No. CN114972967A. This invention discloses a method and system for identifying and counting aircraft parts. It uses machine vision to acquire images of parts on a conveyor belt, uses LabelImg for annotation, and employs a DarkNet-53 multi-branch deep convolutional network for training and detection to obtain a density map. The number of parts is estimated by integrating the density map. By directly acquiring tiled images of aircraft parts using a camera, part identification and counting can be achieved, avoiding errors caused by manual identification and counting or traditional identification and counting methods. Summary of the Invention

[0008] To address the aforementioned technical problems, the purpose of this invention is to provide a method and system for detecting parts by segmentation of few samples in small-batch scenarios.

[0009] The objective of this invention is achieved through the following technical solution:

[0010] A method and system for few-sample instance segmentation and part detection in small-batch scenarios, comprising:

[0011] A. Determine the detection area, fix the camera position to acquire the video of the object to be labeled and the reference background image, and assign labels to the object to be labeled;

[0012] B discretizes the object video into multiple images, and inputs them along with a reference image into a reference-based foreground segmentation model to obtain the alpha mask of the target foreground. Instance annotations are then generated by combining these with sample labels, and conventional instance segmentation data augmentation techniques are used to form a training set for the parts.

[0013] C uses a parts training set Training set D of key human body parts human Train a lightweight Mask R-CNN instance segmentation model to obtain the model weights W for this batch.i ;

[0014] D designs a human-machine collaborative detection process to perform frame extraction on video input, while simultaneously detecting key human body parts and components from a fixed perspective to obtain instance information of the components.

[0015] A few-sample instance segmentation part inspection system for small-batch scenarios includes an inspection device and a host computer;

[0016] The inspection device includes a camera, a support system, an inspection platform, a rotating device, and an auxiliary lighting system, used to acquire video and background information of parts from a fixed perspective and to perform part inspection applications in this fixed scene;

[0017] The host computer is used to separate the target foreground, enhance small sample instance segmentation data, and train and infer subsequent instance segmentation models.

[0018] Compared with the prior art, one or more embodiments of the present invention may have the following advantages:

[0019] By employing a human-machine collaborative approach, detection tasks are assigned to machines, while target movement tasks are assigned to humans, achieving high-precision and low-cost detection in small-batch detection. Defining the actual detection range determines the data collection scope, improving sample validity and detection performance. Foreground segmentation algorithms extract alpha masks from samples in videos to generate original instance datasets, significantly reducing the workload of instance sample data annotation. A lightweight Mask R-CNN model, combined with the human-machine collaborative detection process, reduces data processing volume and increases detection speed. Attached Figure Description

[0020] Figure 1 is a flowchart of a few-sample instance segmentation part detection method for small-batch scenarios;

[0021] Figure 2 shows the detailed process of human-machine collaborative testing;

[0022] Figure 3 is a schematic diagram of the detection device. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in further detail below with reference to the embodiments and accompanying drawings.

[0024] Figure 1 shows a few-sample instance segmentation part detection method for small-batch scenarios, including the following steps:

[0025] Step 10 uses a human-machine collaborative approach to determine the detection area, fixes the camera position to acquire the video of the object to be labeled and the reference background image, and assigns labels to the object to be labeled;

[0026] Step 20 discretizes the video into multiple images, inputs them along with the reference image into a reference-based foreground segmentation model, obtains the Alpha mask of the target foreground, generates instance annotations by combining sample labels, and then uses conventional instance segmentation data augmentation techniques to form a parts training set.

[0027] Step 30 uses the parts training set and the human body key parts training set to train a lightweight improved Mask R-CNN model and obtains the model weights for this batch.

[0028] Step 40: Design a human-machine collaborative detection process, perform frame extraction on the video input, and simultaneously detect key parts and components of the human body from a fixed perspective.

[0029] Example: Inspection of a single batch of parts

[0030] The detection area described in step 10 above is obtained through human-machine collaboration. The detection area Ω is determined by statistically analyzing the part placement habits of different people on the detection platform. det Multiply by the redundancy factor c to obtain the data acquisition area Ω sample According to Ω sample Place the rotating device and the object to be labeled, and capture video. Simultaneously capture the corresponding background image and assign a text label of "video-background target type".

[0031] Step 20 above includes extracting video frames of the target at a certain ratio, inputting the video frames and reference background into the improved BGM v2 model to obtain a refined target alpha mask, obtaining an object mask based on the alpha mask, generating corresponding instance segmentation annotation data, forming the original annotation data for this batch of parts, and then combining it with a multi-layer conventional instance segmentation data augmentation method to generate the final training data for this batch.

[0032] The improved BGM v2 used is batch-independent and utilizes the public dataset D for foreground segmentation in part inspection tasks. public Once trained, it is plug-and-play and requires no retraining to adapt to the testing of different batches of parts.

[0033] The improvement of BGM v2 lies in changing the selection algorithm for the region to be optimized during the optimization phase from a single error map pixel-based selection to a local region-based algorithm. First, the w×h error map output by the BGM v2 base network is interpolated to size w'×h':

[0034]

[0035] Then the error map is divided into M×N 5×5 blocks, resulting in a set P = {p 1,1 ,...,p 1,N ,...,pM,1 ,...,p M,N} Calculate the Hadamard product of each block with a 5×5 Gaussian filter kernel:

[0036]

[0037] Use the top-k algorithm to extract the k largest values ​​Y = topk(X), and then divide block p. m,n The seed point s is obtained by restoring the center to the original coordinate system. i =(u m ,v n ),in:

[0038]

[0039] Use the seed point as a reference point to generate k 8×8 regions to be optimized.

[0040] Step 30 above includes using the training dataset of this batch of parts. With human body part dataset D human Train a lightweight Mask R-CNN instance segmentation model to obtain the batch weights W. i The D used human This is a batch-independent dataset;

[0041] As shown in Figure 2, the human-machine collaborative testing process described above includes:

[0042] ① The operator moves the part into the inspection platform, and the operator's body part is detected;

[0043] ②The operator puts down the part, moves their hand away, the part is detected, but the operator's part is not detected, and the part information is recorded.

[0044] ②The operator moves the part away from the inspection platform, the operator's position is detected, and the recording of part information is stopped.

[0045] As shown in Figure 3, this embodiment also provides a few-sample instance segmentation part detection system for small-batch scenarios, including a detection device and a host computer; wherein:

[0046] The inspection device includes: a support system 1, a camera 2, an auxiliary lighting system 3, an inspection platform 4, and a rotation device, which are used to acquire video and background information of the parts from a fixed perspective and to perform part inspection applications in this fixed scene;

[0047] The host computer is used to separate the target foreground, enhance small sample instance segmentation data, and train and infer subsequent instance segmentation models.

[0048] While the embodiments disclosed in this invention are as described above, the content is merely for the purpose of facilitating understanding of the invention and is not intended to limit the invention. Any person skilled in the art to which this invention pertains may make any modifications and variations in form and detail of the implementation without departing from the spirit and scope disclosed herein; however, the scope of patent protection for this invention shall still be determined by the scope defined in the appended claims.

Claims

1. A method for detecting parts by segmenting few samples in small-batch scenarios, characterized in that, include: A. Determine the detection area, fix the camera position to acquire the video of the object to be labeled and the reference background image, and assign labels to the object to be labeled; B discretizes the object video into multiple images, and inputs them along with a reference image into a reference-based foreground segmentation model to obtain the alpha mask of the target foreground. Instance annotations are then generated by combining these with sample labels, and conventional instance segmentation data augmentation techniques are used to form a training set for the parts. ； C uses a parts training set Training set for key parts of the human body Train a lightweight Mask R-CNN instance segmentation model to obtain the model weights for this batch. D involves designing a human-machine collaborative detection process to perform frame extraction on the video input, simultaneously detecting key human body parts and components from a fixed viewpoint to obtain instance information of the parts; B includes extracting a certain proportion of object video frames, inputting the video frames and reference background into an improved BGM v2 model to obtain a refined target alpha mask, and generating an object mask based on the alpha mask to generate corresponding instance segmentation annotation data, forming the original annotation data for this batch of parts, and then combining it with a multi-layer conventional instance segmentation data augmentation method to generate the final training data for this batch. The improvement of BGM v2 lies in changing the selection algorithm for the region to be optimized during the optimization phase from one pixel-based error map to a local region-based algorithm. Specifically, this includes: first, processing the output of the BGM v2 base network... Error plot interpolation to size ： (1) Then the error map is divided into M×N 5×5 blocks to obtain the set. Calculate the Hadamard product of each block with a 5×5 Gaussian filter kernel: (2) Use the topk algorithm to extract the k largest values. , block The seed point is obtained by restoring the center to the original coordinate system. ,in: (3) Generate k 8×8 regions to be optimized using the seed point as a reference point; The human-machine collaborative detection process includes: the operator moves the part into the detection platform and the operator's body part is detected; the operator puts down the part, moves his hand away, the part is detected, the operator's body part is not detected, and the part information is recorded; the operator moves the part away from the detection platform, the operator's body part is detected, and the recording of part information stops.

2. The method for detecting parts by segmenting few samples in small-batch scenarios as described in claim 1, characterized in that, In section A, the detection area is obtained through human-machine collaboration, and is determined by statistically analyzing the part placement habits of different people on the detection platform. Multiply by the redundancy factor c to obtain the data acquisition area. ,according to Place the rotating device and the object to be labeled, and capture video; simultaneously capture the corresponding background image and assign a text label of type "video-background target".

3. The method for detecting parts by segmenting few samples in small-batch scenarios as described in claim 1, characterized in that, The method used in C It is a batch-independent dataset, which can be built once and applied to the detection of any batch of parts; the Mask R-CNN model used has been modified by model pruning.

4. A few-sample instance segmentation part detection system for small-batch scenarios, used to execute the method according to any one of claims 1-3, characterized in that, The system includes a detection device and a host computer; the detection device includes a camera, a support system, a detection platform, a rotation device and an auxiliary lighting system, used to collect part video-background information from a fixed perspective and to perform part detection applications in this fixed scene; the host computer is used to separate the target foreground, realize the enhancement of small sample instance segmentation data, and the training and inference of subsequent instance segmentation models.

Images