Part identification method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a part identification method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: acquiring a first image comprising a to-be-identified part; inputting the first image into a first neural network to obtain the position of the to-be-identified part in the first image; segmenting the image according to the position of the to-be-identified part in the first image to obtain a second image including the to-be-identified part; inputting the second image into a second neural network to obtain a preliminarily identified part number; according to the preliminarily identified part number, judging whether the to-be-identified part has a similar part, and if the to-be-identified part has the similar part, inputting the second image into a third neural network to obtain a finally identified part number; and if the similar part does not exist, taking the preliminarily identified part number as the finally identified part number. Therefore, by adopting a multi-neural network collaborative identification strategy, the identification capability of small targets is enhanced, and the identification accuracy of similar parts is also remarkably improved.

Description

Part Recognition Method, Device, Computer Equipment, Storage Medium and Program Product Technical Field

[0001] This application relates to the technical field of deep network applications, and particularly to a part recognition method, device, computer equipment, storage medium and program product. Background Art

[0002] For the supply of auto parts, picking and bundling need to be carried out in the warehouse before shipment. When picking and bundling, it is necessary to confirm the specifications and models of the parts to prevent the part models shipped from being inconsistent with the shipment requirements, resulting in economic and goodwill losses.

[0003] A car usually contains tens of thousands of parts with different shapes and sizes. Many auto parts are also divided into left parts and right parts according to the orientation on the car. The left and right parts are highly similar, but they cannot be mixed, which poses challenges in manual recognition. In the past, when picking and bundling, manual visual inspection was used to check the specifications and models of the parts, resulting in low recognition efficiency and easy errors. When problems are found after the goods are delivered, manual recognition cannot be used to trace and improve the error-prone links. Summary of the Invention

[0004] Based on this, in view of the technical problems of low recognition efficiency and easy errors in manual visual inspection of part specifications and models in the prior art, it is necessary to provide a part recognition method, device, computer equipment, storage medium and program product.

[0005] To solve the above technical problems, the technical solution of the present invention is as follows: The first embodiment of the present invention provides a part recognition method, including: obtaining a first image including the part to be recognized; inputting the first image into a first neural network to obtain the position of the part to be recognized in the first image; segmenting the image according to the position of the part to be recognized in the first image to obtain a second image including the part to be recognized; inputting the second image into a second neural network to obtain a preliminarily recognized part number; according to the preliminarily recognized part number, determining whether there are similar parts for the part to be recognized. If there are similar parts, inputting the second image into a third neural network to obtain a finally recognized part number; if there are no similar parts, using the preliminarily recognized part number as the finally recognized part number.

[0006] Further, the training process of the first neural network is: collecting images of various parts at different photographing angles and annotating the specific positions of the parts in the collected images to obtain a first data set; using the first data set to train the first neural network to obtain a trained first neural network.

[0007] Furthermore, the image is segmented to obtain a second image including the part to be identified, including: determining the region where the part to be identified is located based on the position of the part to be identified in the first image, segmenting the region where the part to be identified is located from the first image to obtain the second image.

[0008] Furthermore, the training process of the second neural network is as follows: images of various parts taken from different angles are collected, and the part categories are labeled in the collected images to obtain a second dataset; the second neural network is trained using the second dataset to obtain a trained second neural network.

[0009] Furthermore, the training process of the third neural network is as follows: for similar parts of each part type, multiple images are taken from different angles to obtain a third dataset; the third neural network is trained using the third dataset to obtain a trained third neural network.

[0010] Furthermore, based on the preliminarily identified part number, determining whether there are similar parts for the part to be identified includes: querying a preset database based on the preliminarily identified part number, and determining whether there are similar parts for the part to be identified based on the query results.

[0011] A second embodiment of the present invention provides a part identification device, the device comprising: an acquisition module, the acquisition module acquiring a first image including a part to be identified; a first identification module, the first identification module inputting the first image into a first neural network to obtain the position of the part to be identified in the first image; a segmentation module, the segmentation module segmenting the image according to the position of the part to be identified in the first image to obtain a second image including the part to be identified; a second identification module, the second identification module inputting the second image into a second neural network to obtain a preliminarily identified part number; and a third identification module, the third identification module determining whether there are similar parts to the part to be identified based on the preliminarily identified part number; if there are similar parts, inputting the second image into the third neural network to obtain a final identified part number; if there are no similar parts, using the preliminarily identified part number as the final identified part number.

[0012] A third embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method as described in the first embodiment of the present invention.

[0013] A fourth embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first embodiment of the present invention.

[0014] A fifth embodiment of the present invention provides a computer program product including computer instructions that, when executed by a processor, implement the steps of the method described in the first embodiment of the present invention.

[0015] Compared to existing technologies, the beneficial effects of this invention are as follows: A part identification method of this invention first acquires a first image including the part to be identified; then, the first image is input into a first neural network to obtain the position of the part to be identified in the first image; based on the position of the part to be identified in the first image, the image is segmented to obtain a second image including the part to be identified; the second image is input into a second neural network to obtain a preliminary identified part number; finally, based on the preliminary identified part number, it is determined whether there are similar parts to the part to be identified. If similar parts exist, the second image is input into a third neural network to obtain the final identified part number; if no similar parts exist, the preliminary identified part number is used as the final identified part number. Thus, this invention employs a multi-neural network collaborative identification strategy. One network focuses on extracting the global morphological features of the part to determine the main specifications and models; another network focuses on capturing local fine structures, such as edges, holes, threads, and other discriminative details, specifically for distinguishing highly similar part variants. This hierarchical feature extraction and decision-making mechanism enhances the ability to identify small targets and significantly improves the accuracy of distinguishing between similar parts. Attached Figure Description

[0016] Figure 1 is a flowchart illustrating a part identification method according to an embodiment of the present invention; Figure 2 is a schematic diagram illustrating an imaging device according to an embodiment of the present invention; Figure 3 is a flowchart illustrating a part body detection process according to an embodiment of the present invention; Figure 4 is a flowchart illustrating a similar part identification process according to an embodiment of the present invention; Figure 5 is a structural schematic diagram illustrating a part identification device according to an embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0018] Traditional visual inspection and manual measurement methods are inefficient and subjective, easily affected by worker fatigue and lack of concentration, leading to missed or false detections. Furthermore, manual visual inspection is prone to confusion when identifying highly similar left-right or similar parts, failing to meet the demands of high-precision inspection. Traditional image processing-based solutions do not consider the cost and challenges of image acquisition in part recognition, nor do they address the differentiation of similar parts, using a single network for part identification. For image acquisition in part recognition, considering cost and efficiency, a single camera with fixed position and angle can be used to construct the acquisition device. This same acquisition device needs to capture parts of different sizes, requiring the ability to scale large parts to fill the entire camera's field of view while maintaining clarity for small parts that occupy only a few pixels. Existing technologies either employ expensive autofocus scaling acquisition equipment or fail to differentiate between similar parts when identifying small components.

[0019] A first embodiment of the present invention provides a part identification method, as shown in FIG1, comprising: acquiring a first image including a part to be identified; inputting the first image into a first neural network to obtain the position of the part to be identified in the first image; segmenting the image according to the position of the part to be identified in the first image to obtain a second image including the part to be identified; inputting the second image into a second neural network to obtain a preliminarily identified part number; determining whether there is a similar part to the part to be identified based on the preliminarily identified part number; if there is a similar part, inputting the second image into a third neural network to obtain a final identified part number; if there is no similar part, using the preliminarily identified part number as the final identified part number.

[0020] In this embodiment, a dedicated part detection network (first neural network) is introduced before specification recognition. This network first locates the specific position and size of the part in the image, then cuts out an independent image of each part from the original image, and inputs it into the subsequent classification network for training and recognition. This design effectively isolates background interference, highlights the distinguishing features of individual parts, thereby better preserving fine-grained information and comprehensively improving the accuracy and reliability of part specification and model recognition. Meanwhile, some automotive parts may have similar or left / right counterparts. After initially identifying parts that may have similar features, this embodiment activates a deep classification network (third neural network) specifically optimized for similar parts for secondary precise recognition. The core of this process is to solve the problem of distinguishing subtle differences between parts that are similar in appearance but different in category or specification. This secondary recognition mechanism effectively avoids misjudgment by traditional single recognition models when dealing with "similar-looking parts," significantly improving the accuracy and reliability of sorting, quality inspection, or automated assembly processes.

[0021] In a further embodiment, the training process of the first neural network is as follows: images of various parts taken from different camera angles are collected, and the specific positions of the parts are marked in the collected images to obtain a first dataset; the first neural network is trained using the first dataset to obtain a trained first neural network.

[0022] Specifically, in step 1, an industrial camera is used to take pictures of various parts from different angles. The shooting equipment used is shown in Figure 2. The shooting equipment uses a green tabletop to create a contrast with the color of the parts for easy identification. An adjustable clamp is used to hold the stack of sheet-like parts with their sides facing up for easy photo counting. A retractable monitor stand is used to flexibly adjust the position and avoid obstructing the operation. A brown cover is used to prevent the camera light from shining directly into the eyes of the operators. In step 2, the collected images are labeled in the LabelStudio tool, outlining the specific location of the parts (i.e., BBOX boxes) and labeling the part category. In step 3, the dataset is divided into two parts: training and validation. The number of samples can be divided into 90% training and 10% validation. The image text of each sample is... Each label corresponds to a file with the same filename and the .txt extension. The content of the label file is organized in the format of class (category label, currently labeled as 0), x, y (x and y represent the coordinates of the object's center point in the image), width, and height (width and height represent the object's width and height); Step 4, create the dataset training configuration file dataset.yaml, with path pointing to the directory where the datasets train and validate are located; Step 5, write the training code file train.py; Step 6, configure the Python runtime environment using conda, this example uses Python 3.11; Step 7, install PyTorch, this example uses the GPU version configuration.

[0023] Step 8: Install the network model package; Step 9: Execute the train.py code and observe the changes in the output recall and mAP. When recall and mAP reach 0.99 or above, the business requirements of this invention are met, and training can be terminated; Step 10: If an abnormality of insufficient video memory occurs during training, the batch_size parameter can be adjusted downwards appropriately; Step 11: After training is completed, use the network weight file obtained from training to predict all the collected images.

[0024] In a further embodiment, the first neural network may be YOLOv8.

[0025] In a further embodiment, segmenting the image to obtain a second image including the part to be identified includes: determining the region where the part to be identified is located based on the position of the part to be identified in the first image, segmenting the region where the part to be identified is located from the first image, and obtaining the second image.

[0026] In a specific embodiment, the predicted image results from the trained first neural network use `results.xywhn` as the coordinates of the detected main part. These coordinates are used to cut the original image, resulting in a high-resolution image containing only a single part. `xywhn` represents the coordinates and size proportionally. When cutting the original image, the coordinates and size of the original image need to be calculated. The calculation formula is as follows:

[0027] In the formula, X origin Y origin Let W be the coordinates of the original image. origin H orign The width and height of the original image, X n Y n To normalize the coordinates of the network prediction results, W n H n Normalize the width and height of the network prediction results.

[0028] In this embodiment, the usual object detection and recognition process involves annotating objects on the captured image, training a model, and then using the model for recognition. In this embodiment, the recognition process involves identifying the object's location, cropping sub-images of the object from the original image, and then using the sub-images to identify the object category. This effectively alleviates the problem of detail loss due to insufficient pixels without significantly increasing hardware costs.

[0029] This embodiment also provides a main body detection process for a part, including: S1: Scanning the label of the automotive part to be identified and obtaining the part number from the label; S2: Calling the AI ​​recognition interface to trigger the camera to capture an image; S3: Placing a single part on the workbench for easy AI image capture and recognition; S4: After receiving the start recognition signal, the AI ​​recognition interface calls the camera hardware shown in Figure 2 to take a picture and obtain the image to be identified; S5: The AI ​​service receives the image data sent by the camera; S6: The main body detection process begins, as shown in Figure 3; S7: Loading the image using OpenCV and converting the input image into a 3D NumPy matrix; S8: Calling the constructor of the Python class YOLO to load the main body recognition AI model Mainpart_detect; S9: Calling the predict method of the Mainpart_detect model to convert the NumPy data from the above steps into a 3D matrix. S10: In the YOLO backbone network, the NumPy matrix is ​​processed by image convolution CNN for feature extraction, feature concatenation, and feature BN normalization. S11: The BN-processed data is processed by SPPF spatial feature pyramid pooling to preserve the overall and detailed features of the part. S12: Based on the extracted features, the part image is classified and regressed by region to obtain the specific positional ratio (xywhn) of the part in the image. S13: xywhn uses a proportional representation of coordinates and size. When cutting the original image, the coordinates and size of the original image need to be calculated. S14: Based on the position of the part, the part image is cut from the original image to obtain the NumPy 3D matrix of the part image. S15: Part detection is complete.

[0030] In a further embodiment, the training process of the second neural network is as follows: images of various parts taken from different angles are collected, and the part categories are labeled in the collected images to obtain a second dataset; the second neural network is trained using the second dataset to obtain a trained second neural network.

[0031] Specifically, in step 12, a new dataset is created using the cut images, and a parts classification and recognition model (second neural network) is trained; in step 13, the dataset.yaml file in step 4 is modified so that the dataset path points to the path of the image dataset in step 12; in step 14, the train.py script is executed for training; in step 15, after training is completed, there are two network models: part body detection and part model recognition.

[0032] In a further embodiment, the second neural network may be YOLOv8.

[0033] This embodiment also provides a preliminary part number identification process for a part, including: S16: Loading the part classification AI model Part_classify (second neural network); S17: Calling the predict method of the Part_classify model and inputting the matrix data of the main body of the part; S18: After feature extraction through the YOLO backbone network, performing Sigmoid calculation for classification prediction to obtain the corresponding part number.

[0034] In a further embodiment, the training process of the third neural network is as follows: for similar parts of each part type, multiple images are taken from different angles to obtain a third dataset; the third neural network is trained using the third dataset to obtain a trained third neural network.

[0035] Specifically, in step 16, the part samples are validated using two network models according to the method of part body detection -> original image cutting -> part model recognition; in step 17, the validation results may show some errors in the recognition of similar parts, so the similar part recognition model is trained; in step 18, for each type of similar part, images are retaken from different angles, with 100 images taken for each similar part; in step 19, the images of similar parts are stored in a new path according to the directory structure of the training set of the part classification and recognition model; in step 20, the dataset.yaml in step 13 is modified so that the dataset path points to the path of the image dataset in step 19; in step 21, the train.py script is executed for training; in step 22, after training is completed, the similar part recognition model (third neural network) is obtained.

[0036] In a further embodiment, the third neural network may be YOLOv8.

[0037] This embodiment also provides a similar part identification process, including: S19: Determine whether there are similar parts for the part based on the identified part number. Specifically, based on the initially identified part number, a query is performed in a preset database, and the query result is used to determine whether there are similar parts for the part to be identified. If so, the AI ​​model Similar_classify for similar part identification is loaded to further confirm the part number; S20: The similar part identification process begins, as shown in Figure 4; S21: The identified part number is compared with the tag part number to determine whether the identification result is consistent; S22: The identification result is displayed on the workbench screen; S23: If the identification result is abnormal, the part is rearranged, the part's posture and angle are adjusted, and the image is re-acquired for identification; S24: After correct identification, the derivation confirmation button is pressed to complete the identification.

[0038] This invention presents a parts image acquisition and recognition system based on a fixed single camera, aiming to achieve high-precision automated recognition of parts of various specifications with low hardware costs. The core feature of this system is the use of a single camera with fixed position and angle for image acquisition, eliminating the need for expensive motorized zoom or autofocus modules, thus significantly reducing equipment investment and system complexity.

[0039] In the image acquisition stage, the system needs to address the imaging challenges of parts of different sizes: large parts need to fill the entire field of view to utilize the full image resolution, while small parts need to maintain clearly discernible details within a limited number of pixels. To address this, this solution optimizes both optical configuration and software processing. Optically, by selecting an appropriate focal length lens and combining it with depth-of-field control, clear imaging of parts of different sizes is ensured at a fixed object distance. Software-wise, adaptive image segmentation and resolution enhancement technologies are introduced. Before recognizing small parts, local region extraction and image super-resolution reconstruction are performed, effectively mitigating the problem of detail loss due to insufficient pixels without significantly increasing hardware costs.

[0040] In terms of recognition architecture, this invention abandons the traditional single-network model and adopts a multi-neural network collaborative recognition strategy. The system first uses an object detection network to locate the part region in the image. Then, the cropped part image is input into two different classification sub-networks for processing: one network focuses on extracting the global morphological features of the part to determine its main specifications and model; the other network focuses on capturing local fine structures, such as edges, holes, and threads, specifically for distinguishing highly similar part variants. This hierarchical feature extraction and decision-making mechanism enhances the ability to recognize small targets and significantly improves the accuracy of distinguishing similar parts.

[0041] In summary, this invention, through the organic combination of a fixed camera acquisition scheme and a multi-network collaborative recognition framework, effectively solves the key challenges in multi-scale part imaging and high-precision recognition while controlling hardware costs. It is particularly suitable for industrial inspection scenarios involving a large number of similar parts and significant size differences, such as automotive parts and electronic components, and has good engineering application and promotion value.

[0042] A second embodiment of the present invention provides a part identification device, as shown in FIG5. The device includes: an acquisition module, which acquires a first image including a part to be identified; a first identification module, which inputs the first image into a first neural network to obtain the position of the part to be identified in the first image; a segmentation module, which segments the image according to the position of the part to be identified in the first image to obtain a second image including the part to be identified; a second identification module, which inputs the second image into a second neural network to obtain a preliminarily identified part number; and a third identification module, which determines whether there are similar parts to the part to be identified based on the preliminarily identified part number. If there are similar parts, the third identification module inputs the second image into the third neural network to obtain a final identified part number; if there are no similar parts, the preliminarily identified part number is used as the final identified part number.

[0043] A third embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in the first embodiment of the present invention.

[0044] A fourth embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first embodiment of the present invention.

[0045] A fifth embodiment of the present invention provides a computer program product including computer instructions that, when executed by a processor, implement the steps of the method described in the first embodiment of the present invention.

[0046] The processor mentioned above is the processor in the computer device described in the above embodiments. The computer-readable storage medium may be a computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0047] In this embodiment, the module can be implemented in software so that it can be executed by various types of processors. For example, an identified executable code module may include one or more physical or logical blocks of computer instructions, which may be constructed as objects, procedures, or functions. Nevertheless, the executable code of the identified module does not need to be physically located together, but may include different instructions stored in different bits, which, when logically combined, constitute the module and achieve the module's intended purpose.

[0048] Through the above description of the implementation methods, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, disk, optical disk) and includes several instructions for executing the methods described in the various embodiments of this application.

[0049] The exemplary embodiments described above are with reference to the accompanying drawings. Many different forms and embodiments are feasible without departing from the spirit and teachings of this application. Therefore, this application should not be construed as limiting the exemplary embodiments set forth herein. Rather, these exemplary embodiments are provided to make this application complete and convey the scope of this application to those skilled in the art. In these drawings, component dimensions and relative dimensions may be exaggerated for clarity. The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. As used herein, unless clearly indicated otherwise, the singular forms “a,” “an,” and “the” are intended to include all such forms. It will be further understood that the terms “comprising” and / or “including”, when used in this specification, indicate the presence of the stated features, integers, steps, operations, components, and / or elements, but do not exclude the presence or addition of one or more other features, integers, steps, operations, components, and / or groups thereof. Unless otherwise indicated, when stated, a range of values ​​includes the upper and lower limits of the range and any subranges in between.

[0050] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles described in this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for identifying parts, characterized in that, include: Acquire a first image including the part to be identified; The first image is input into the first neural network to obtain the position of the part to be identified in the first image; the image is segmented according to the position of the part to be identified in the first image to obtain a second image including the part to be identified; the second image is input into the second neural network to obtain the preliminary identified part number; Based on the preliminarily identified part number, it is determined whether there are similar parts to the part to be identified. If there are similar parts, the second image is input into the third neural network to obtain the final identified part number. If no similar parts exist, the part number initially identified will be used as the final identified part number.

2. The part identification method according to claim 1, characterized in that, The training process of the first neural network is as follows: images of various parts taken from different angles are collected, and the specific positions of the parts are marked in the collected images to obtain the first dataset; the first neural network is trained using the first dataset to obtain the trained first neural network.

3. The part identification method according to claim 1, characterized in that, The process of segmenting the image to obtain a second image including the part to be identified includes: determining the region where the part to be identified is located based on the position of the part to be identified in the first image, segmenting the region where the part to be identified is located from the first image, and obtaining the second image.

4. The part identification method according to claim 1, characterized in that, The training process of the second neural network is as follows: images of various parts taken from different angles are collected, and the part categories are labeled in the collected images to obtain the second dataset; the second neural network is trained using the second dataset to obtain the trained second neural network.

5. The part identification method according to claim 1, characterized in that, The training process of the third neural network is as follows: for similar parts of each part type, multiple images are taken from different angles to obtain a third dataset; the third neural network is trained using the third dataset to obtain a trained third neural network.

6. The part identification method according to any one of claims 1 to 5, characterized in that, Determining whether there are similar parts to the part to be identified based on the preliminarily identified part number includes: querying a preset database based on the preliminarily identified part number, and determining whether there are similar parts to the part to be identified based on the query results.

7. A parts identification device, characterized in that, The device includes: an acquisition module that acquires a first image including a part to be identified; a first identification module that inputs the first image into a first neural network to obtain the position of the part to be identified in the first image; a segmentation module that segments the image according to the position of the part to be identified in the first image to obtain a second image including the part to be identified; a second identification module that inputs the second image into a second neural network to obtain a preliminarily identified part number; and a third identification module that determines whether there are similar parts to the part to be identified based on the preliminarily identified part number. If there are similar parts, the second image is input into the third neural network to obtain a final identified part number; if there are no similar parts, the preliminarily identified part number is used as the final identified part number.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes computer instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 6.

Images