Wire number identification method based on single-stage neural network algorithm YOLOv5

Abstract

The invention belongs to the technical field of aviation intelligent manufacturing, and discloses a lead number identification method based on a single-stage neural network algorithm YOLOv5, and the method comprises the steps: collecting a lead image through employing a fixed-focus macro camera as an image collection device, marking a number region of the collected lead image to form a YOLOv5 network data set, and carrying out the recognition of the lead number of the lead image through the YOLOv5 network data set; using the data set to train a model capable of identifying and cutting a numbered region by using a YOLOv5 neural network, then using the trained numbered region identification model to cut the numbered region of the collected lead image, and then performing image feature enhancement operation on the cut numbered region image. And then marking tags character by character on the numbered images after feature enhancement to form a YOLOv5 network data set, and inputting the marked numbered data set into a YOLOv5 neural network to train a numbered recognition model. The problem that an existing number identification method is unstable is solved, rapid identification and assembly position matching of the assembly wire number are achieved, and the purpose of improving the assembly efficiency and accuracy is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of wire number recognition in aviation intelligent manufacturing, and specifically relates to a wire number recognition method based on a single-stage neural network algorithm YOLOv5. Background Art

[0002] In a complex assembly system, assemblers sometimes need to match and connect hundreds of wires with different functions and shapes. In order to improve the accuracy of assembly, specific numbers are often marked on the wires to distinguish wires in different positions. During assembly, workers can match the numbers on the wires with assembly standard files or corresponding databases to confirm the assembly position of the wires or conduct inspections after assembly. However, these wires used for signal transmission are often thin, with the thinnest diameter being only two millimeters, and the numbers printed on the wires are small. When assemblers use visual confirmation of the wire numbers, it often takes a long time and is prone to errors. The traditional OCR (Optical Character Recognition) methods based on computer vision, such as feature-based target detection and template matching, require manual design of detection features and rules, which is prone to large design workload and unstable detection results when facing complex and changeable actual situations. In addition, traditional OCR methods have poor generalization performance and are sensitive to character deformation, rotation, font changes, etc., especially for flexible objects with many styles such as assembly wires. Additional processing is required to deal with these situations, which makes them difficult to implement in actual assembly scenarios. Deep learning can automatically extract and recognize this information through self-learning and training. With the training of a large amount of data, its model has a higher ability to process and judge complex scenes, and can achieve higher accuracy and stability.

[0003] In summary, it is necessary to propose a wire number recognition method based on the single-stage neural network algorithm YOLOv5 to help assemblers quickly identify and match wire numbers.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the invention and therefore it may contain information that does not form the prior art that is known to a person of ordinary skill in the art. Summary of the invention

[0005] The present invention application proposes a wire number recognition method based on the single-stage neural network algorithm YOLOv5, which realizes the rapid recognition of assembly wire numbers and assembly position matching, thereby achieving the purpose of improving assembly efficiency and accuracy.

[0006] In order to achieve the above invention objectives, the technical solutions adopted in this application are as follows:

[0007] A wire number recognition method based on a single-stage neural network algorithm YOLOv5 comprises the following steps:

[0008] S1, using an image acquisition device to acquire images of numbered wires containing various types, various fonts, and various characters, and constructing a wire image dataset;

[0009] S2, labeling and segmenting the numbered areas of the wire images in the wire image dataset to construct training set data and test set data;

[0010] S3, constructing a YOLOv5 neural network, inputting the wire image dataset with numbered area annotation to train and optimize the numbered area recognition model;

[0011] S4, using the trained YOLOv5 wire number area recognition model to perform number area recognition and cropping on the wire image dataset, and saving the cropped wire number area image;

[0012] S5, performing image feature enhancement on the saved wire number area image and saving it, and constructing a number area image data set;

[0013] S6, annotating and segmenting the wire number area image after feature enhancement to construct training set data and test set data;

[0014] S7, constructing a YOLOv5 neural network, inputting the labeled number area character data set to train and optimize the wire number character recognition model;

[0015] S8, using the trained YOLOv5 wire number area recognition model to crop the wire image for the number area, then enhancing the image features of the cropped number area image, and inputting it into the YOLOv5 wire number character recognition model to perform character recognition and output the number information.

[0016] As a further solution of the present invention: in step S1, a fixed-focus macro camera is used as an image acquisition device to acquire images.

[0017] As a further solution of the present invention: in the step S2, the numbered areas in the collected wire image are framed and marked, and the marked image number area position information file is saved as a data set for the YOLOv5 number area recognition model.

[0018] As a further solution of the present invention: in step S5, the processing method specifically includes the following steps:

[0019] S51, converting the cropped wire number area image from a color image to a grayscale image;

[0020] S52, performing Gaussian filtering to reduce noise on the converted grayscale image;

[0021] S53, performing adaptive threshold segmentation on the denoised image to form a binary image.

[0022] As a further solution of the present invention: in step S6, the wire number area image after feature enhancement is annotated character by character, and the data ratio is adjusted to ensure a balanced number of annotated characters, thereby forming a number area character data set.

[0023] As a further solution of the present invention: in step S8, the wire image acquired by the image acquisition device is directly input into the trained numbered area recognition model for reasoning to obtain a cropped image of the numbered area.

[0024] As a further solution of the present invention: in the step S8, feature enhancement processing is performed on the numbered area image cropped from the wire image to output a binary image.

[0025] As a further solution of the present invention: in the step S8, the feature-enhanced numbering region image is input into the trained YOLOv5 wire numbering character model for inference to obtain the wire numbering data.

[0026] As a further solution of the present invention: the image data set used in training the numbered character recognition model is the image of the entire numbered area, and the trained model can distinguish easily confused characters such as "6", "9", "M", and "W" through global position information.

[0027] Compared with the prior art, the beneficial effects of this application are:

[0028] 1. The present invention uses a macro camera with adjustable height and fixed focus as an image acquisition device, and can quickly acquire clear wire images.

[0029] 2. The present invention uses the single-stage neural network YOLOv5 as the basic network, which can realize the rapid recognition of the wire numbering area and the characters in the numbering area in the image, and the model reasoning speed of the area and the characters is less than 100ms.

[0030] 3. The present invention collects wire number images of various types, various fonts, and containing various characters as data sets, and the trained wire number area recognition model and wire number character recognition model can realize efficient and high-precision recognition of wire numbers of different types, different fonts, and containing different characters.

[0031] 4. The present invention uses an image processing method to perform feature enhancement processing on the wire number area and then uses a model to perform character recognition, which can reduce image complexity and improve the speed and accuracy of model reasoning.

[0032] 5. The present invention uses a fixed-focus macro camera as an image acquisition device and a single-stage neural network YOLOv5 as an inference model. In the application, the present invention can realize accurate and rapid recognition of wire numbers, thereby helping assemblers to quickly recognize and match assembly wire numbers.

[0033] The present application is further described below in conjunction with the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0035] In the attached picture:

[0036] Figure 1 It is a flow chart of a wire number recognition method based on the single-stage neural network algorithm YOLOv5 in the present invention;

[0037] Figure 2 A YOLOv5 neural network structure diagram of a wire number recognition method based on a single-stage neural network algorithm YOLOv5 in the present invention;

[0038] Figure 3 It is a labeling image of the numbered area in the collected wire image in the present invention;

[0039] Figure 4 It is a labeling image of the numbered characters in the wire numbering area image after cutting and enhancing the image features in the present invention;

[0040] Figure 5 The effect images are of the actual application of the wire number recognition method based on the single-stage neural network algorithm YOLOv5 in the present invention, wherein: (a) is the collected wire image, (b) is the number area image cropped by the number area recognition model, (c) is the number area image after image feature enhancement, and (d) is the number information image recognized by the number character recognition model. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0042] like Figures 1 to 3As shown, the present invention discloses a wire number recognition method based on a single-stage neural network algorithm YOLOv5, which comprises the following steps:

[0043] Step 1: Use a fixed-focus macro camera as an image acquisition device to collect images of numbered wires containing various types, fonts, and characters, and construct a wire image dataset;

[0044] Step 2: label and segment the numbered regions in the above dataset images to construct training set data and test set data for the numbered region model;

[0045] Step 3: construct a YOLOv5 neural network and input the above-annotated wire image dataset to train and optimize the numbered area recognition model;

[0046] Step 4: Use the trained YOLOv5 wire number area recognition model to perform number area recognition and cropping on the wire image dataset, and save the cropped wire number area image;

[0047] Step 5, performing image feature enhancement processing on the saved wire number area image and saving it;

[0048] Step 6: annotate and segment the wire number area image after feature enhancement to form training set data and test set data;

[0049] Step 7: construct a YOLOv5 neural network, input the labeled number area character data set to train and optimize the wire number recognition model;

[0050] Step eight, use the trained YOLOv5 wire number area recognition model to crop the wire image for the number area, then enhance the image features of the cropped number area image and input it into the YOLOv5 wire number character recognition model for character recognition and output the number information.

[0051] Optionally, in step 2, the numbered areas in the collected wire image are marked by box selection, and the image numbered area position information file is saved as a data set for the YOLOv5 numbered area recognition model.

[0052] Optionally, in step 5, the numbered region image cropped from the original image and the inverse feature enhancement processing include the following steps:

[0053] Step S51, converting the wire number area image from a color image to a grayscale image;

[0054] Step S52, performing Gaussian filtering to reduce noise on the converted grayscale image;

[0055] Step S53, performing adaptive threshold segmentation on the denoised image to convert it into a binary image.

[0056] Preferably, in step six, the wire number region image after feature enhancement is annotated character by character, and the data ratio is adjusted to ensure a balanced number of annotated characters, thereby constructing a number region character data set.

[0057] In one embodiment, the identification method of the present invention is used as follows:

[0058] 1. Collect wire images

[0059] like Figure 5 As shown in (a), a macro camera with adjustable height and fixed focus is used to capture images of the numbered wires.

[0060] 2. Identification and segmentation of numbered regions

[0061] like Figure 5 As shown in (b), the captured image is input into the trained numbered region recognition model for region recognition. If a character region is detected, the character region is cropped and saved.

[0062] 3. Enhanced numbering area features

[0063] like Figure 5 As shown in (c), the image feature enhancement method is to grayscale the cropped numbered region image and perform Gaussian noise reduction and adaptive thresholding.

[0064] Grayscale: The wire image captured by the camera is an RGB color image. The RGB image is composed of three color channels: red (R), green (G), and blue (B). The value range of each channel is 0 to 255. A grayscale image is an image with only one channel. Each pixel only contains grayscale values, which usually range from 0 to 255. Before converting an RGB image to a grayscale image, the RGB image needs to be converted to another color space. In this embodiment, the RGB image is first converted to the YUV color space, and then the brightness channel Y is selected as the pixel value of the grayscale image, and the brightness value is mapped to the range of 0 to 255 to obtain a grayscale image.

[0065] For the conversion from RGB to YUV, the calculation formula for the brightness component Y is as follows:

[0066] Y=0.299×R+0.587×G+0.114×B;

[0067] Map the brightness value Y to the range of 0 to 255 to obtain a grayscale image.

[0068] Gaussian Noise Reduction: Gaussian filter is a linear filter that uses Gaussian function to smooth the image. Gaussian function is a bell-shaped curve in the spatial domain, which can perform smoothing operations in the image, retaining the overall structure of the image while reducing the noise in the image. The core idea of ​​Gaussian filter is to apply a small window around each pixel, and the pixel values ​​in the window are weighted averaged by Gaussian weights to get the new value of the pixel. During the filtering process, pixels closer to the target pixel will have higher weights.

[0069] For each pixel in the image, the Gaussian filter is filtered by the following steps:

[0070] 1) Define a filter kernel of size N×N (usually an odd number), where N is a positive integer representing the size of the filter.

[0071] 2) Place the filter kernel at each pixel position in the image.

[0072] 3) For each pixel position, convolve the filter kernel with the pixel value in the image and use the result as the new value of that pixel.

[0073] 4) The convolution operation is to multiply each element in the filter kernel with the pixel value at the corresponding position in the image, and take the sum of all products as the new pixel value.

[0074] 5) In the convolution operation, the element values ​​in the filter kernel are calculated by a Gaussian function and are used to assign weights to the surrounding pixels.

[0075] Adaptive thresholding: Gaussian weighted average adaptive thresholding is an image segmentation method that calculates the local threshold based on the weighted average of the local area. In this method, each pixel in the local area is assigned a different weight, which is calculated based on the Gaussian function. The principle is as follows:

[0076] 1) Local region division: First, the image is divided into many small local regions (usually square or rectangular). The size of these local regions can be set, usually called "block size".

[0077] 2) Gaussian weight calculation: For each local area, a Gaussian weighted average is calculated as the local threshold. The Gaussian weighted average is calculated based on the weighted average of the pixels in the local area, and the weight is calculated based on the Gaussian function. The form of the Gaussian function is:

[0078]

[0079] Where (x, y) is the position of the pixel relative to the center of the local region, n is the size of the local region (the number of rows or columns of pixels), and σ is the standard deviation of the Gaussian kernel.

[0080] 3) Local threshold calculation: For each local area, multiply the gray value of the pixel by the corresponding Gaussian weight, and calculate the weighted average as the local threshold. This local threshold will be used to divide the pixels in the local area into foreground and background.

[0081] 4) Thresholding operation: For each pixel in the image, it is compared with the local threshold of the corresponding local area. If the pixel value is greater than the local threshold, the pixel is marked as white (255), otherwise it is marked as black (0).

[0082] 5) Output image: Based on the result of the thresholding operation, a binary image is generated, which contains the target object or area of ​​interest in the original image.

[0083] 4. Character Recognition

[0084] like Figure 5 As shown in (d), the feature-enhanced image is sent to the trained numbered character recognition model for inference, and the numbered character recognition result is output and displayed.

[0085] So far, the purpose of the present invention has been achieved.

[0086] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A wire number recognition method based on the single-stage neural network algorithm YOLOv5, characterized in that: The steps include: S1, using an image acquisition device to acquire images of numbered wires containing various types, various fonts, and various characters, and constructing a wire image dataset; S2, labeling and segmenting the numbered areas of the wire images in the wire image dataset to construct training set data and test set data; S3, constructing a YOLOv5 neural network, inputting the wire image dataset with numbered area annotation to train and optimize the numbered area recognition model; S4, using the trained YOLOv5 wire number area recognition model to perform number area recognition and cropping on the wire image dataset, and saving the cropped wire number area image; S5, performing image feature enhancement on the saved wire number area image and saving it, and constructing a number area image data set; S6, annotating and segmenting the wire number area image after feature enhancement to construct training set data and test set data; S7, constructing a YOLOv5 neural network, inputting the labeled number area character data set to train and optimize the wire number character recognition model; S8, using the trained YOLOv5 wire number area recognition model to crop the wire image for the number area, then enhancing the image features of the cropped number area image, and inputting it into the YOLOv5 wire number character recognition model to perform character recognition and output the number information.

2. The wire number recognition method based on the single-stage neural network algorithm YOLOv5 according to claim 1, characterized in that: In step S1, a fixed-focus macro camera is used as an image acquisition device to acquire images.

3. The wire number recognition method based on the single-stage neural network algorithm YOLOv5 according to claim 1, characterized in that: In the step S2, the numbered areas in the collected wire image are marked by box selection, and the marked image number area position information file is saved as a data set for the YOLOv5 number area recognition model.

4. The wire number recognition method based on the single-stage neural network algorithm YOLOv5 according to claim 1, characterized in that: In step S5, the processing method specifically includes the following steps: S51, converting the cropped wire number area image from a color image to a grayscale image; S52, performing Gaussian filtering to reduce noise on the converted grayscale image; S53, performing adaptive threshold segmentation on the denoised image to form a binary image.

5. The wire number recognition method based on the single-stage neural network algorithm YOLOv5 according to claim 1, characterized in that: In step S6, the feature-enhanced wire number region image is annotated character by character, and the data ratio is adjusted to ensure a balanced number of annotated characters, thereby forming a number region character data set.

6. The wire number recognition method based on the single-stage neural network algorithm YOLOv5 according to claim 1, characterized in that: In step S8, the wire image acquired by the image acquisition device is directly input into the trained numbered area recognition model for inference to obtain a cropped image of the numbered area.

7. The wire number recognition method based on the single-stage neural network algorithm YOLOv5 according to claim 6 is characterized in that: In the step S8, feature enhancement processing is performed on the numbered area image cut out from the conductor image to output a binary image.

8. The wire number recognition method based on the single-stage neural network algorithm YOLOv5 according to claim 7 is characterized in that: In step S8, the feature-enhanced numbering region image is input into the trained YOLOv5 wire numbering character model for inference to obtain wire numbering data.

9. The wire number recognition method based on the single-stage neural network algorithm YOLOv5 according to any one of claims 1 to 8, characterized in that: The image data set used in the training of the numbered character recognition model is the image of the entire numbered area, and the trained model can distinguish easily confused characters such as "6", "9", "M", and "W" through global position information.

Images