Aircraft standard component identification and counting method based on deep learning
Through deep learning technology combining object detection and optical character recognition, the problem of heavy counting of aircraft standard parts and large counting errors is solved, automatic recognition and counting is realized, work efficiency and accuracy are improved, and manual recording is replaced. A more reliable and convenient inventory method.
Patent Information
- Application Number
- CN202411217699.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-02
- Publication Date
- 2024-12-20
AI Technical Summary
The quantity counting of aircraft standard parts before entering the warehouse is heavy, and the single delivery volume is large, resulting in large counting errors, high labor costs, and inaccurate inspection of large batches of standard parts.
Using deep learning-based methods, combined with object detection and optical character recognition technology, standard parts recognition and counting are realized through data preparation and processing, model selection and training, model optimization and deployment applications, and offline deployment is used by Raspberry Pi to increase counting. Flexibility and accuracy of the program.
It realizes automatic identification of standard parts types and calculation of small and large-batch standard parts quantity, reduces manual counting errors, improves work efficiency and accuracy, replaces human eye recognition and manual recording, and improves work quality.
Smart Images
Figure CN119169266A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of intelligent manufacturing of aviation manufacturing enterprises and provides a method for identifying and counting aircraft standard parts based on deep learning. Background Art
[0002] Before aircraft standard parts are put into storage, they need to be counted. Due to the large variety of standard parts and the large single delivery volume, the counting of standard parts is extremely arduous. In order to reduce the deviation of the overall number of standard parts delivered and improve work efficiency, deep learning technologies such as target detection and optical character recognition are used in combination with electronic weighing to automatically identify the type of standard parts and calculate the number of standard parts in the pallet, solving the problem of counting large quantities of standard parts. This can reduce the intensity of manual counting, reduce the counting errors, and make the weighed number of standard parts closer to the actual number.
[0003] Deep learning is a branch of machine learning, which learns data features by building multi-level neural networks. Common counting models include convolutional neural networks, yolo series models, etc. Common model evaluation indicators include precision, recall, mAP, etc. Among them, precision is the ratio of the number of correctly predicted objects to the total number of predicted objects, recall is the ratio of the number of correctly predicted objects to the number of real objects, and mAP is the area under the pr curve composed of precision and recall. Common optimizers include Momentum, RMSProp, Adam, etc. Among them, the momentum method Momentum accelerates training by accumulating the exponential decay average of previous gradients, the adaptive learning method RMSProp accelerates training by taking the square root of past gradients, and the adaptive moment estimation Adam calculates the adaptive learning rate of each parameter and combines the first-order moment and the second-order moment to dynamically adjust the learning rate.
[0004] Currently, the number of objects is mainly calculated through target detection using neural network models such as YOLO. However, when counting large quantities of objects, there are problems such as object occlusion and overlap, which lead to inaccurate detection. A more effective method to solve such problems is needed. Summary of the invention
[0005] In order to solve the problems of difficulty in identifying standard parts, heavy counting tasks, high labor costs, and inaccurate detection of large quantities of standard parts, the present invention proposes an aircraft standard parts identification and counting method based on deep learning, which realizes the functions of standard parts identification, weighing number identification, batch standard parts calculation, information recording, etc.
[0006] The method of the present invention is achieved by the following steps (such as Figure 1 ):
[0007] A method for aircraft standard parts recognition and counting based on deep learning, the steps are as follows:
[0008] S1: Data preparation and processing, collecting different types of standard parts image data and performing preprocessing and data enhancement; S2: Algorithm model selection and training, using standard parts images as training data sets and test data sets, selecting different types of neural network models for training; S3: Algorithm model optimization, adjusting the parameters of each neural network model, and improving the algorithm model evaluation indicators; S4: Service release, deploying the model with the best evaluation indicator as a service and releasing it, providing available interface services to the outside world; S5: Application deployment, deploying applications in both online and offline methods; S6: Standard parts identification and counting.
[0009] Step S1 Data preparation and processing: In the actual working environment of the production workshop, use a high-definition camera to take pictures of different types of standard parts. The number of pictures should be between 600 and 1000. For the pictures taken, use methods such as de-approximation and deblurring to reduce duplication and display low-quality pictures. According to the category of standard parts, mark the name of the standard parts as the category name of the training model, and then use a rectangular frame to select each standard part on the standard parts picture to complete the picture annotation. Use data enhancement methods such as flipping, random occlusion, and brightness adjustment to eliminate noise interference such as occlusion, lighting, and position. At the same time, expand the number of pictures to enrich the image data and improve the generalization ability of the model.
[0010] Step S2: Algorithm model selection and training: Divide the data-enhanced standard parts image dataset into a training dataset and a test dataset in an 8:2 ratio. Select target detection models, such as the real-time target detection yolo series, Baidu Paddle Yolo series, regional convolutional neural network rcnn series and other neural network models. After initializing common hyperparameters, use the selected multiple models to perform multiple rounds of training based on the graphics processing unit GPU.
[0011] Step S3: Algorithm model optimization: continuously adjust the hyperparameter size of each model, such as input image size, batch size, epoch, learning rate, weight decay, optimizer, momentum factor and other model parameters. When the evaluation index of the model is greater than the preset threshold, select the model with the best evaluation performance from multiple optimized models as the subsequent service release model.
[0012] Step S4 Service Release: Use the selected model with the best evaluation performance to release the service and provide an interface connection to external applications. Receive json format data, such as base64 encoding of images and thresholds, etc. Return json format data, such as error code error_code, error message error_msg, result message result, etc.
[0013] Step S5 deploys the application: deploys the application based on two hardware devices, ordinary computers and Raspberry Pi, to improve the ease of use. Write a program on an ordinary computer, and you can send a base64-encoded post request containing a standard part image to the model service published by the server online, so as to obtain data information such as the standard part category name and rectangular box position returned by the model. On the Raspberry Pi, a variety of model compression methods are used to compress the model with the best evaluation index into a smaller model, and the model with the highest recognition accuracy greater than the threshold is selected as the service publishing model, and deployed offline on the Raspberry Pi device to realize service publishing, interface calling and other functions.
[0014] Step S6: Standard part identification and counting (such as Figure 2 ): Place small batches of standard parts on the weighing device, use the program to open camera 1, take pictures of standard parts, upload them to the interface service for processing, return the standard part anchor frame and standard part category name, and then calculate the number of standard parts. Open camera 2, take pictures of the weighing numbers on the display screen of the electronic weighing device, call the OCR model to recognize the numbers, then place large batches of standard parts on the weighing device, recognize the weighing numbers, and calculate the number of large batches of standard parts by comparing them with small batches of standard parts. If the calculation is successful, use the database to record the operation time, standard part category name, small batch standard part quantity, small batch standard part weight, large batch standard part quantity, large batch standard part weight, etc. Otherwise, restart step S6.
[0015] Beneficial effects of the present invention:
[0016] Aiming at the problems of heavy inventory work, large counting errors, high labor costs, low work efficiency, and inaccurate detection of large batches of standard parts, this paper proposes a method for identifying and counting aircraft standard parts based on deep learning. This method uses deep learning technologies such as target detection, optical character recognition, and model compression to achieve the effects of automatically identifying the type of standard parts and calculating the number of small and large batches of standard parts.
[0017] This method uses target detection technology to improve the accuracy of standard parts recognition and counting. Optical character recognition technology is used to read weighing numbers to solve the problem of large-volume standard parts counting difficulties. Model compression technology is used to deploy model services offline based on Raspberry Pi to improve the flexibility of the counting program. Counting information is recorded in the database instead of manual recording, which is convenient for later query and maintenance.
[0018] In summary, the standard parts identification and counting method provided by the present invention can solve the problems existing in the warehousing process of existing aircraft standard parts, and replaces the methods of human eye identification of standard parts types, manual counting by personnel, manual recording, etc., making the standard parts warehousing and inventory work more reliable, efficient, and convenient, and improving the work quality. There are two ways of use: one is to use an ordinary computer online to achieve fast and accurate counting by calling the server API; the other is to use the Raspberry Pi offline, compress the model, and deploy it offline based on the local model to achieve flexible and convenient use. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 The figure is a flow chart of the method of the present invention.
[0020] Figure 2 This is a processing flow chart of the standard part identification and counting functions of the present invention.
[0021] Figure 3 This is a functional structure diagram of standard parts identification and counting according to the present invention. DETAILED DESCRIPTION
[0022] The following is an example of the method for standard parts identification and counting of the present invention, which is mainly implemented using python3.8 and paddlepaddle-v2.2.2 framework to illustrate the technical solution of the present invention. The implementation process is as follows.
[0023] 1. Data preparation and processing. In the actual working environment of the production workshop, under different environmental, lighting, and background conditions, a high-definition camera was used to vertically shoot pictures of 5 commonly used types of standard parts. A total of 649 pictures were collected with a resolution of 1920×1080. For the pictures taken, pillow and opencv were used to process duplicate and blurred pictures. According to the category of standard parts, on each picture, the rectangular box tool of labelImg was used to select each standard part to complete the picture labeling. ImageEnhance was used to perform data enhancement operations such as flipping, random occlusion, and brightness adjustment on the pictures to eliminate noise interference. At the same time, the number of pictures was expanded to 2384 to enrich the image data and improve the generalization ability of the model.
[0024] 2. Algorithm model selection and training. Use paddle's data loader DataLoader to divide the image dataset into training dataset and test dataset in a ratio of 8:2. Select common target detection models, such as yolo_darknet, ppyolo_r50, cascade_rcnn_r50, ppyolo_tiny, ppyolov2_r50vd, ppyolov2_r101vd, ppyolo_mobilenetv3_large, ppyolo_mobilenetv3_small and other neural network models. Initialize the common hyperparameters for model training to form a tuple p(in, ba, ep, lr, wd, op, mo), where in is the input image size inputsize, ba is the batch size batchsize, ep is the training round epoch, lr is the learning rate learningrate, wd is the weight decay weightdecay, op is the optimizer optimizer, and mo is the momentum factor momentum. For the training data set and the test data set, the parameter selection tuple p is used. Based on the hardware environment of 2-core CPU, 8G memory, and 1 a100 graphics card, the above-mentioned multiple models are used for ep training times.
[0025] 3. Algorithm model optimization. Constantly adjust the size of each parameter in the tuple, use the above multiple models for continuous training, and use the model evaluation value to is the measurement standard, where mAP is the mean average precision, pre is the precision, and rec is the recall. When the evaluation value v of the model>96%, stop training this model and continue training the next model. During actual training, for the standard parts dataset in step 1, when in=(608,608), ba=2, ep=150, lr=0.000052, wd=0.005, op=Momentum, mo=0.9, the evaluation indicators of the model PPYOLOv2_R50vd_DCNV2 are mAP=97.23%, pre=95.12%, rec=96.41%. According to the formula, the evaluation value v=96.25% is calculated as the optimal performance indicator, and this model is selected as the subsequent service release model.
[0026] 4. Service publishing. Use paddle serving based on the GPU environment to publish the selected model and provide interface connection services to the outside world. The interface receives the post request data sent by the client. The json format of the request is j=
[0027] {"image":"base64 encoding",threshold:"threshold"}, where the image key value corresponds to the base64 encoding of the passed image, and the threshold key value corresponds to the passed image recognition confidence threshold. The interface returns a json object consisting of the error code error_code, error message error_msg, and result message result.
[0028] 5. Deploy the application. Use tkinter to write the front-end page, and design the counting interface and weighing interface respectively. The counting interface is used to identify the type and quantity of standard parts, and the weighing interface is used to identify the numbers on the display screen of the electronic weighing device. Deploy the application based on two hardware devices, ordinary computers and Raspberry Pi, to improve the convenience of application use. On an ordinary computer, provide the required json format data j to the model service published online to the server, and obtain the model return value error code error_code, error message error_msg, result information result, etc. According to the result value, return the tuple data r(loc,name,score), where loc is the value containing the height, width, left border, right border, etc. of the rectangular box, name is the name of the standard part category, and score is the confidence. According to the r value, add a rectangular box to frame each standard part on the picture, display the standard part category, and count Calculate the number of standard parts, where l k is the number of each type of standard parts returned, and n is the number of standard parts types. On the Raspberry Pi 4B, the model is first pruned and quantized to reduce model parameters and speed up the Raspberry Pi inference model. When the accuracy of the compressed model is greater than 93%, stop compressing the model. The model pruning accuracy is 93.5%, and the quantization compression accuracy is 95.2%. The optimal model is the quantization compression model. The quantization compression model is deployed offline to the Raspberry Pi to detect standard parts. The specific process is the same as that of an ordinary computer.
[0029] 6. Identification and counting of standard parts. Place a small batch of standard parts on the tray of the electronic weighing equipment, use OpenCV to open camera 1, vertically shoot the picture of the standard parts on the tray, convert the picture into base64 encoding, send a post request, upload it to the interface service for processing, return the standard part anchor frame and standard part name, and then calculate the number of standard parts C1. Open camera 2, shoot the weighing number on the display screen of the electronic weighing equipment, call the paddle OCR model to recognize the number W1. Place a large batch of standard parts on the tray of the weighing equipment, shoot the weighing number in the display box of the weighing equipment, call the paddle OCR model to recognize the number W2, and calculate the number of large batch standard parts by comparing it with the small batch standard parts. If the calculation is successful, use the database to record the operation time time, standard parts category type, small batch standard parts quantity small_count, small batch standard parts weight small_weight, large batch standard parts quantity big_count, large batch standard parts weight big_weight, etc. Otherwise, restart step 6.
Claims
1. A method for identifying and counting aircraft standard parts based on deep learning, characterized in that: Here are the steps: S1: Data preparation and processing, collecting different types of standard parts image data and performing preprocessing and data enhancement; S2: Algorithm model selection and training, using standard parts images as training data sets and test data sets, and selecting different types of neural network models for training; S3: Algorithm model optimization, adjust the parameters of each neural network model, and improve the algorithm model evaluation indicators; S4: Service release: deploy the model with the best evaluation index as a service and release it, providing available interface services to the outside world; S5: Deploy applications, deploy applications online and offline; S6: Standard parts identification and counting.
2. The method for recognizing and counting aircraft standard parts based on deep learning according to claim 1, characterized in that: The step S1 data preparation and processing is as follows: in the actual working environment of the production workshop, different types of standard parts pictures are taken, and the number of pictures should be between 600 and 1000; for the taken pictures, de-approximation and de-blurring methods are used to reduce duplication and pictures with low display quality; according to the standard parts category, the standard parts name is marked as the training model category name, and then each standard part is selected using a rectangular frame on the standard parts picture to complete the picture annotation work; using flipping, random occlusion, and brightness adjustment data enhancement methods to eliminate occlusion, illumination, and position noise interference, and at the same time expand the number of pictures to enrich the image data and improve the generalization ability of the model.
3. The method for identifying and counting aircraft standard parts based on deep learning as claimed in claim 1 or 2, characterized in that: The step S2 algorithm model selection and training: the data-enhanced standard parts image data set is divided into a training data set and a test data set in an 8:2 ratio; the target detection model is selected, and the real-time target detection yolo series, Baidu Paddle Yolo series, and regional convolutional neural network rcnn series neural network models are used. After initializing the commonly used hyperparameters, multiple selected models are used to perform multiple rounds of training based on the graphics processing unit GPU.
4. The method for identifying and counting aircraft standard parts based on deep learning as claimed in claim 1 or 2, characterized in that: The step S3 algorithm model optimization: continuously adjust the hyperparameter size of each model, including input image size inputsize, batch size batchsize, training round epoch, learning rate learningrate, weight decay weightdecay, optimizer optimizer, momentum factor momentum model parameters. When the evaluation index of the model is greater than the preset threshold, select the model with the best evaluation performance from multiple optimized models as the subsequent service release model.
5. The method for recognizing and counting aircraft standard parts based on deep learning as claimed in claim 3, characterized in that: The step S3 algorithm model optimization: continuously adjust the hyperparameter size of each model, including input image size inputsize, batch size batchsize, training round epoch, learning rate learningrate, weight decay weightdecay, optimizer optimizer, momentum factor momentum model parameters. When the evaluation index of the model is greater than the preset threshold, select the model with the best evaluation performance from multiple optimized models as the subsequent service release model.
6. The method for recognizing and counting aircraft standard parts based on deep learning as claimed in claim 1, 2 or 5, characterized in that: The step S5 deploys the application: deploys the application based on two hardware devices, ordinary computer and Raspberry Pi, to improve the convenience of use; A base64-encoded post request containing a standard part image is sent online to the model service published by the server, so as to obtain the standard part category name and rectangular box position data information returned by the model; on the Raspberry Pi, a variety of model compression methods are used to compress the model with the best evaluation index into a smaller model, and the one with the highest recognition accuracy greater than the threshold is selected as the service publishing model, which is deployed offline on the Raspberry Pi device to realize service publishing and interface calling functions.
7. The method for recognizing and counting aircraft standard parts based on deep learning as claimed in claim 3, characterized in that: The step S5 deploys the application: deploys the application based on two hardware devices, ordinary computer and Raspberry Pi, to improve the convenience of use; A base64-encoded post request containing a standard part image is sent online to the model service published by the server, so as to obtain the standard part category name and rectangular box position data information returned by the model; on the Raspberry Pi, a variety of model compression methods are used to compress the model with the best evaluation index into a smaller model, and the one with the highest recognition accuracy greater than the threshold is selected as the service publishing model, which is deployed offline on the Raspberry Pi device to realize service publishing and interface calling functions.
8. The method for recognizing and counting aircraft standard parts based on deep learning as claimed in claim 4, characterized in that: The step S5 deploys the application: deploys the application based on two hardware devices, ordinary computer and Raspberry Pi, to improve the convenience of use; A base64-encoded post request containing a standard part image is sent online to the model service published by the server, so as to obtain the standard part category name and rectangular box position data information returned by the model; on the Raspberry Pi, a variety of model compression methods are used to compress the model with the best evaluation index into a smaller model, and the one with the highest recognition accuracy greater than the threshold is selected as the service publishing model, which is deployed offline on the Raspberry Pi device to realize service publishing and interface calling functions.