Monitoring method for preventing mistaken loading and neglected loading of automobile parts based on machine vision

Abstract

According to the machine vision-based monitoring method for preventing mistaken loading and neglected loading of the automobile parts, a camera is installed beside a final assembly production line of an automobile manufacturing plant, a shot video stream is transmitted to a monitoring computer in real time through a network cable, and the computer reads a PLC signal of the production line once every two seconds to obtain the real-time position state of an automobile; when the position of the automobile reaches a designated position, the computer stores a video frame of the camera at the moment, the stored picture is calculated through a YOLOv5s model algorithm to obtain an assembly configuration PR number of a current automobile part, the recognized PR number is compared with a PR number of a real-time access automobile configuration database, if the two PR numbers are different, it is indicated that the automobile assembly part is wrong, and if the two PR numbers are not different, it is indicated that the automobile assembly part is wrong. At the moment, the computer sends a signal to turn on a field warning lamp, and sends a mobile phone to remind a corresponding person in charge of nailing. The method can be applied to monitoring of all parts capable of being shot by the camera in automobile assembly, the labor cost input is greatly reduced, and the detection accuracy and efficiency are high.

Description

Technical Field

[0001] The invention relates to the technical field of automobile production and assembly, and in particular to a machine vision-based monitoring method for preventing mis-assembly and missing assembly of automobile parts. Background Art

[0002] As the general public's understanding of automobile products improves, they have put forward more requirements for automobile parts configuration, which has also led to a richer configuration of parts in the automobile production process. In the process of automobile production and assembly, how to avoid the risk of incorrect or missing assembly of automobile parts during assembly on the production line has become a difficult problem in the production process. In the past, the general assembly workshop mostly solved this problem by increasing the number of exit personnel for inspection. From the actual operation experience, the reliability of human inspection is very low, especially for parts involving multiple models and multiple configurations. Inspection personnel may make identification errors, resulting in the flow of incorrectly configured cars into the hands of consumers, resulting in after-sales complaints and after-sales repair claims and economic losses.

[0003] In view of the above problems, the inventors of the present invention finally obtained the present invention after a long period of research and practice. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a method for monitoring automobile parts for preventing wrong or missing installation based on machine vision, which can be applied to the monitoring of all parts that can be photographed by a camera in an automobile assembly line and has high reliability.

[0005] The technical solution adopted by the present invention is:

[0006] A method for monitoring automobile parts for error-proof missing assembly based on machine vision is provided, comprising the following steps:

[0007] S1. Obtaining an algorithm model for image detection of parts to be monitored by training a neural network model;

[0008] S2. Install a camera next to the final assembly line, and transmit the video stream captured by the camera to the monitoring computer in real time;

[0009] S3, the monitoring computer reads the production line PLC signal to obtain the real-time position of each vehicle on the production line;

[0010] S4. When the vehicle arrives at the designated shooting position of the production line, the monitoring computer saves the video frame shot by the camera at this time, and identifies the assembly configuration PR number of the required monitoring part in the currently saved picture through algorithm model calculation;

[0011] S5. Compare the identified PR number with the actual production requirement PR number. If the two PR numbers are the same, it is determined that the parts installed on the car are correct; if the two PR numbers are different, it is determined that the parts assembled on the car are wrong.

[0012] Furthermore, in step S1, the image detection algorithm model of the required monitoring parts is obtained by training the neural network model, which specifically includes:

[0013] S1.1. For each part, obtain pictures of its different configurations;

[0014] S1.2, feature labeling of the acquired images to form model training data sets and test data sets;

[0015] S1.3. Based on the formed training data set and test data set, the YOLOv5s algorithm model is used to perform network training. After the training is completed, the algorithm model for image detection of the required monitored parts is obtained.

[0016] Furthermore, in step S1.2, Labelimg software is used to perform feature annotation on the image.

[0017] Furthermore, in step S1.3, based on the YOLOv5s algorithm model, Mosaic data enhancement operation, adaptive anchor box calculation, and adaptive image scaling are applied to improve the model training speed and network accuracy.

[0018] Furthermore, in step S2, a camera is set at a workstation after the assembly of the parts to be monitored is completed.

[0019] Furthermore, in step S4, the monitoring computer inputs the saved image into the algorithm model. The judgment threshold set by the algorithm model is 0.45. After the algorithm model operates, when the threshold of the recognized object is greater than 0.45, the PR number of the image recognition at this time is output and the detection box is marked in the image.

[0020] Furthermore, the monitoring method also includes step S6, when it is determined that the installed parts are wrong, the monitoring computer issues an alarm instruction.

[0021] Furthermore, the monitoring method further includes step S7, storing the vehicle configuration information, the PR number of the image recognition, and the determination result in a database.

[0022] Compared with the prior art, the beneficial effects of the present invention are:

[0023] The overall cost of the monitoring method of the present invention is very low. It only requires the purchase of cameras installed on the production line. All parts detection programs in the entire factory are run on a server. Through the application of this technology, the input of one worker can be reduced for each part, and the reliability is high. It can meet the parts detection requirements within the production rhythm, thereby improving the quality of vehicle production and preventing cars with wrongly installed parts from entering the hands of customers. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 A flow chart showing a monitoring method according to an embodiment of the present invention;

[0026] Figure 2 A monitoring schematic diagram of a monitoring method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0027] The above and other technical features and advantages of the present invention are described in more detail below in conjunction with the accompanying drawings.

[0028] The present invention installs a camera next to the final assembly line of the automobile production plant, and the captured video stream is transmitted to the monitoring computer in real time through the network line. The computer reads the production line PLC signal every 2 seconds to obtain the real-time position status of the automobile. When the automobile position reaches the specified position, the computer saves the video frame of the camera at this time, and the saved picture is calculated by the YOLOv5s model algorithm to obtain the assembly configuration PR number of the current automobile parts. The identified PR number is compared with the PR number of the real-time access vehicle configuration database. If the two PR numbers are different, it means that the parts assembled by the automobile are wrong. At this time, the computer sends a signal to turn on the red light of the on-site warning light, and sends a mobile phone DingTalk reminder to the corresponding person in charge. This technology can be applied to the monitoring of all parts of automobile assembly that can be captured by the camera.

[0029] like Figure 1 , Figure 2 As shown, in a specific embodiment, a monitoring method for preventing errors and missed assembly of automobile assembly based on machine vision recognition is based on the YOLOv5s algorithm model, and applies Mosaic data enhancement operation, adaptive anchor frame calculation, and adaptive image scaling to improve the model training speed and network accuracy, so as to achieve the accuracy of target detection of real-time and rapid recognition of current parts.

[0030] Taking the tail label parts of automobiles as an example, there are a total of 8 types in the final assembly workshop of our company's Foshan factory. At present, the final inspection personnel need to first determine the engine configuration of the vehicle before inspection, and then check whether the tail label is installed incorrectly. This manual inspection method has the risk of personnel identification errors and requires a lot of inspection time, which increases the cost of inspection personnel.

[0031] The monitoring method of the present invention is specifically implemented in the following steps:

[0032] Step S1: The camera is installed at the workstation after the tail label parts are assembled. Camera 2 is used to obtain pictures of tail label parts with different configurations. There are 8 configurations in our company's actual tail label parts application. 200-300 pictures are collected for each tail label configuration, which are used for training and testing data sets respectively.

[0033] Step S2: Use Labelimg software to annotate the prepared images to form a model training data set, with a total of 2516 images; select 30 images for each configuration, annotate them, and form a test data set for model training.

[0034] Step S3: Based on the training and test data sets prepared in step S2, the YOLOv5 algorithm model is used for network training. After the training is completed, an algorithm model for image detection of tail-label parts is obtained.

[0035] Step S4: The computer program is set to read the PLC signal controlling the production line every 2 seconds to obtain the real-time position information of each vehicle 1 on the production line.

[0036] Step S5: When the vehicle arrives at the designated shooting position, the computer 3 saves the video frame image shot by the camera at this time.

[0037] Step S6: Input the real-time saved image to the YOLOv5s algorithm detection program. The program sets the judgment threshold of 0.45. After the algorithm operation, when the threshold of the recognized object is greater than 0.45, the PR number of the image recognition at the end of the label is output (the configuration is determined by the PR number in production), and the detection box 4 is marked in the image ( Figure 2 Also shown is a detection frame 5) taking a hub part as an example.

[0038] Step S7: Compare the PR number identified by the image with the actual production requirement PR number. If the PR numbers are the same, it is determined that the tail label part actually installed is correct. If the PR numbers are different, it is determined that the tail label actually installed is wrong.

[0039] Step S8: After determining that the installed tail label is wrong, the computer program issues a command to turn the on-site alarm light red, and at the same time sends a message to the production manager's mobile phone, such as a DingTalk APP reminder message.

[0040] Step S9: All vehicle configuration information, PR number of image recognition, and determination results are stored in the database for subsequent tracing and query.

[0041] The overall cost of the detection system corresponding to this monitoring method is very low. It only requires the purchase of cameras installed on the production line, and the rest is human and technical investment. In a practical example, the camera used is a Hikvision full-color POE camera, with a unit cost of less than 500 yuan. The detection of the wheel hub requires the installation of two cameras on the left and right sides of the production line. All parts detection programs in the entire factory are run on a server, and the total investment in the detection system for each part is about 1,000 yuan. Through the application of this technology, the investment of one worker can be reduced for each part, and the reliability is high.

[0042] At present, manufacturers still rely on manual inspection to achieve error prevention of installed parts. The present invention uses machine vision recognition to prevent errors. The system has low investment. Each camera can replace one worker, which greatly reduces the labor cost investment. The detection accuracy and efficiency are high. According to the actual operation results, the system is better than manual inspection and can meet the requirements of parts inspection within the production cycle, thereby improving the quality of vehicle production and preventing cars with wrongly installed parts from entering the market.

[0043] Those skilled in the art will appreciate that the algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0044] The above are only preferred embodiments of the present invention, which are only illustrative and not restrictive. The structures and connection modes of the components in the present invention can be changed, and any equivalent transformations and improvements based on the technical solution of the present invention should not be excluded from the protection scope of the present invention.

Claims

1. A method for preventing wrong or missing assembly of automobile parts based on machine vision, characterized in that: The following steps are involved: S1. Obtaining an algorithm model for image detection of parts to be monitored by training a neural network model; S2. Install a camera next to the final assembly line, and transmit the video stream captured by the camera to the monitoring computer in real time; S3, the monitoring computer reads the production line PLC signal to obtain the real-time position of each vehicle on the production line; S4. When the vehicle arrives at the designated shooting position of the production line, the monitoring computer saves the video frame shot by the camera at this time, and identifies the assembly configuration PR number of the required monitoring part in the currently saved picture through algorithm model calculation; S5. Compare the identified PR number with the actual production requirement PR number. If the two PR numbers are the same, it is determined that the parts installed on the car are correct; if the two PR numbers are different, it is determined that the parts assembled on the car are wrong.

2. A monitoring method according to claim 1, characterized in that: In step S1, the image detection algorithm model of the required monitoring parts is obtained by training the neural network model, which specifically includes: S1.

1. For each part, obtain pictures of its different configurations; S1.2, feature labeling of the acquired images to form model training data sets and test data sets; S1.

3. Based on the formed training data set and test data set, the YOLOv5s algorithm model is used to perform network training. After the training is completed, the algorithm model for image detection of the required monitored parts is obtained.

3. A monitoring method as claimed in claim 2, characterized in that: In the step S1.2, Labelimg software is used to perform feature annotation on the image.

4. A monitoring method as claimed in claim 3, characterized in that: In step S1.3, based on the YOLOv5s algorithm model, Mosaic data enhancement operation, adaptive anchor box calculation, and adaptive image scaling are applied to improve the model training speed and network accuracy.

5. A monitoring method according to claim 1, characterized in that: In step S2, the camera is arranged at a workstation after the assembly of the parts to be monitored is completed.

6. A monitoring method according to claim 1, characterized in that: In step S4, the monitoring computer inputs the saved image into the algorithm model. The judgment threshold set by the algorithm model is 0.

45. After the algorithm model operates, when the threshold for identifying the object is greater than 0.45, the PR number of the image recognition at this time is output and the detection box is marked in the image.

7. A monitoring method according to claim 1, characterized in that: The method further comprises step S6, wherein when it is determined that the installed parts are wrong, the monitoring computer issues an alarm instruction.

8. A monitoring method according to claim 1, characterized in that: The method further includes step S7, wherein the vehicle configuration information, the PR number of the image recognition, and the determination result are stored in a database.

Images