Intelligent stamping forming manufacturing method and system based on machine learning

Abstract

The invention provides an intelligent stamping forming manufacturing method and system based on machine learning, and relates to the technical field of intelligent manufacturing, the method comprises the steps that a sample image data set and a real-time production video of part samples on a production line are collected, and the sample image data set comprises normal sample images and abnormal sample images; preprocessing the sample image data set to obtain a labeled image data set; constructing an initial production line sample recognition model based on a convolutional neural network, and inputting a training set and a test set divided by the annotated image data set into the initial production line sample recognition model for training to obtain a production line sample recognition model; and inputting the real-time production video into the production line sample identification model for detection so as to identify abnormal parts on the production line. According to the invention, the quality control precision of the part sample can be improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent manufacturing technology, and in particular to an intelligent stamping manufacturing method and system based on machine learning. Background Art

[0002] The production process from blanks to parts forming is relatively complicated. In order to avoid quality problems in the parts forming process, each process must have strict quality control, otherwise it may directly affect the actual service performance of the parts. At present, the parts forming process still mainly relies on manual operation and manual visual inspection, which is inefficient and easily affected by subjective factors, and has a certain degree of randomness. The effect lacks accuracy and stability. In addition, during the forming process, if the blank deformation accuracy is not enough and the production equipment cannot be controlled to compensate, the forming size of the part will not meet the standard and the product will be scrapped. With the development of machine vision and deep learning technology, automated production lines have gradually become an important means to improve production efficiency and product quality. The automation of production lines will become the mainstream mode of processing and production in the manufacturing industry.

[0003] The Chinese patent with publication number CN116681301A discloses a data-driven autonomous decision-making method for a forging process, which includes the following steps: (1) collecting process data and forging quality inspection data in the forging production process to establish a forging process database; (2) mining frequent item sets from the forging process database to abstract nodes, node attributes and ternary relationships; (3) constructing a forging process knowledge graph based on node relationships, and completing the forging process knowledge graph through knowledge reasoning; (4) constructing and training a forging process reinforcement learning decision network based on the forging process knowledge graph. However, the decision-making method provided by the above application usually requires the knowledge graph to perform simple association calculations, resulting in insufficient accuracy of sample detection results. Therefore, it is very necessary to provide an intelligent stamping manufacturing method and system based on machine learning to improve the quality control accuracy of part samples. Summary of the invention

[0004] In view of this, the present invention proposes an intelligent stamping manufacturing method and system based on machine learning. Through the model built based on the YOLOv5s algorithm and combined with the training of normal and abnormal samples, it is possible to accurately identify abnormal parts on the production line and improve the quality control accuracy of part samples.

[0005] The present invention provides an intelligent stamping forming manufacturing method based on machine learning, the method comprising:

[0006] Collecting a sample image data set and real-time production video of part samples on a production line, wherein the sample image data set includes normal sample images and abnormal sample images;

[0007] Preprocessing the sample image dataset to obtain a labeled image dataset;

[0008] Building an initial production line sample recognition model based on the YOLOv5s algorithm, inputting the training set and the test set divided by the labeled image data set into the initial production line sample recognition model for training, so as to obtain a production line sample recognition model;

[0009] The real-time production video is input into the production line sample recognition model for detection to identify abnormal parts on the production line.

[0010] On the basis of the above technical solution, preferably, the preprocessing of the sample image dataset to obtain the labeled image dataset specifically includes:

[0011] Performing enhancement processing on the normal sample image and the abnormal sample image respectively to obtain enhanced images, wherein the enhancement processing includes adding strong light, shadow and steam mist effects to the normal sample image and the abnormal sample image;

[0012] LabelImg software is used to annotate defective parts of the enhanced image to obtain a data set for training the initial production line sample recognition model, and the data set is divided into a training set and a test set in a ratio of 8:2.

[0013] On the basis of the above technical solution, preferably, the initial production line sample recognition model is constructed based on the YOLOv5s algorithm, including:

[0014] Segment each sample image in the labeled image dataset into a plurality of sub-images;

[0015] Pre-placing two bounding boxes in each sub-image, and predicting the coordinates, category, and confidence of each bounding box in the sub-image according to the YOLOv5s algorithm;

[0016] Based on the non-maximum suppression algorithm, the coordinates, category, and confidence of each bounding box are obtained to obtain a locally unique prediction box.

[0017] More preferably, the network structure of the production line sample recognition model includes a backbone network, a neck network and a head network, wherein:

[0018] The backbone network is used to calculate the feature information of the image targets in the labeled image dataset in different convolutional layers through convolution and generate corresponding sample feature maps;

[0019] The neck network includes an FPN module and a PANet module. The neck network performs split-path fusion on multiple feature maps output by the backbone network, and obtains fusion features of three different scales using three fusion routes respectively.

[0020] The head network consists of three convolution modules, which perform image target detection on the fused features of three different scales respectively, and obtain the intersection of the predicted box and the real box according to the loss function.

[0021] More preferably, the training set and the test set divided by the annotated image data set are input into the initial production line sample recognition model for training, specifically including:

[0022] Inputting the sample image in the training set to the input end of the initial production line sample recognition model to fill the image edge of the sample image to obtain a filled sample image;

[0023] Inputting the filled sample image into the backbone network so that the backbone network performs feature processing on the filled sample image and generates a sample feature map corresponding to the filled sample image;

[0024] Inputting the sample feature map into the neck network, fusing the feature maps of different dimensions, and obtaining fused features of three different scales;

[0025] Inputting the fused features of the three different scales into the head network, performing image target detection on the fused features of the three different scales respectively, and obtaining the detection target of the initial production line sample recognition model;

[0026] Comparing the detection target with the label set to obtain a loss function that drives the initial production line sample recognition model to update parameters;

[0027] When the rate of change of the loss function value is greater than a preset threshold, the structural parameters in the initial production line sample recognition model training process are adjusted until the rate of change of the loss function value is less than or equal to the preset threshold, and the model training is terminated.

[0028] More preferably, the loss function is expressed as:

[0029]

[0030] Among them, L GIoU represents the loss function, A represents the predicted box area, B represents the real box area, and C represents the area of ​​the minimum enclosing rectangle of the predicted box area and the real box area.

[0031] More preferably, the method further comprises:

[0032] When the production line sample recognition model recognizes an abnormal part, a control signal is sent to the production equipment, wherein the control signal includes a secondary processing signal, a scrap processing signal, and a continue operation signal.

[0033] In a second aspect of the present application, there is provided an intelligent stamping manufacturing system based on machine learning, the intelligent stamping manufacturing system comprising an image acquisition module, a model training module and an abnormality recognition module, wherein:

[0034] The image acquisition module is used to collect sample image data sets and real-time production videos of part samples on the production line, wherein the sample image data sets include normal sample images and abnormal sample images;

[0035] The model training module is used to preprocess the sample image data set to obtain a labeled image data set, build an initial production line sample recognition model based on a convolutional neural network, and input the training set and the test set divided by the labeled image data set into the initial production line sample recognition model for training to obtain a production line sample recognition model;

[0036] The abnormality identification module is used to input the real-time production video into the production line sample identification model for detection to identify abnormal parts on the production line.

[0037] In a third aspect of the present application, an electronic device is provided, comprising a processor, a memory, a user interface and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory.

[0038] In a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program is executed by a processor to implement the steps of an intelligent stamping and forming manufacturing method based on machine learning.

[0039] The intelligent stamping manufacturing method and system based on machine learning provided by the present invention have the following beneficial effects compared with the prior art:

[0040] (1) Through the model built based on the YOLOv5s algorithm and combined with the training of normal and abnormal samples, it can accurately identify abnormal parts on the production line and improve the quality control accuracy of part samples. In addition, the continuous detection of real-time production videos realizes real-time monitoring of the production process and can timely detect and handle abnormal situations. By learning a large number of sample images, the production line sample recognition model can adapt to different types of parts and various possible abnormal situations, and has strong versatility and adaptability. The real-time production video and production line sample recognition model are collected for detection, which realizes the automatic identification of abnormal parts on the production line and reduces the need for manual detection.

[0041] (2) By filling the edges of the sample images, the consistency and integrity of the input data are ensured, which helps to improve the robustness and generalization ability of the model. At the same time, the backbone network can effectively extract the multi-level features of the image through feature processing, providing rich feature information for subsequent target detection. Based on the neck network, feature maps of different dimensions are fused and three fused features of different scales are generated, which helps the model to better handle targets of different sizes and proportions. The head network performs target detection on the fused features of three different scales, which improves the detection ability of the production line sample recognition model for targets of different sizes and enhances the adaptability of the production line sample recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A schematic diagram of the steps of an intelligent stamping forming manufacturing method based on machine learning provided by the present invention;

[0044] Figure 2 A schematic diagram of the training process of the production line sample recognition model provided by the present invention;

[0045] Figure 3 A schematic diagram of the structure of the air conditioning system optimization control device provided by the present invention;

[0046] Figure 4 A schematic diagram of the framework of the production line sample recognition model provided by the present invention;

[0047] Figure 5 This is a schematic structural diagram of an electronic device provided by the present invention.

[0048] Explanation of the accompanying drawings: 1. Intelligent stamping manufacturing system; 11. Image acquisition module; 12. Model training module; 13. Abnormal recognition module; 2. Electronic device; 21. Processor; 22. Communication bus; 23. User interface; 24. Network interface; 25. Memory. DETAILED DESCRIPTION

[0049] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] The present invention discloses an intelligent stamping forming manufacturing method based on machine learning, referring to Figure 1 The method includes steps S1 to S4.

[0051] Step S1, collecting a sample image data set and a real-time production video of part samples on a production line, wherein the sample image data set includes normal sample images and abnormal sample images.

[0052] In this step, install high-resolution, high-frame-rate industrial cameras at key locations on the production line and ensure that the cameras are facing the production equipment to obtain clear images of the parts. Set up high-speed storage devices to process large amounts of image and video data, establish a data management system to classify, mark, and store the collected samples, and integrate the camera system with the production line control system to ensure that image acquisition is synchronized with the production process. When the production line is running normally, use industrial cameras to capture normal morphological images of parts, and ensure that a sufficient number of normal samples are collected to cover various normal production situations. Simulate or record various defects that may occur in the production process, and use the same camera system to capture abnormal part images. If the number of abnormal samples is insufficient, consider using data enhancement technology. Set the industrial camera to continuous video acquisition mode to capture the part flow process on the production line in real time. Set up an automatic trigger system to increase the sampling frequency when possible anomalies are detected.

[0053] Step S2: preprocess the sample image dataset to obtain a labeled image dataset.

[0054] This step also includes steps S21 to S22.

[0055] Step S21 , performing enhancement processing on the normal sample image and the abnormal sample image respectively to obtain enhanced images, wherein the enhancement processing includes adding strong light, shadow and vapor mist effects to the normal sample image and the abnormal sample image.

[0056] In this step, since the sample data volume of abnormal morphology is small and the production environment state is single, it is necessary to enhance the collected images. By adding effects such as strong light, shadow, and steam fog, the data set can be expanded and the robustness of the model can be improved. Therefore, Python image processing libraries such as OpenCV or Pillow (PIL) can be used to implement enhancement processing. The strong light effect can be achieved by increasing the brightness of the image, or by adjusting the V channel in the HSV (hue, saturation, brightness) space of the image, or by using an addition operation to add a constant value to each pixel. The shadow effect can be achieved by reducing the brightness in certain areas of the image, or by creating a gradient mask and then applying it to the original image, or by using a subtraction operation to subtract a constant from the pixel values ​​in certain areas. The steam fog effect can be achieved by simulating the effect of fog or steam, or by adding Gaussian noise and performing Gaussian blur, or by creating a translucent white mask layer and then mixing it with the original image.

[0057] Step S22, using LabelImg software to label defective parts of the enhanced image, obtain a data set for training the initial production line sample recognition model, and divide the data set into a training set and a test set in a ratio of 8:2.

[0058] In this step, start the LabelImg software, create a new project or open an existing project, import the enhanced image into the software, define the label category for the project, such as "normal", "defect A", "defect B", etc., set it according to the actual situation, and check each image one by one. For abnormal samples, carefully mark the defective parts, draw bounding boxes or polygons to accurately locate the defective areas, and for normal samples, you can mark them as "normal" as a whole, or do not make special labels. Select the data format of the YOLOv5s algorithm for export, usually in txt format, each image corresponds to a txt file, containing label information and bounding box coordinates, and ensure that the exported annotation file matches the corresponding image file name. Put the image file in the images folder and the annotation file in the labels folder. Use a script or manual method to randomly divide the data set in an 8:2 ratio, move 80% of the images and their corresponding annotation files to the train folder, and move the remaining 20% ​​of the images and their corresponding annotation files to the val folder. Create a YAML format configuration file (for example, data.yaml). Specify the paths, number of categories and category names of the training and validation sets, and other necessary configuration information in the configuration file.

[0059] Step S3, constructing an initial production line sample recognition model based on the YOLOv5s algorithm, inputting the training set and the test set divided by the annotated image data set into the initial production line sample recognition model for training to obtain the production line sample recognition model.

[0060] In this embodiment, deep learning technology is used to build a production line sample recognition model using the YOLOv5s algorithm. YOLOv5s divides the image into several S*S blocks, places two bounding boxes in each grid, predicts the coordinates, category and confidence of each bounding box through a convolutional neural network, and then obtains a local unique prediction box through non-maximum suppression.

[0061] This step includes steps S31 to S33.

[0062] Step S31, dividing each sample image in the labeled image data set into a plurality of sub-images.

[0063] In this step, the size of the sub-image is determined (for example, 640x640 pixels, which is a common input size for YOLOv5), and the size of the overlapping area is determined. The entire image is traversed using a sliding window technique, and the image is cut according to the defined sub-image size and stride. For areas on the edge of the image that may not be large enough to form a complete sub-image, they can be filled or ignored. A unique identifier is generated for each sub-image and the sub-image is saved to a specified directory.

[0064] Step S32, pre-place two bounding boxes in each sub-image, and predict the coordinates, category, and confidence of each bounding box in the sub-image according to the YOLOv5s algorithm.

[0065] Step S33, based on the non-maximum suppression algorithm, the coordinates, category and confidence of each bounding box, obtain a locally unique prediction box.

[0066] In this step, the network structure of the production line sample recognition model includes a backbone network, a neck network, and a head network, where:

[0067] The backbone network is used to calculate the feature information of image targets in the labeled image dataset in different convolutional layers through convolution and generate the corresponding sample feature maps.

[0068] In this embodiment, the backbone network calculates the feature information of the image target in different convolutional layers through convolution. Since the input image resolution is high, in order to avoid excessive data volume occupying a large amount of memory, the backbone network needs to downsample the image to reduce the number of parameters. The downsampling method is to perform a Focus operation on the image. The Focus module will copy the input image of 640×640×3 to 4 copies and then slice it. By sampling and splicing alternate columns, the two-dimensional plane information is converted into channel information. While ensuring the integrity of the image features, the number of input channels is deepened, and the network's receptive field is expanded. Finally, the size of the image is adjusted through 1×1 convolution to achieve the purpose of obtaining image features.

[0069] The neck network includes the FPN module and the PANet module. The neck network performs branch fusion on multiple feature maps output by the backbone network and obtains fusion features of three different scales using three fusion routes.

[0070] In this embodiment, the neck network fuses the feature information extracted from the neck network features by upsampling to obtain richer feature information. The feature fusion network mainly uses the Feature Pyramid Network (FPN) and PAN (Path Aggregation Network) network to upsample and downsample the convolutional features of different feature layers output in the backbone feature extraction network to change the scale of the feature layer, and fuses them with the feature layer of the same scale in the upper network to output a multi-scale feature map. The FPN+PAN structure realizes bidirectional fusion from top to bottom and from bottom to top, which can enhance the recognition ability of the model and has better detection performance for targets of different sizes.

[0071] The head network consists of three convolution modules, which perform image target detection on the fused features of three different scales respectively, and obtain the intersection of the predicted box and the real box according to the loss function.

[0072] In this embodiment, the head network is a classification regression network, which is used to output the category information and position information of the target. In order to better judge the intersection of the prediction box and the ground truth box, and considering that the prediction box and the ground truth box may have position deviations, the YOLOv5s algorithm uses GIoU as the loss function to obtain the intersection of the prediction box and the ground truth box.

[0073] In one example, if Figure 2 As shown in the figure, the SPP module in the backbone network is replaced by SPPF. Compared with the SPP module, the SPPF module has faster detection speed and higher efficiency, and can solve the multi-scale problem to a certain extent. Adding the CBAM attention mechanism before the SPPF module of the backbone network can strengthen the feature information of the important feature layer to a certain extent and enhance the detection ability of the network. Replacing Conv in the backbone network and the neck network with DCNv2 (deformable convolution) can adaptively adjust the sampling position of the convolution kernel to make it more suitable for the actual shape of the target object, thereby improving the accuracy of feature extraction. Replacing C3 in the backbone network and the neck network with C3Ghost can effectively reduce the number of parameters and calculation amount of the model while maintaining high accuracy. By setting the training rounds of the model, it can continuously learn different sample features in the production process until the best mAP@0.5 and FPS are achieved.

[0074] This step also includes steps S34 to S39.

[0075] Step S34, inputting the sample image in the training set to the input end of the initial production line sample recognition model to fill the image edge of the sample image to obtain a filled sample image.

[0076] Step S35, inputting the filled sample image into the backbone network, so that the backbone network performs feature processing on the filled sample image and generates a sample feature map corresponding to the filled sample image.

[0077] In this step, if Figure 3 As shown in the figure, the sample image after input padding is passed through the initial convolution layer for basic feature extraction, and the feature map is passed through multiple C3Ghost modules to gradually extract more advanced and abstract features. Each C3Ghost module uses a Ghost convolution structure to effectively reduce the amount of calculation, and uses deformable convolution at key locations to enhance the network's adaptability to the target shape. The feature map passes through the CBAM attention mechanism to highlight important spatial and channel information. Finally, the feature map passes through the SPPF module for multi-scale feature fusion to obtain the final sample feature map.

[0078] Step S36, input the sample feature map into the neck network, fuse the feature maps of different dimensions, and obtain fused features of three different scales.

[0079] Step S37, inputting the fused features of three different scales into the head network, performing image target detection on the fused features of three different scales respectively, and obtaining the detection target of the initial production line sample recognition model.

[0080] Step S38, comparing the detection target with the label set to obtain a loss function that drives the initial production line sample recognition model to update parameters.

[0081] Step S39, when the rate of change of the loss function value is greater than a preset threshold, adjust the structural parameters in the initial production line sample recognition model training process until the rate of change of the loss function value is less than or equal to the preset threshold, and then end the model training.

[0082] The expression of the loss function is:

[0083]

[0084] Among them, L GIoU represents the loss function, A represents the predicted box area, B represents the real box area, and C represents the area of ​​the minimum enclosing rectangle of the predicted box area and the real box area.

[0085] Step S4, inputting the real-time production video into the production line sample recognition model for detection to identify abnormal parts on the production line.

[0086] In this step, when the production line sample recognition model identifies an abnormal part, a control signal is sent to the production equipment, wherein the control signal includes a secondary processing signal, a scrap processing signal, and a continue operation signal. That is, during real-time detection, if an abnormal state of a part occurs in the production process, the abnormal part of the production process is identified according to the different characteristic signs of the production process learned in the production line sample recognition model, and the result of the model prediction is framed in the real-time screen, and the signal is sent to the production equipment at the same time.

[0087] In one example, production equipment monitors and collects data in real time through sensors installed in key locations, such as pressure sensors, displacement sensors, temperature sensors, and flow sensors, to detect key parameters such as pressure, slider displacement, temperature, and pipeline flow during the operation of the production equipment; the PLC receives the sensor signal and performs necessary signal conditioning and conversion; based on the collected signal and the preset control strategy, the control system analyzes and processes the data collected by the sensor, allowing the PLC control system to determine whether to proceed to the next process or move to the processing area for secondary processing or scrapping, and generate control instructions.

[0088] The control system adjusts the execution structure to scrap and reprocess the blanks according to the control instructions obtained by the algorithm (such as the opening of the proportional servo valve, and then controls the speed of the hydraulic cylinder's working slide to achieve accurate manufacturing of the blanks). It also performs closed-loop control through PLC to ensure the stability and accuracy of the production process.

[0089] By building a model based on the YOLOv5s algorithm and combining the training of normal and abnormal samples, it is possible to accurately identify abnormal parts on the production line and improve the quality control accuracy of part samples. In addition, continuous detection of real-time production videos enables real-time monitoring of the production process and can promptly detect and handle abnormal situations. By learning from a large number of sample images, the production line sample recognition model can adapt to different types of parts and various possible abnormal situations, and has strong versatility and adaptability. Real-time production videos and production line sample recognition models are collected for detection, which realizes the automatic identification of abnormal parts on the production line and reduces the need for manual detection.

[0090] Based on the above method, the embodiment of the present application discloses an intelligent stamping forming manufacturing system based on machine learning, referring to Figure 3 The intelligent stamping manufacturing system 1 includes an image acquisition module 11, a model training module 12 and an abnormality recognition module 13, wherein:

[0091] The image acquisition module 11 is used to collect sample image data sets and real-time production videos of part samples on the production line, wherein the sample image data sets include normal sample images and abnormal sample images;

[0092] The model training module 12 is used to preprocess the sample image data set to obtain the labeled image data set, build an initial production line sample recognition model based on a convolutional neural network, and input the training set and test set divided by the labeled image data set into the initial production line sample recognition model for training to obtain a production line sample recognition model;

[0093] The abnormality identification module 13 is used to input the real-time production video into the production line sample identification model for detection to identify abnormal parts on the production line.

[0094] In one example, the model training module 12 is used to perform enhancement processing on normal sample images and abnormal sample images respectively to obtain enhanced images, wherein the enhancement processing includes adding strong light, shadow and mist effects to the normal sample images and the abnormal sample images; using LabelImg software to label defective parts of the enhanced images to obtain a data set for training the initial production line sample recognition model, and dividing the data set into a training set and a test set in a ratio of 8:2.

[0095] In one example, the model training module 12 is used to segment each sample image in the labeled image dataset into multiple sub-images; pre-place two bounding boxes in each sub-image, and predict the coordinates, category and confidence of each bounding box in the sub-image according to the YOLOv5s algorithm; based on the non-maximum suppression algorithm, the coordinates, category and confidence of each bounding box, obtain a locally unique prediction box.

[0096] In one example, the network structure of the production line sample recognition model includes a backbone network, a neck network, and a head network, wherein:

[0097] The backbone network is used to calculate the feature information of the image targets in the labeled image dataset in different convolutional layers through convolution and generate the corresponding sample feature maps;

[0098] The neck network includes the FPN module and the PANet module. The neck network performs split fusion on multiple feature maps output by the backbone network and obtains fusion features of three different scales using three fusion routes.

[0099] The head network consists of three convolution modules, which perform image target detection on the fused features of three different scales respectively, and obtain the intersection of the predicted box and the real box according to the loss function.

[0100] In one example, the model training module 12 is used to input the sample images in the training set into the input end of the initial production line sample recognition model to fill the image edges of the sample images and obtain the filled sample images; input the filled sample images into the backbone network so that the backbone network performs feature processing on the filled sample images and generates a sample feature map corresponding to the filled sample images; input the sample feature map into the neck network, fuse the feature maps of different dimensions, and obtain fused features of three different scales; input the fused features of three different scales into the head network, perform image target detection on the fused features of three different scales respectively, and obtain the detection target of the initial production line sample recognition model; compare the detection target with the label set to obtain the loss function that drives the initial production line sample recognition model to update the parameters; when the rate of change of the loss function value is greater than the preset threshold, adjust the structural parameters in the initial production line sample recognition model training process until the rate of change of the loss function value is less than or equal to the preset threshold, and then end the model training.

[0101] In one example, the loss function is expressed as:

[0102]

[0103] Among them, L GIoU represents the loss function, A represents the predicted box area, B represents the real box area, and C represents the area of ​​the minimum enclosing rectangle of the predicted box area and the real box area.

[0104] In one example, when the production line sample recognition model recognizes an abnormal part, a control signal is sent to the production equipment, wherein the control signal includes a secondary processing signal, a scrap processing signal, and a continue operation signal.

[0105] See also Figure 4 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown, the electronic device 2 may include: at least one processor 21 , at least one network interface 24 , a user interface 23 , a memory 25 , and at least one communication bus 22 .

[0106] The communication bus 22 is used to realize the connection and communication between these components.

[0107] The user interface 23 may include a display screen (Display) and a camera (Camera), and the optional user interface 23 may also include a standard wired interface and a wireless interface.

[0108] The network interface 24 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0109] Among them, the processor 21 may include one or more processing cores. The processor 21 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 25, and calling data stored in the memory 25. Optionally, the processor 21 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 21 can integrate one or more combinations of a central processing unit (Central Processing Unit, CPU), a graphics processor (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 21, and it can be implemented separately through a chip.

[0110] Among them, the memory 25 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 25 includes a non-transitory computer-readable storage medium. The memory 25 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 25 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store data involved in the above-mentioned method embodiments, etc. The memory 25 may also be optionally at least one storage device located away from the aforementioned processor 21. As Figure 4 As shown, the memory 25 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program for an intelligent stamping manufacturing method based on machine learning.

[0111] exist Figure 5In the electronic device 2 shown, the user interface 23 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 21 can be used to call an application program stored in the memory 25 for an intelligent stamping forming manufacturing method based on machine learning. When executed by one or more processors, the electronic device executes one or more methods in the above-mentioned embodiments.

[0112] A computer-readable storage medium stores instructions, which, when executed by one or more processors, enable the computer to execute one or more methods in the above-mentioned embodiments.

[0113] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0114] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0115] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0116] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0117] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0118] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0119] The above are only exemplary embodiments of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, it will be easy for those skilled in the art to think of other embodiments of the present disclosure. This application is intended to cover any variation, use or adaptive change of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the technical field that are not recorded in the present disclosure.

[0120] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An intelligent stamping manufacturing method based on machine learning, characterized in that: The method comprises: Collecting a sample image data set and real-time production video of part samples on a production line, wherein the sample image data set includes normal sample images and abnormal sample images; Preprocessing the sample image dataset to obtain a labeled image dataset; Building an initial production line sample recognition model based on the YOLOv5s algorithm, inputting the training set and the test set divided by the labeled image data set into the initial production line sample recognition model for training, so as to obtain a production line sample recognition model; The real-time production video is input into the production line sample recognition model for detection to identify abnormal parts on the production line.

2. The method according to claim 1, characterized in that The preprocessing of the sample image dataset to obtain the labeled image dataset specifically includes: Performing enhancement processing on the normal sample image and the abnormal sample image respectively to obtain enhanced images, wherein the enhancement processing includes adding strong light, shadow and steam mist effects to the normal sample image and the abnormal sample image; LabelImg software is used to annotate defective parts of the enhanced image to obtain a data set for training the initial production line sample recognition model, and the data set is divided into a training set and a test set in a ratio of 8:

2.

3. The method according to claim 1, characterized in that The initial production line sample recognition model is constructed based on the YOLOv5s algorithm, including: Segment each sample image in the labeled image dataset into a plurality of sub-images; Pre-placing two bounding boxes in each sub-image, and predicting the coordinates, category, and confidence of each bounding box in the sub-image according to the YOLOv5s algorithm; Based on the non-maximum suppression algorithm, the coordinates, category, and confidence of each bounding box are obtained to obtain a locally unique prediction box.

4. The method according to claim 1, characterized in that The network structure of the production line sample recognition model includes a backbone network, a neck network and a head network, wherein: The backbone network is used to calculate the feature information of the image targets in the labeled image dataset in different convolutional layers through convolution, and generate corresponding sample feature maps; The neck network includes an FPN module and a PANet module. The neck network performs split-path fusion on multiple feature maps output by the backbone network, and obtains fusion features of three different scales using three fusion routes respectively. The head network consists of three convolution modules, which perform image target detection on the fused features of three different scales respectively, and obtain the intersection of the predicted box and the real box according to the loss function.

5. The method according to claim 4, characterized in that Inputting the training set and the test set divided by the labeled image data set into the initial production line sample recognition model for training, specifically including: Inputting the sample image in the training set to the input end of the initial production line sample recognition model to fill the image edge of the sample image to obtain a filled sample image; Inputting the filled sample image into the backbone network so that the backbone network performs feature processing on the filled sample image and generates a sample feature map corresponding to the filled sample image; Inputting the sample feature map into the neck network, fusing the feature maps of different dimensions, and obtaining fused features of three different scales; Inputting the fused features of the three different scales into the head network, performing image target detection on the fused features of the three different scales respectively, and obtaining the detection target of the initial production line sample recognition model; Comparing the detection target with the label set to obtain a loss function that drives the initial production line sample recognition model to update parameters; When the rate of change of the loss function value is greater than a preset threshold, the structural parameters in the initial production line sample recognition model training process are adjusted until the rate of change of the loss function value is less than or equal to the preset threshold, and the model training is terminated.

6. The method according to claim 5, characterized in that The expression of the loss function is: Among them, L GIoU represents the loss function, A represents the predicted box area, B represents the real box area, and C represents the area of ​​the minimum enclosing rectangle of the predicted box area and the real box area.

7. The method according to claim 1, characterized in that The method further comprises: When the production line sample recognition model recognizes an abnormal part, a control signal is sent to the production equipment, wherein the control signal includes a secondary processing signal, a scrap processing signal, and a continue operation signal.

8. An intelligent stamping manufacturing system based on machine learning, characterized in that: The intelligent stamping manufacturing system (1) comprises an image acquisition module (11), a model training module (12) and an abnormality recognition module (13), wherein: The image acquisition module (11) is used to acquire a sample image data set and real-time production video of part samples on a production line, wherein the sample image data set includes normal sample images and abnormal sample images; The model training module (12) is used to pre-process the sample image data set to obtain a labeled image data set, construct an initial production line sample recognition model based on a convolutional neural network, and input the training set and the test set divided by the labeled image data set into the initial production line sample recognition model for training to obtain a production line sample recognition model; The abnormality identification module (13) is used to input the real-time production video into the production line sample identification model for detection, so as to identify abnormal parts on the production line.

9. An electronic device, characterized in that: The electronic device (2) comprises a processor (21), a memory (25), a user interface (23) and a network interface (24), wherein the memory (25) is used to store instructions, the user interface (23) and the network interface (24) are used to communicate with other devices, and the processor (21) is used to execute the instructions stored in the memory (25) so that the electronic device (2) executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images