A method and device for correcting the linear deviation of copper bonding machine strip based on deep learning
The deep learning-based method for copper laminating machines uses CNN and RNN for precise material strip deviation correction, addressing accuracy and stability issues, improving production efficiency and reliability.
Patent Information
- Application Number
- CN202411125771.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-08-16
AI Technical Summary
Existing copper laminating machines suffer from insufficient correction accuracy and stability due to reliance on mechanical or visual detection systems, which struggle to identify complex features and dynamic changes in material strips, leading to inaccurate corrections and reduced production efficiency.
A deep learning-based method using a convolutional neural network (CNN) for spatial feature extraction and a recurrent neural network (RNN) with temporal attention for real-time image analysis, combined with a mechanical correction mechanism, to accurately correct material strip deviations.
The method significantly improves correction accuracy and stability, enabling rapid, automated, and intelligent deviation corrections, enhancing production efficiency and reliability while maintaining equipment versatility and user-friendliness.
Smart Images

Figure CN118651707B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of copper laminating machines, and in particular to a method and a device for correcting the linear deviation of a material strip of a copper laminating machine based on deep learning. Background Art
[0002] At present, the fully automatic roll-to-roll double-sided copper laminating machine is a device dedicated to the production of multi-layer circuit boards. It can continuously attach copper foil to copper clad laminates or prepregs to form a multi-layer circuit board structure.
[0003] In the multi-layer board gluing process of the fully automatic roll-to-roll double-sided copper laminating machine, the material strip is prone to deviation during the conveying process. The current material strip deviation correction methods used by the fully automatic roll-to-roll double-sided copper laminating machine mainly include the following:
[0004] Photoelectric correction system: The edge position of the material strip is monitored in real time through a photoelectric sensor (such as an infrared or laser sensor). When the material strip is detected to be offset, the sensor signal is fed back to the control system, which controls the correction device to automatically adjust the position of the guide roller and guide the material strip back to the center position. This method has a fast response speed and high accuracy.
[0005] Mechanical deviation correction device: includes components such as linkage-type automatic deviation adjustment device and anti-deviation cleaning rollers. The linkage-type device has a relatively complex structure and realizes deviation correction through a series of linkage mechanisms. The anti-deviation cleaning rollers are rollers installed on the U-shaped conveyor belt. They use their surface structure or rotation direction to help the material belt remain centered, thereby realizing the deviation correction function.
[0006] However, the existing correction methods and correction devices have the following disadvantages:
[0007] 1. Insufficient correction accuracy and stability: Traditional correction devices rely on simple mechanical or visual detection systems, which are difficult to accurately identify the various complex features and dynamic changes of the material strip, resulting in inaccurate correction and affecting production quality.
[0008] 2. The existing non-deep learning correction system is insufficient in image processing speed and automated decision-making. The analysis and response time is long, and it is impossible to achieve rapid correction, which affects production efficiency. The degree of automation is low, requiring more manual monitoring and manual intervention, which reduces the reliability and efficiency of continuous operations.
[0009] Therefore, there is a need for a deep learning-based copper laminating machine material strip linear correction method and correction device that can solve the above problems. Summary of the invention
[0010] The present invention provides a method and device for correcting the linear deviation of the material strip of a copper plating machine based on deep learning. The present invention uses deep learning technology, especially combines the spatial feature extraction capability of a convolutional neural network and the timing analysis capability of a recurrent neural network, to perform a refined analysis of the real-time image of the material strip, capture more complex features such as edges and textures, and the dynamic changes of the material strip during the conveying process, and significantly improves the accuracy and stability of the correction.
[0011] The technical solution adopted by the present invention to solve the above technical problems is:
[0012] A method for correcting the straight line deviation of a copper bonding machine strip based on deep learning, comprising the following steps:
[0013] Step S1: Install the detection mechanism, install the sensor in the detection mechanism on the sensor bracket, and use the sensor to collect the image of the material belt in the conveying process in real time;
[0014] Step S2: Use a convolutional neural network to extract spatial features of the strip image collected by the sensor to obtain feature information of each frame of the image;
[0015] Step S3: Arrange the feature information vectors of each frame of image in chronological order to form an image feature sequence;
[0016] Step S4: input the image feature sequence into the recurrent neural network to capture the timing information of the material strip;
[0017] Step S5: Design a temporal attention mechanism and integrate it into the recurrent neural network;
[0018] Step S6: constructing a deep learning model integrating a convolutional neural network and a recurrent neural network;
[0019] Step S7: Use the labeled data set to train and verify the deep learning model, and verify the effectiveness and generalization ability of the deep learning model through the cross-validation method;
[0020] Step S8: Integrate the trained and verified deep learning model into the copper laminating machine control system, and perform real-time deviation correction operations through an electrically connected deviation correction mechanism.
[0021] Furthermore, in step S1, the sensor is mounted on the sensor bracket, and real-time acquisition of the image of the material belt during the conveying process by the sensor includes:
[0022] Step S1-1: Determine the installation position of the sensor according to the material belt transmission path;
[0023] Step S1-2: Install the sensor on the sensor bracket and configure the parameters of the sensor, including setting the sampling frequency and exposure parameters of the sensor.
[0024] Furthermore, the step S2 uses a convolutional neural network model to extract spatial features from the strip image collected by the sensor, and obtains feature information of each frame of the image including:
[0025] Step S2-1: pre-processing the strip image collected by the sensor;
[0026] Step S2-2: construct a convolutional neural network model;
[0027] Step S2-3: extracting spatial features of the strip image through a convolutional neural network model;
[0028] Step S2-4: vectorize the extracted spatial features.
[0029] Furthermore, the step S2-1 of preprocessing the strip image collected by the sensor includes:
[0030] Step S2-1-1: adjusting the original image collected by the sensor to the standard size required by the convolutional neural network;
[0031] Step S2-1-2: converting the adjusted image into a grayscale image;
[0032] Step S2-1-3: Normalize the pixel values of the grayscale image.
[0033] Furthermore, the step S2-2 of constructing a convolutional neural network model includes:
[0034] Step S2-2-1: Determine the size of the input layer according to the image size;
[0035] Step S2-2-2: design a multi-layer convolution layer, each layer containing multiple convolution kernels, for learning image features, which include edge features, texture features, contrast features, straight line features, shape features, motion direction features and time series features;
[0036] Step S2-2-3: Calculate the image features through the convolution layer and output a new set of feature maps;
[0037] Step S2-2-4: Perform a pooling operation on the new feature map to reduce the dimension of the feature map and reduce the amount of calculation;
[0038] Step S2-2-5: Compile the convolutional neural network model in the deep learning framework, specify the optimizer, loss function and evaluation index;
[0039] Step S2-2-6: Prepare training data sets and validation data sets to train the convolutional neural network model. During the training process, the model parameters are updated through back propagation and gradient descent.
[0040] Furthermore, the step S5 designs a temporal attention mechanism and combines it into a recurrent neural network, including:
[0041] Step S5-1: Design a function to calculate the attention score for each time step, comparing the hidden state h_t of the current time step with the hidden state of the previous time step or the mean of the entire time series, using dot product and weighted sum;
[0042] Step S5-2: Use the softmax function to convert the calculated raw attention score into a probability distribution, ensuring that the sum of the weights of all time steps is 1, thus forming a valid attention weight vector;
[0043] Step S5-3: For each time step t, the attention weight α_t of the time step is calculated according to the designed attention score function;
[0044] Step S5-4: Using the calculated attention weight α_t, perform weighted summation on the hidden states h_1, h_2, ..., h_T generated by the recurrent neural network at all time steps to generate a context vector c;
[0045] Step S5-5: Fuse the context vector c with the hidden state h_t of the current time step;
[0046] Step S5-6: Use a deep learning framework to write the corresponding code according to the designed attention mechanism structure;
[0047] Step S5-7: Integrate the code module with the attention mechanism into the recurrent neural network to ensure that the attention mechanism is applied at each time step or at the required level.
[0048] Furthermore, the deep learning model integrating the convolutional neural network model and the recurrent neural network is constructed in step S6, including:
[0049] Step S6-1: Use a deep learning framework to define convolutional neural networks and recurrent neural networks as reusable modules or layers;
[0050] Step S6-2: Using the output of the convolutional neural network as the input of the recurrent neural network, ensuring that the data flow seamlessly transitions from the convolutional neural network to the recurrent neural network directly as the input of the recurrent neural network;
[0051] Step S6-3: Select loss function and optimizer;
[0052] Step S6-4: compile the model in the framework, specify the loss function, optimizer and evaluation index, and complete the model definition;
[0053] A copper plating machine material strip linear deviation correction device based on deep learning, comprising a support frame, a conveying frame, a detection mechanism and a deviation correction mechanism, wherein the conveying frame is fixedly connected to the upper end of the support frame, a supporting guide roller is connected to the conveying frame, the deviation correction mechanism is installed on the conveying frame, the deviation correction mechanism and the detection mechanism are electrically connected, a control touch screen is installed on the support frame, and the control touch screen and the deviation correction mechanism are electrically connected;
[0054] The deviation correction mechanism includes a driving cylinder A, a fixed beam and a flexible pressure roller. The driving cylinder A is fixedly connected to the upper end of the conveying frame. A driving rod is driven and connected to the output end of the driving cylinder A. A connecting block is fixedly connected to the top of the driving rod. The fixed beam is fixedly connected between the two connecting blocks. Two driving cylinders B that are symmetrically arranged with each other are arranged at the bottom of the fixed beam. The driving cylinder B and the fixed beam are connected to each other through a guide rod. A spring is sleeved on the guide rod. One end of the spring is fixedly connected to the bottom of the fixed beam, and the other end of the spring is fixedly connected to the driving cylinder B. The flexible pressure roller is connected between the two driving cylinders B. The flexible pressure roller is used to correct the deviation of the material strip.
[0055] Furthermore, the detection mechanism includes a sensor, a sensor bracket and an adjustment component, the sensor is fixedly connected to the sensor bracket, the sensor bracket is fixedly connected to one side of the adjustment component, the adjustment component includes an adjustment sleeve, an adjustment column, an adjustment knob, a cross bar and a bracket, the cross bar is fixedly connected to the bracket, the adjustment sleeve is connected to the cross bar, one end of the adjustment column and the adjustment knob are interconnected, and the other end of the adjustment column passes through the adjustment sleeve and is fixedly connected to the sensor bracket.
[0056] Furthermore, the adjusting column and the adjusting sleeve are fixed to each other via a hand bolt A, and the cross bar and the adjusting sleeve are fixed to each other via a hand bolt B and a hand bolt C.
[0057] The advantages of the present invention are: the present invention provides a method for straight line deviation correction of copper bonding machine strip based on deep learning, which has the following advantages:
[0058] 1. Improve the accuracy and stability of deviation correction: The present invention uses deep learning technology, especially the spatial feature extraction capability of convolutional neural network (CNN) and the timing analysis capability of recurrent neural network (RNN), to perform refined analysis on the real-time image of the material belt, capture more complex features such as edges and textures, and the dynamic changes of the material belt during the conveying process, which significantly improves the accuracy and stability of deviation correction.
[0059] 2. The real-time image processing capability of the deep learning model combined with the present invention can complete image analysis and correction decisions within milliseconds. The deep learning model in the present invention can continuously improve the correction performance through continuous learning and model optimization. Once the model training is completed, it only needs to input real-time data for rapid response in actual application. The maintenance cost is low and the upgrade is convenient. In conjunction with the control system of the copper plating machine, it can realize automated and intelligent correction operations, greatly improving production efficiency and the reliability of continuous operation.
[0060] 3. The present invention utilizes a deep learning model, especially after the introduction of the temporal attention mechanism. This method can better understand and learn the characteristics of material strips of different materials, colors, and patterns. Even in the face of new scenes or slight changes, it can effectively correct deviations and show good generalization performance.
[0061] The present invention also provides a copper laminating machine strip linear deviation correction device based on deep learning, which has the following advantages:
[0062] 1. The present invention can quickly respond to the instructions of the control system and realize precise adjustment of the material belt through the driving cylinder A and the flexible pressure roller in the correction device. The design of the flexible pressure roller not only ensures the appropriate correction force, but also avoids damage to the material belt. It is suitable for material belts of various materials and improves the versatility and adaptability of the equipment.
[0063] 2. The present invention provides a highly adjustable sensor position through the sensor bracket and the adjustment component in the detection mechanism, so that the sensor can accurately capture the key area of the material belt and ensure the accuracy of the correction. The present invention facilitates quick on-site adjustment through the hand-tightened bolt design of the adjustment component, without the need for complex tools, thereby improving the convenience of maintenance.
[0064] 3. The present invention ensures the stability and durability of the entire correcting device in continuous operation through the structural design of the support frame, the conveying frame and the correcting mechanism. It can maintain a good working condition even in a high-intensity production environment, thereby extending the service life of the equipment.
[0065] 4. The present invention, through the integration of the control touch screen, can enable the operator to intuitively monitor the correction process, adjust the correction parameters in time, achieve the friendliness and efficiency of human-computer interaction, and further improve the automation management level of the overall system.
[0066] To sum up, the present invention combines deep learning technology with precise mechanical design, which not only greatly improves the accuracy and automation level of deviation correction, but also enhances the flexibility, stability and user-friendliness of the equipment. It is an important technological innovation in the copper plating machine manufacturing industry, and has a significant role in improving production efficiency, reducing costs and ensuring product quality. BRIEF DESCRIPTION OF THE DRAWINGS
[0067] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0068] Figure 1 It is a schematic diagram of the three-dimensional structure of the present invention;
[0069] Figure 2 It is a schematic diagram of the main structure of the present invention;
[0070] Figure 3 A flow chart of a method for linear deviation correction of a copper plating machine strip based on deep learning provided by the present invention.
[0071] Among them: 1. Material belt; 2. Flexible pressure roller; 3. Sensor;
[0072] 4. Support frame; 5. Control touch screen; 6. Conveyor frame;
[0073] 7. Driving cylinder A; 8. Fixed beam; 9. Driving rod;
[0074] 10. Connecting block; 11. Driving cylinder B; 12. Guide rod;
[0075] 13. Spring; 14. Sensor bracket; 15. Hand-tightening bolt A;
[0076] 16. Adjustment sleeve; 17. Hand-tightening bolt B; 18. Hand-tightening bolt C;
[0077] 19. Crossbar; 20. Bracket; 21. Support guide roller;
[0078] 22. Adjustment knob. 23. Adjustment column. DETAILED DESCRIPTION
[0079] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0080] Embodiment 1: Figure 3 A flow chart of a method for straight line deviation correction of a copper bonding machine strip based on deep learning provided by the present invention, such as Figure 3 A method for straight line deviation correction of a copper bonding machine strip based on deep learning is shown, characterized in that it includes the following steps:
[0081] Step S1: Install the sensor 3 on the sensor bracket 14, and collect the image of the material belt 1 in the conveying process in real time through the sensor 3, specifically, including:
[0082] It includes: Step S1-1: determining the installation position of the sensor 3 according to the transmission path of the material belt 1, and the installation position is selected at or near the key node where the material belt 1 is prone to deviation, to ensure that the sensor 3 can clearly capture the movement state of the material belt 1. Specifically, the sensor 3 uses any one of the CMOS image sensors 3 or CCD image sensors 3 to capture visual information, and can monitor the surface features of the material in real time. The sensor 3 can convert the received optical image into electrical signals, which can then be digitized and sent to a computer or processor for further image processing and analysis.
[0083] Step S1-2: Install the sensor 3 on the sensor bracket 14 to ensure that it will not be displaced due to vibration and other factors during operation. Adjust the field of view angle and focal length of the sensor 3 and calibrate the field of view of the sensor 3: Adjust the field of view angle, focal length and other parameters of the sensor 3 to ensure that its field of view covers the width of the strip 1 and the key offset area, and the image is clear and undistorted (multiple sets of sensors 3 can be used to monitor the strip 1 from different angles or cover different areas in all directions). Then configure the parameters of the sensor 3, including setting the sampling frequency and exposure parameters of the sensor 3, including:
[0084] Step S1-2-1: Determine the sampling frequency: According to the conveying speed of the material belt 1, the rate at which the offset may occur, and the requirements of the prediction model for data real-time performance, set the image acquisition frequency. The sampling frequency should be high enough to capture the subtle dynamic changes of the material belt 1, but not to cause data redundancy and excessive processing burden. For example, the speed of the material belt 1 during the conveying process directly affects the selection of the sampling frequency. Generally speaking, the sampling frequency should be at least twice the speed of the material belt 1 (satisfying the Nyquist theorem) to avoid image distortion or information loss due to insufficient sampling. For example, if the speed of the material belt 1 is 5m / s, in order to ensure image quality, the sampling frequency should be at least 10Hz; since the rate of offset of the material belt 1 determines how fast the sampling frequency is required to capture the dynamic changes of the offset, if the offset occurs very quickly, a higher sampling frequency is required to ensure that no important offset events are missed. For example, assuming that the maximum offset rate of the material belt 1 is 0.9 m / s, a sampling frequency of at least 1.8 Hz is required to capture the complete changes of each offset cycle. Since the deep learning prediction model has specific requirements for the continuity and real-time performance of the input data, the model requires a sequence of images with multiple continuous time steps to predict future deviation trends. Therefore, the sampling frequency should be able to meet the time step required by the model. For example, if the model needs to sample twice per second to form a time window, the sampling frequency should be set to 2Hz. Taking into account the hardware processing power and storage limitations, the sampling frequency should not be too high to avoid data processing bottlenecks or waste of storage resources. It is necessary to avoid data redundancy as much as possible while ensuring that key dynamic information is captured. For example, if the system processing power can only support the processing of 5 images per second, and the deviation rate of strip 1 is low, then the sampling frequency is set to 5Hz.
[0085] Step S1-2-2: Set exposure parameters: Optimize exposure time, gain, and aperture parameters to ensure that images with moderate contrast and rich details are obtained under current lighting conditions. For fast-moving strips 1, a short exposure time is required to prevent image blur; for low-light environments, the gain needs to be appropriately increased or infrared fill light needs to be used.
[0086] Step S1-2-3: Data encapsulation: compress and encode the image sequence in a certain format (including JPEG or PNG format), and attach metadata information such as acquisition timestamp and sensor 3ID to facilitate subsequent data transmission and model processing.
[0087] Through the above steps, the sensor 3 can effectively collect the image of the material strip 1 in real time, providing key visual input data for the subsequent prediction model and correction control. This process covers the complete logical link from hardware installation, parameter configuration to data processing and monitoring feedback, ensuring the accurate and timely acquisition and effective use of image data.
[0088] Step S2: Using a convolutional neural network to extract spatial features from the image of the strip 1 collected by the sensor 3 to obtain feature information of each frame of the image, including the following steps:
[0089] Step S2-1: pre-processing the image of the material strip 1 collected by the sensor 3, specifically, including:
[0090] Step S2-1-1: The original image collected by the sensor 3 is adjusted to the standard size required by the convolutional neural network. The adjustment method includes: first, according to the design requirements of the convolutional neural network model used, the target size of the input image is clarified, including the width, height and number of color channels (for example, for RGB images, the number of color channels is 3). If the aspect ratio of the original image does not match the target size, the image is first scaled proportionally to keep the image content undistorted. Specifically, this is achieved by calculating the scaling factor that maintains the aspect ratio of the original image, and then the width and height of the image are adjusted by this factor. Then, the center part of the scaled image is cropped to the target size by cropping. The above method is suitable for the situation where the center area of the image is the most important or the background information has little effect on the correction task. Then fill it, add a blank area (usually black or average color) at the edge of the image to expand the image to the target size. The filling strategy adopts uniform filling or uses filling methods such as mirroring and reflection to reduce edge effects so that its size completely matches the target size. The processed image is directly passed as input to the subsequent convolutional neural network for feature extraction.
[0091] Through the above steps, the consistency of image size is ensured, which provides standardized input for subsequent convolutional neural network processing and is an indispensable part of the deep learning image processing process.
[0092] Step S2-1-2: Convert the adjusted image into a grayscale image. Specifically, we first need to understand that color images are represented by the RGB (red, green, and blue) color model, and each pixel consists of three components, representing the intensity of the three colors red, green, and blue, respectively. A grayscale image is a single-channel image. Each pixel has only one intensity value, representing the brightness of the point. The RGB value of each pixel is mapped to a single grayscale value through the weighted average method. For example, the RGB value of an image pixel is (R, G, B), which is converted to a grayscale value G_gray. The conversion process is as follows: Use the weighting coefficient: G_{gray} = 0.21R + 0.72G+ 0.07B, multiply the red, green, and blue components in the image by 0.21, 0.72, and 0.07, respectively, and then add them together to get the grayscale value (in actual programming, use Python's PIL library to directly call the built-in function to complete the conversion); after the conversion is completed, the original color image will be converted into an image with only black and white grayscale, and the color information of each pixel is replaced by a single brightness value, providing a more concise basis for subsequent image analysis.
[0093] In summary, the above steps convert the adjusted image into a grayscale image, which not only reduces the amount of data processing and improves the processing speed, but also helps to focus on the structure and brightness information of the image and ignore color differences in subsequent feature extraction and analysis.
[0094] Step S2-1-3: Normalize the pixel values of the grayscale image (between 0 and 1 to improve the stability and speed of model training). The purpose of normalization is to improve numerical stability. The deep learning model involves a large number of matrix operations during the training process. Using a smaller range of values (between 0 and 1) can reduce the risk of numerical overflow and improve the stability of the calculation. Normalization can accelerate the convergence of optimization algorithms such as gradient descent, because a smaller value range makes the gradient update smoother and more effective; for example, assuming that the original grayscale value of a pixel in a grayscale image is P, which ranges from 0 to 255, the normalization process is as follows: subtract the minimum grayscale value in the grayscale image from all pixel values to ensure that the converted range starts from 0, and divide the obtained pixel value by the maximum grayscale value in the grayscale image (usually 255), thereby mapping the pixel value to between 0 and 1. For an 8-bit grayscale image, the mathematical expression is: , where Pnormalized refers to the normalized pixel value, and the original pixel grayscale value P is converted to between 0 and 1 after step-by-step processing (subtracting the minimum value and dividing it by the range of the maximum value, or in certain cases directly dividing it by a maximum possible value, such as 255 in a grayscale image). This processing makes all pixel values on the same scale, which is convenient for further image processing and input of machine learning models. Through normalization, the consistency of subsequent model input is ensured, laying a good foundation for subsequent convolutional neural network feature extraction.
[0095] Step S2-2: construct a convolutional neural network model;
[0096] Step S2-3: extracting spatial features of the image of the material strip 1 through a convolutional neural network model;
[0097] Specifically, step S2-2 and step S2-3 include the following steps:
[0098] 1. Determine the size of the input layer according to the image size. For example, assuming that the image of strip 1 is 256x256 pixels, the input layer size is or 256x256 (grayscale image).
[0099] 2. Design multiple convolution layers, each layer contains multiple convolution kernels (filters) for learning image features, including edge features, texture features, contrast features, straight line features, shape features, motion direction features and timing features; edge features: edges are the boundaries that distinguish different material or color areas, which are crucial for identifying the boundary of the material strip 1. Clear edge features can help the system quickly locate the position deviation of the material strip 1 relative to the ideal path. Texture features refer to the texture differences on the surface of the material strip 1 (such as smoothness and pattern repeatability) that can form unique patterns in the image, which can be used to assist in judging the direction and offset of the material strip 1. Texture features are particularly effective when there is a clear texture contrast between the material strip 1 and the background or the contact surface of the guide rail. Contrast features: Effective features can be extracted using the brightness difference between the material strip 1 and its background. For example, when the color of the material strip 1 is in sharp contrast with the background, the position of the material strip 1 can be judged by analyzing the brightness distribution of the image. Straight line features: Ideally, the material strip 1 should travel along a straight line path. The straight line features in the image can be detected by methods such as Hough transform, and it can be evaluated whether the material strip 1 deviates from the preset straight line path. Shape features: Although the shape does not change much on the continuous strip 1, the consistency and integrity of the overall shape are also important for detecting whether the strip 1 is twisted or folded. The shape features can help identify the overall contour and deformation of the strip 1. Motion direction features: In a continuous image sequence, by analyzing the changes in the position of the strip 1 between adjacent frames and extracting the motion direction and speed features, the deviation trend of the strip 1 can be predicted and forward-looking correction control can be achieved. Timing features: When using a recurrent neural network (RNN) or other timing models to process a continuous image sequence, features can be extracted from the timing changes to capture the dynamic information of the movement of the strip 1, which is crucial for accurate correction.
[0100] 3. Calculate the image features through the convolution layer and output a new set of feature maps; specifically, the calculation process includes:
[0101] 4. Input and filter (convolution kernel) initialization: First, the convolution layer receives an input, usually a feature map of the previous layer (such as image data or the output of another convolution layer). At the same time, a set of filters (convolution kernels) are defined for the layer, each filter is a set of weight coefficients, which are optimized during the training process to capture specific types of features.
[0102] 5. Convolution operation: For each filter, slide it over the input feature map (also called convolution) and perform the following operations at each position:
[0103] 6. Element-wise multiplication and summation: Each element of the filter is element-wise multiplied with the element at the corresponding position on the input feature map, and then the product result is summed over all elements of the filter to obtain a scalar value. This process is repeated when the filter moves to the next position.
[0104] 7. Sliding step (Stride): The filter moves on the input feature map according to the specified step size. The step size determines the number of pixels that the filter crosses each time it moves.
[0105] 8. Padding: Add zero values (Zero-padding) to the edges of the input feature map to maintain the size of the output feature map or adjust the size as required.
[0106] 9. Generate feature maps: Through the above operations, each filter will generate a new two-dimensional feature map after traversing the entire input feature map, reflecting the response intensity distribution of the input under the features defined by the filter. Therefore, a convolutional layer will output multiple feature maps, one for each filter.
[0107] 10. Activation function: After the convolution operation, a nonlinear activation function (such as ReLU) is applied to each newly generated feature map to increase the expressive power of the network and enable it to learn more complex features.
[0108] The final output of the convolutional layer is a combination of feature maps generated by all filters, which together constitute the input of the next layer. Each feature map carries an abstract representation of the original input data at a specific spatial scale and feature.
[0109] In summary, the calculation process of the convolution layer includes the convolution operation of the filter and the input data, the optional padding operation, and the application of the activation function, which finally produces a new set of feature maps as output.
[0110] 11. Perform pooling operations (such as maximum pooling or average pooling) on the new feature map to reduce the dimension of the feature map and reduce the amount of calculation;
[0111] 12. Compile the convolutional neural network model in a deep learning framework (PyTorch or TensorFlow can be used), specify the optimizer, loss function, and evaluation metrics;
[0112] 13. Prepare training data sets and validation data sets to train the convolutional neural network model. During the training process, update the model parameters through back propagation and gradient descent. Specifically, select the output of a certain layer (such as the penultimate convolutional layer) as the feature vector.
[0113] Through the above steps, not only a convolutional neural network model is constructed, but also the spatial features of the image of the material strip 1 collected by the sensor 3 are effectively extracted. These features can then be used for further analysis, such as in a recurrent neural network model combined with a temporal attention mechanism.
[0114] Step S2-4: vectorize the extracted spatial features. Specifically, for example, there are N feature maps, each feature map has H×W pixels. For each feature map, its two-dimensional structure H×W is flattened into a one-dimensional vector, that is, the pixel values of H rows and W columns are arranged in order into a long vector. If there are multiple feature maps, each feature map is flattened separately, and then all the flattened vectors can be connected in series to form a final large vector. The vectors obtained after flattening multiple feature maps are connected in series to form a final feature vector. The final feature vector contains the key information of all feature maps and is a continuous one-dimensional structure, which is suitable for subsequent time series analysis. Through the above steps, the high-level spatial features of the image are effectively converted into a one-dimensional vector, which provides a suitable input format for the subsequent time series analysis or further deep learning model processing.
[0115] Step S3: Arrange the feature information vectors of each frame of image in chronological order to form an image feature sequence, which specifically includes:
[0116] 1. Timestamp Recording Recording time information: For each frame of the image, the timestamp of its acquisition is recorded at the same time. The timestamp ensures that the time point corresponding to each feature vector can be accurately known during subsequent processing, which is crucial for analyzing the temporal relationship in the image sequence.
[0117] 2. Sort feature vectors in chronological order: According to the timestamp of each frame, all feature vectors are sorted in chronological order. This step is the key to forming a time series, ensuring that the feature vectors in the sequence reflect the dynamic changes in the actual image sequence.
[0118] 3. Construct sequence structure: connect the sorted feature vectors in order to form a one-dimensional feature sequence. This sequence can be regarded as a list of vectors, in which each element is a feature vector at a time point.
[0119] 4. Data structure selection: To facilitate subsequent processing, store feature sequences as arrays, lists, or directly construct them into time series data structures (such as Pandas DataFrame or NumPy arrays, etc.), which can be indexed, sliced, and analyzed more efficiently.
[0120] 5. Check sequence coherence: After constructing the sequence, a check is performed to confirm that all expected image frames are correctly included, without omissions or duplications, and in the correct temporal order.
[0121] Through the above steps, the feature information of continuous images is organized into an orderly time series, laying a solid foundation for the next step of time series analysis. This time series not only retains the temporal and spatial information of the image, but also makes it possible to understand the dynamic behavior and trends in the image sequence;
[0122] Step S4: input the image feature sequence into the recurrent neural network to capture the timing information of the material strip 1;
[0123] Step S5: Design a temporal attention mechanism and integrate it into the recurrent neural network, specifically including:
[0124] Step S5-1: Design a function to calculate the attention score for each time step, comparing the hidden state h_t of the current time step (generated by the RNN) with the hidden state of the previous time step or the mean of the entire time series, using dot product and weighted sum;
[0125] Step S5-2: Use the softmax function to convert the calculated raw attention score into a probability distribution, ensuring that the sum of the weights of all time steps is 1, thus forming a valid attention weight vector;
[0126] Step S5-3: For each time step t, the attention weight α_t of the time step is calculated according to the designed attention score function;
[0127] Step S5-4: Using the calculated attention weight α_t, perform weighted summation on the hidden states h_1, h_2, ..., h_T generated by the recurrent neural network at all time steps to generate a context vector c;
[0128] Step S5-5: Fuse the context vector c with the hidden state h_t of the current time step. Specifically, it can be transformed by simple concatenation, addition or through an additional fully connected layer to better utilize the timing and context information;
[0129] Step S5-6: Use a deep learning framework (such as TensorFlow, PyTorch) to write the corresponding code according to the designed attention mechanism structure;
[0130] Step S5-7: Integrate the code module with the attention mechanism into the recurrent neural network to ensure that the attention mechanism is applied at each time step or at the required level.
[0131] Through the above steps, the temporal attention mechanism was successfully designed and integrated into the recurrent neural network, which enhanced the model's ability to process temporal data. For the analysis and correction of the image sequence of material strip 1, it can more accurately capture the key frame information that affects the correction decision.
[0132] Step S6: Construct a deep learning model that integrates a convolutional neural network (to process spatial features) and a recurrent neural network (to combine temporal attention to process time series features), including:
[0133] Step S6-1: Use a deep learning framework (such as TensorFlow or PyTorch) to define convolutional neural networks and recurrent neural networks as reusable modules or layers;
[0134] Step S6-2: Use the output of the convolutional neural network (i.e., the feature vector of each frame image) as the input of the recurrent neural network, ensure that the data flow seamlessly transitions from the convolutional neural network to the recurrent neural network directly as the input of the recurrent neural network, and ensure that the output dimension of the convolutional neural network matches the expected input dimension of the recurrent neural network;
[0135] Step S6-3: Select the loss function (such as mean squared error MSE for regression tasks and cross entropy loss for classification tasks) and optimizer (such as Adam, SGD);
[0136] Step S6-4: Compile the model in the framework, specify the loss function, optimizer and evaluation index, and complete the model definition. Specifically, the framework in this application refers to a software platform or toolkit for building, training and deploying deep learning models. The framework provides a high-level API that allows developers to define neural network structures, perform data preprocessing, train models, evaluate performance, and make predictions in a more abstract way without having to write all the mathematical operations and optimization algorithm codes from the bottom. Specifically, the framework uses TensorFlow developed by Google, which can support static and dynamic graph execution modes) to compile the model, which refers to the neural network structure and its parameter set in deep learning. Specifically, the model defines how data is transformed through a series of layers (such as convolutional layers, fully connected layers, loop layers, etc.) to ultimately produce a predicted output. The structure (or architecture) of the model is designed by the developer, and the parameters (weights and biases) are learned during the training process through optimization algorithms (such as gradient descent) to minimize the difference (loss function) between the model prediction and the true label. The process of compiling a model in a framework refers to defining the structure and configuration of the model using the API provided by the framework, including selecting the model type (such as convolutional neural network, recurrent neural network, etc.), defining the type and connection method of the layer, setting the loss function and optimizer, etc., and then notifying the framework to prepare the execution environment of the model. This "compile" step does not necessarily involve compilation in the traditional sense (such as conversion to machine code), but refers more to the configuration and preparation stage of the model, ensuring that the model can run correctly within a given framework and can be trained and predicted.
[0137] Step S7: Use the labeled data set to train and verify the deep learning model, and verify the effectiveness and generalization ability of the deep learning model through the cross-validation method;
[0138] Step S8: Integrate the deep learning model after training and verification into the copper pasting machine control system, and perform real-time correction operation through the electrical connection correction mechanism. Specifically, according to the calculation requirements and real-time requirements of the model, select or configure a suitable hardware platform (such as embedded devices, industrial computers, servers, etc.) and software environment (such as operating system, deep learning framework, interface library, etc.), encapsulate the trained model into an easy-to-call API or service, write the necessary interface code, so that it can be seamlessly connected with the copper pasting machine control system or other upper-level application software, and then integrate the model into the overall control system of the copper pasting machine to ensure that the model prediction results can be smoothly passed to the correction execution mechanism, and perform system joint debugging to ensure that the model runs stably and responds in time. Install the system integrated with the deep spatiotemporal graph neural network model on the actual copper pasting machine, connect the equipment, configure the parameters, and start the model operation. Monitor the running status of the model in real time, collect the model prediction results, the correction execution status and the actual correction effect data, and display the key indicators by controlling the touch screen 5.
[0139] The present invention completes the entire process from model construction to on-site deployment and monitoring through the above steps, ensuring that the model can effectively monitor the deviation of the material strip 1 and achieve accurate deviation correction.
[0140] Figure 1 is a schematic diagram of the three-dimensional structure of the present invention, Figure 2 It is a schematic diagram of the main structure of the present invention, as shown in Figure 1 and Figure 2 The shown device is a linear deviation correction device for a copper plating machine material strip based on deep learning, comprising a support frame 4, a conveying frame 6, a detection mechanism and a deviation correction mechanism. The conveying frame 6 is fixedly connected to the upper end of the support frame 4. The support frame 4 in the present invention serves as the basic framework of the entire deviation correction device, providing stable support to ensure the stable installation and operation of the conveying frame 6, the deviation correction mechanism and the detection mechanism. The conveying frame 6 in the present invention is fixed to the upper end of the support frame 4, responsible for carrying and conveying the material strip 1, and is a channel for the flow of materials. Cooperating with the support guide roller 21, it ensures that the material strip 1 passes through the deviation correction device smoothly and continuously. The conveying frame 6 is connected with the support guide roller 21. The deviation correction mechanism is installed on the conveying frame 6. The deviation correction mechanism and the detection mechanism are electrically connected. A control touch screen is installed on the support frame 4. The control touch screen and the deviation correction mechanism are electrically connected. The control touch screen serves as a human-computer interaction interface, which can display the deviation correction status and allow the operator to set parameters and monitor the deviation correction process.
[0141] The deviation correction mechanism in the present invention includes a driving cylinder A7, a fixed beam 8 and a flexible pressure roller 2. The driving cylinder A7 is fixedly connected to the upper end of the conveying frame 6. The driving cylinder A7 is used to provide a power source. According to the deviation information fed back by the control system of the copper pasting machine, the cylinder is controlled to extend and retract by an electrical signal to realize the driving deviation correction action. In the present invention, a driving rod 9 is connected to the output end of the driving cylinder A7, and a connecting block 10 is fixedly connected to the top of the driving rod 9. The fixed beam 8 is fixedly connected between the two connecting blocks 10. Two driving cylinders B11 symmetrically arranged are arranged at the bottom of the fixed beam 8. The driving cylinder B11 and the fixed beam 8 are connected to each other through a guide rod 12. A spring 13 is sleeved on the guide rod 12. One end of the spring 13 is fixedly connected to the bottom of the fixed beam 8. Then, the other end of the spring 13 is fixedly connected to the driving cylinder B11, and the flexible pressure roller 2 is connected between the two driving cylinders B11. Specifically, the two driving cylinders B11 are connected to the flexible pressure roller 2 through a driving guide rod 12 (not shown in the figure), which can drive the flexible pressure roller 2 to move back and forth. The flexible pressure roller 2 in the present invention is used to correct the deviation of the material belt 1. Specifically, the flexible pressure roller 2 can be made of a flexible material such as polyurethane. When the material belt 1 deviates, the flexible pressure roller 2 can apply longitudinal pressure to the material belt 1 by pushing the driving cylinder A7 to press the material belt 1, and then the position of the material belt 1 is adjusted by the driving cylinder B11 to guide the material belt 1 back to its center position, ensuring that the deviation correction action is both powerful and gentle to avoid damage to the material belt 1.
[0142] The detection mechanism in the present invention includes a sensor 3, a sensor bracket 14 and an adjustment component. The sensor 3 is fixedly connected to the sensor bracket 14. The sensor 3 adopts an image sensor 3 for capturing the real-time image of the material strip 1 and providing visual input for the deep learning model. The sensor bracket 14 is fixedly connected to one side of the adjustment component. The adjustment component includes an adjustment sleeve 16, an adjustment column, an adjustment knob, a cross bar 19 and a bracket 20. The cross bar 19 is fixedly connected to the bracket 20. The adjustment sleeve 16 is connected to the cross bar 19. One end of the adjustment column and the adjustment knob are interconnected, and the other end of the adjustment column passes through the adjustment sleeve 16 and is fixedly connected to the sensor bracket 14. The adjustment column and the adjustment sleeve 16 in the present invention are fixed to each other by a hand-tightening bolt A15, and the cross bar 19 and the adjustment sleeve 16 are fixed to each other by hand-tightening bolts B17 and hand-tightening bolts C18. The present invention can manually adjust the position of the sensor 3 to adapt to material strips 1 of different widths or materials to ensure detection accuracy.
[0143] Working principle: During the conveying process of the material strip 1, the image information of the surface of the material strip 1 is continuously collected through the sensor 3, the image is preprocessed and feature extracted through the deep learning model, and the center line position of the material strip 1 is identified through the convolutional neural network to determine whether there is an offset. If the material strip 1 is detected to be offset, the model will calculate the offset and send the correction instruction to the correction device in the form of an electrical signal. After the correction device receives the signal, the driving cylinder A7 and the driving cylinder B11 can drive the flexible pressure roller 2 according to the deviation direction and size of the material strip 1, and apply longitudinal pressure and lateral pressure to the material strip 1, thereby realizing the rapid and accurate correction of the offset of the copper laminating machine material strip 1, thereby improving production efficiency and product quality.
[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A copper plating machine material strip linear deviation correction device based on deep learning, comprising a support frame (4), a conveying frame (6), a deviation correction mechanism and a detection mechanism, wherein the conveying frame (6) is fixedly connected to the upper end of the support frame (4), a supporting guide roller (21) is connected to the conveying frame (6), the deviation correction mechanism is installed on the conveying frame (6), the deviation correction mechanism and the detection mechanism are electrically connected, a control touch screen is installed on the support frame (4), and the control touch screen and the deviation correction mechanism are electrically connected; It is characterized in that The deviation correction mechanism comprises a driving cylinder A (7), a fixed beam (8) and a flexible pressure roller (2), wherein the driving cylinder A (7) is fixedly connected to the upper end of the conveying frame (6), a driving rod (9) is drivingly connected to the output end of the driving cylinder A (7), a connecting block (10) is fixedly connected to the top of the driving rod (9), the fixed beam (8) is fixedly connected between the two connecting blocks (10), two driving cylinders B (11) symmetrically arranged are arranged at the bottom of the fixed beam (8), the driving cylinder B (11) and the fixed beam (8) are connected to each other through a guide rod (12), a spring (13) is sleeved on the guide rod (12), one end of the spring (13) is fixedly connected to the bottom of the fixed beam (8), and the other end of the spring (13) is fixedly connected to the driving cylinder B (11), the flexible pressure roller (2) is connected between the two driving cylinders B (11), and the flexible pressure roller (2) is used to correct the deviation of the material strip (1).
2. According to the deep learning-based copper bonding machine strip linear deviation correction device according to claim 1, it is characterized in that: The detection mechanism comprises a sensor (3), a sensor bracket (14) and an adjustment component, wherein the sensor (3) is fixedly connected to the sensor bracket (14), the sensor bracket (14) is fixedly connected to one side of the adjustment component, and the adjustment component comprises an adjustment sleeve (16), an adjustment column (23), an adjustment knob (22), a cross bar (19) and a bracket (20), wherein the cross bar (19) is fixedly connected to the bracket (20), the adjustment sleeve (16) is connected to the cross bar (19), one end of the adjustment column (23) and the adjustment knob (22) are mutually connected, and the other end of the adjustment column (23) passes through the adjustment sleeve (16) and is mutually fixedly connected to the sensor bracket (14).
3. According to the deep learning-based copper bonding machine strip linear deviation correction device according to claim 2, it is characterized in that: The adjusting column (23) and the adjusting sleeve (16) are fixed to each other via a hand bolt A (15), and the cross bar (19) and the adjusting sleeve (16) are fixed to each other via a hand bolt B (17) and a hand bolt C (18).
4. A linear deviation correction method for a copper bonding machine strip linear deviation correction device based on deep learning according to claim 2, characterized in that: The following steps are involved: Step S1: installing a detection mechanism, installing a sensor (3) in the detection mechanism on a sensor bracket (14), and collecting images of the material belt (1) in the conveying process in real time through the sensor (3); Step S2: using a convolutional neural network to extract spatial features from the material strip image collected by the sensor (3) to obtain feature information of each frame of the image; Step S3: Arrange the feature information vectors of each frame of image in chronological order to form an image feature sequence; Step S4: inputting the image feature sequence into a recurrent neural network to capture the timing information of the material strip (1); Step S5: Design a temporal attention mechanism and integrate it into the recurrent neural network; Step S6: constructing a deep learning model integrating a convolutional neural network and a recurrent neural network; Step S7: Use the labeled data set to train and verify the deep learning model, and verify the effectiveness and generalization ability of the deep learning model through the cross-validation method; Step S8: Integrate the trained and verified deep learning model into the copper laminating machine control system, and perform real-time deviation correction operations through an electrically connected deviation correction mechanism.
5. The linear deviation correction method of the copper bonding machine strip linear deviation correction device based on deep learning according to claim 4 is characterized in that: In step S1, the sensor (3) is mounted on the sensor bracket (14), and the image of the material belt (1) during the conveying process is collected in real time by the sensor (3), including: Step S1-1: determining the installation position of the sensor (3) according to the conveying path of the material belt (1); Step S1-2: Mount the sensor (3) on the sensor bracket (14), and configure the parameters of the sensor (3), including setting the sampling frequency and exposure parameters of the sensor (3).
6. The linear deviation correction method of the copper bonding machine strip linear deviation correction device based on deep learning according to claim 5 is characterized in that: The step S2 uses a convolutional neural network model to extract spatial features from the strip image collected by the sensor (3), and obtains feature information of each frame of the image, including: Step S2-1: pre-processing the material strip image collected by the sensor (3); Step S2-2: construct a convolutional neural network model; Step S2-3: extracting spatial features of the strip image through a convolutional neural network model; Step S2-4: vectorize the extracted spatial features.
7. The linear deviation correction method of the copper bonding machine strip linear deviation correction device based on deep learning according to claim 6 is characterized in that: The step S2-1 of pre-processing the strip image collected by the sensor (3) comprises: Step S2-1-1: adjusting the original image collected by the sensor (3) to the standard size required by the convolutional neural network; Step S2-1-2: converting the adjusted image into a grayscale image; Step S2-1-3: Normalize the pixel values of the grayscale image.
8. The linear deviation correction method of the copper bonding machine strip linear deviation correction device based on deep learning according to claim 6 is characterized in that: The step S2-2 of constructing a convolutional neural network model includes: Step S2-2-1: Determine the size of the input layer according to the image size; Step S2-2-2: design a multi-layer convolution layer, each layer containing multiple convolution kernels, for learning image features, which include edge features, texture features, contrast features, straight line features, shape features, motion direction features and time series features; Step S2-2-3: Calculate the image features through the convolution layer and output a new set of feature maps; Step S2-2-4: Perform a pooling operation on the new feature map to reduce the dimension of the feature map and reduce the amount of calculation; Step S2-2-5: Compile the convolutional neural network model in the deep learning framework, specify the optimizer, loss function and evaluation index; Step S2-2-6: Prepare training data sets and validation data sets to train the convolutional neural network model. During the training process, the model parameters are updated through back propagation and gradient descent.
9. The linear deviation correction method of the copper bonding machine strip linear deviation correction device based on deep learning according to claim 4 is characterized in that: The step S5 designs a temporal attention mechanism and combines it into a recurrent neural network, including: Step S5-1: Design a function to calculate the attention score for each time step, comparing the hidden state h_t of the current time step with the hidden state of the previous time step or the mean of the entire time series, using dot product and weighted sum; Step S5-2: Use the softmax function to convert the calculated raw attention score into a probability distribution, ensuring that the sum of the weights of all time steps is 1, thus forming a valid attention weight vector; Step S5-3: For each time step t, the attention weight α_t of the time step is calculated according to the designed attention score function; Step S5-4: Using the calculated attention weight α_t, perform weighted summation on the hidden states h_1, h_2, ..., h_T generated by the recurrent neural network at all time steps to generate a context vector c; Step S5-5: Fuse the context vector c with the hidden state h_t of the current time step; Step S5-6: Use a deep learning framework to write the corresponding code according to the designed attention mechanism structure; Step S5-7: Integrate the code module with the attention mechanism into the recurrent neural network to ensure that the attention mechanism is applied at each time step or at the required level.
10. The linear deviation correction method of the copper bonding machine strip linear deviation correction device based on deep learning according to claim 4, characterized in that: The deep learning model integrating the convolutional neural network model and the recurrent neural network is constructed in step S6, including: Step S6-1: Use a deep learning framework to define convolutional neural networks and recurrent neural networks as reusable modules or layers; Step S6-2: Using the output of the convolutional neural network as the input of the recurrent neural network, ensuring that the data flow seamlessly transitions from the convolutional neural network to the recurrent neural network directly as the input of the recurrent neural network; Step S6-3: Select loss function and optimizer; Step S6-4: Compile the model in the framework, specify the loss function, optimizer and evaluation index, and complete the model definition.
