Hardware plate production process quality detection method based on image processing

Abstract

The invention aims to provide a hardware plate production process quality detection method based on image processing. The method comprises the following steps: acquiring sensor data generated by a digital twin workshop; acquiring plate image data; inputting the sensor data and the plate image data into a plate quality detection model; and the plate quality detection model outputs a plate quality judgment result. On the basis of performing plate quality detection by machine vision, a digital twin workshop is creatively introduced to perform sensor data simulation, data acquisition is more convenient than data acquisition by mounting a sensor on a real production line, and image data from the plate and the sensor data are fused by constructing a plate quality detection model, so that the plate quality detection accuracy is improved. The plate quality is comprehensively judged, the detection accuracy is high, and meanwhile the detection efficiency can be greatly improved.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a quality detection method for a hardware sheet material production process based on image processing. Background Art

[0002] During the production process, the sheet material needs to be monitored and tested in real time to ensure that the problem is discovered and solved as soon as the production line is abnormal. If the sheet material is not inspected during the production process, the yield of the sheet material will be greatly reduced, and the produced sheet material may have safety hazards, which may endanger the personal and property safety of consumers. Sheet material quality inspection is mainly a series of scientific tests on the sheet material, which aims to ensure the quality and safety of the sheet material and protect human health and personal safety. These tests include but are not limited to the following aspects: Appearance inspection: Observe the surface of the sheet material with the naked eye to see if there are defects such as cracks, scars, bubbles, rust, etc., and whether the surface is flat and smooth, and whether the color is uniform, so as to judge the appearance quality of the sheet material. Size inspection: Measure the length, width, thickness and other dimensional parameters of the sheet material to ensure that the sheet material meets the specified standard size and avoid affecting the use effect due to non-standard size. Chemical composition analysis: By analyzing the content of various elements in the sheet material, it is judged whether the chemical composition of the sheet material meets the standard requirements, which is crucial for the physical properties and service life of the sheet material. Mechanical property test: including tensile strength, yield strength, impact toughness, hardness and other mechanical property tests to ensure that the strength, toughness, hardness and other aspects of the board meet the standard requirements and ensure the use effect and safety. Surface quality inspection: inspect the surface finish, flatness, roughness, coating adhesion and other aspects of the board to ensure that the quality of the board surface meets the standard requirements and affects the appearance and service life of the board. In addition, for specific types of boards, such as artificial boards and wood and bamboo products, additional tests are required, such as formaldehyde emission tests, VOC tests, heavy metal tests, flame retardant and fireproof performance tests, etc., to ensure that the products meet national standards and environmental protection requirements. These tests are not only related to the quality and safety of the board, but also important measures to protect the rights and interests of consumers.

[0003] Currently, the commonly used sheet material quality inspection is production line inspection based on machine vision. The sheet materials on the production line are photographed and the quality of the sheet materials is judged based on the acquired images. Since the production speed on the sheet material production line is very fast and some sheets are very small in size, it is difficult to accurately judge whether there are quality problems with the sheet materials only by means of machine vision. False detection and missed detection are prone to occur, resulting in low accuracy of sheet material quality inspection. Summary of the invention

[0004] The purpose of the present invention is to provide a quality inspection method for the production process of hardware sheet metal based on image processing. This method creatively introduces a digital twin workshop to simulate sensor data on the basis of sheet metal quality inspection by machine vision. This is more convenient than installing sensors on a real production line to obtain data. Then, by constructing a sheet metal quality inspection model and integrating image data and sensor data from the sheet metal, a comprehensive judgment is made on the sheet metal quality. This method has high detection accuracy and can greatly improve detection efficiency. The quality inspection method of the hardware sheet material production process based on image processing includes: Acquire sensor data generated by the digital twin workshop; Acquire sheet material image data; Inputting the sensor data and the sheet material image data into a sheet material quality detection model; The sheet material quality detection model outputs a sheet material quality judgment result.

[0005] Preferably, before inputting the sensor data and the image data into the sheet material quality detection model, the method further includes constructing a sheet material quality detection model, specifically: Constructing the input layer, hidden layer, feedback receiving layer and output layer of the sheet material quality detection model; Set the connection weights between the input layer and the hidden layer; Set the hidden layer node bias and activation function; Set the trend characteristic factor of the feedback receiving layer; Set the output weights between each feedback receiving layer and the hidden layer.

[0006] Preferably, the trend characteristic factors for setting the feedback receiving layer include: The trend characteristic factor of the feedback connection layer is expressed as:

[0007] in, is the trend characteristic factor of the i-th feedback connection layer, and n is the number of attributes of the current feedback connection layer;

[0008] Where t represents the change per unit time, is the output at the current moment, g is the memory factor of the sheet metal quality detection model, and k is the memory sample of the sheet metal quality detection model.

[0009] Preferably, the setting of the output weight between each feedback receiving layer and the hidden layer includes: The influence factor of the output weight between each feedback receiving layer and the hidden layer is between 0 and 1, expressed as:

[0010] The trend characteristic factor is used to correct the influencing factor of the output weight between each feedback receiving layer and the hidden layer:

[0011] The output weight of the feedback receiving layer is expressed as: .

[0012] Preferably, after setting the output weight between each feedback receiving layer and the hidden layer, the step further includes correcting the output of the hidden layer, specifically: Linearly add the feedback receiving layer to the current input, that is, ; The corrected hidden layer output matrix is ​​expressed as:

[0013] in, is the input of the sheet metal quality inspection model at the current moment, is the output weight of the receiving layer at the current moment, Feedback the output weight of the receiving layer at the current moment.

[0014] Preferably, after acquiring the image data of the production workshop, the method further includes correcting the sheet material image data, specifically: The image data is subjected to distortion correction, and the coordinates of the image points after radial distortion correction are expressed as:

[0015] in, is the abscissa after radial correction, is the ordinate after radial correction, is the horizontal coordinate before correction, is the ordinate before correction, is the radial distortion coefficient; The image data is subjected to distortion correction, and the coordinates of the image points after tangential distortion correction are expressed as:

[0016] in, is the horizontal coordinate after tangential correction, is the ordinate after tangential correction, is the tangential distortion coefficient.

[0017] Preferably, after acquiring the image data of the production workshop, the method further includes denoising the sheet material image data, specifically: For pixels in the sheet image , filter the image with a 3×3 template and get the current pixel after filtering It is expressed as:

[0018] After processing Sort the pixels in ascending order to generate a monotonically increasing two-dimensional sequence. Let ϑ be a two-dimensional template, and the median filter output is: .

[0019] Preferably, before inputting the sensor data and the sheet material image data into the sheet material quality detection model, the method further includes training the sheet material quality detection model, specifically: Extracting an adaptive dictionary of sample data by using a K-SVD method, including: setting K-SVD algorithm parameters, the number of iterations and the sparsity of the OMP algorithm during the iteration process, initializing the adaptive dictionary and the number of atoms in the adaptive dictionary; The OMP algorithm is used to solve the coefficient matrix; Randomize dictionary atomic numbers; Find the non-zero elements in the coefficient matrix and record the positions of the non-zero elements; If there are no non-zero elements, calculate the error matrix and record the column with the maximum error; Normalize the column with the maximum error as the atom of the adaptive dictionary; Iterate and update until the preset requirements are met; The sample data is sparsely encoded using orthogonal matching pursuit according to the adaptive dictionary, and the sparse encoding is used as the input of the sheet metal quality detection model.

[0020] The hardware sheet material production process quality inspection system based on image processing includes: The sensor data acquisition module is used to obtain the sensor data generated by the digital twin workshop; An image data acquisition module, used for acquiring sheet material image data; A data transmission module, used for inputting the sensor data and the sheet material image data into a sheet material quality detection model; The data processing module is used for the sheet material quality detection model to output the sheet material quality judgment result.

[0021] An electronic device, characterized in that it includes: a processor and a memory, wherein the memory is used to store computer program codes, and the computer program codes include computer instructions. When the processor executes the computer instructions, the electronic device executes a metal sheet production and quality inspection method based on image processing.

[0022] The beneficial effects of the present invention are: 1. The present invention adopts a combination of image data acquired by machine vision and sensor data simulated by a digital twin workshop to detect the quality of sheet metal from multiple aspects, which can greatly improve the accuracy of sheet metal quality detection; 2. The sensor data acquired by the present invention comes from the simulation of the digital twin workshop rather than from sensors installed in the actual workshop, which can greatly improve the efficiency and accuracy of data acquisition, save detection costs, and improve detection efficiency; 3. The present invention adopts a sheet metal quality detection model to process sheet metal image data and sensor data, which has higher accuracy than ordinary manual judgment and simple artificial intelligence judgment models, and the model can be continuously improved through trend characteristic factors, which is suitable for various sheet metal quality detection scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention. In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor. Figure 1 It is a schematic diagram of the process of quality inspection of hardware sheet material production process based on image processing of the present invention; Figure 2 A schematic diagram of the process of constructing a sheet material quality detection model of the present invention; Figure 3 This is a schematic diagram of the adaptive dictionary process of extracting sample data using the K-SVD method of the present invention; Figure 4 A schematic diagram of the hardware structure of an electronic device of the present invention; Figure 5 It is a structural schematic diagram of a sheet material quality detection model of the present invention. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0026] In addition, the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0027] Currently, the commonly used sheet material quality inspection is production line inspection based on machine vision. The sheet materials on the production line are photographed and the quality of the sheet materials is judged based on the acquired images. Since the production speed on the sheet material production line is very fast and some sheets are very small in size, it is difficult to accurately judge whether there are quality problems with the sheet materials only by means of machine vision. False detection and missed detection are prone to occur, resulting in low accuracy of sheet material quality inspection.

[0028] The present invention adopts a combination of image data acquired by machine vision and sensor data simulated by a digital twin workshop to detect the quality of sheet metal from multiple aspects, which can greatly improve the accuracy of sheet metal quality detection; the sensor data acquired by the present invention comes from the simulation of the digital twin workshop rather than from sensors installed in the actual workshop, which can greatly improve the efficiency and accuracy of data acquisition, save detection costs, and improve detection efficiency; the present invention adopts a sheet metal quality detection model to process sheet metal image data and sensor data, which has higher accuracy than ordinary manual judgment and simple artificial intelligence judgment models, and the model can be continuously improved through trend characteristic factors, and is suitable for various sheet metal quality detection scenarios.

[0029] Quality inspection method for metal sheet production process based on image processing, reference Figure 1 , including the steps of: Step S100, acquiring sensor data generated by the digital twin workshop; The digital twin workshop is an important application of the information-physical fusion system and a new mode of future workshop operation. It is a system-level information-physical system composed of "one hard, one soft, and one network" that simulates the operating status and process of the real workshop through digital technology. The concept of the digital twin workshop involves the reference architecture, operation mode, composition and key technologies of a workshop, aiming to improve production efficiency and product quality and optimize the production process through digital technology. The core of the digital twin workshop lies in the fusion of data, which includes data related to the physical space (PS), virtual space (VS), and service system (SSS), domain knowledge, and a collection of derivative data generated by data fusion. The fusion and optimization of these data drive the operation and interaction of the digital twin workshop. By aggregating data from the physical and information spaces through specific rules, the digital twin workshop can realize the intelligent management and optimization of the workshop, thereby improving production efficiency and product quality.

[0030] Digital twin is sometimes used to refer to the completion of a digital model of a factory's workshop and production line before it is built. In this way, the factory can be simulated and simulated in a virtual cyberspace, and the real parameters can be passed to the actual factory construction. After the workshop and production line are built, the two continue to exchange information in daily operation and maintenance. In actual applications, digital twin workshops have been applied to many fields, such as the washing machine inner drum production line that uses digital twin technology in Haier's Shanghai washing machine interconnected factory, and the "digital twin factory" established in Changzhou. These application cases show the huge potential of digital twin workshops in improving production efficiency and optimizing production processes.

[0031] The benefits of digital twin workshops include improving production efficiency, improving product quality, reducing costs, improving safety, and promoting technological and process innovation and continuous improvement. The digital twin workshop simulates and optimizes the workshop production process through digital technology, helping enterprises quickly discover potential problems and predict and optimize them, thereby improving production efficiency. The core of this solution is to establish a digital twin model to truly and accurately replicate the actual workshop production process, obtain various types of workshop data in real time through sensors and data acquisition equipment, such as temperature, humidity, machine status, etc., and simulate different production scenarios to evaluate production efficiency under various circumstances. In addition, the workshop digital twin solution can further improve production efficiency by optimizing production plans and resource allocation, avoiding idle resources and production blockages. The digital twin workshop can also help enterprises improve their safety production level. Through the modeling and simulation of the digital twin factory, enterprises can predict the dangers and risks that may occur in the production process and take timely measures to prevent accidents. The digital twin factory realizes the visualization and real-time monitoring of the production process, helps enterprises better understand the safety status of the production process, discover and solve problems in time, and improve the level of safe production. In addition, with the continuous development of digital twin technology, this technology plays an increasingly important role in industrial manufacturing, bringing higher benefits and competitiveness to enterprises. The digital twin workshop combines the actual factory with the virtual world through digital modeling and simulation technology, realizes all-round monitoring and management of factory data, and thus helps the actual factory to allocate resources and intelligent production.

[0032] Step S200, acquiring sheet material image data; The acquisition of images of production line products mainly relies on high-precision image acquisition equipment and advanced image processing algorithms. In the production line environment, image acquisition is a key step, which directly affects the accuracy and efficiency of product quality inspection. In order to achieve efficient quality control and automation of the production line, high-precision image acquisition equipment and corresponding image processing algorithms are required. The specific steps and methods for acquiring images of production line products include: Use high-precision image acquisition equipment: On the assembly line, by installing high-definition cameras or other image acquisition equipment, the image information of the product is captured in real time. These devices need to have high resolution and fast response capabilities to ensure that the acquired images can clearly reflect the details of the product. Apply advanced image processing algorithms: Use image processing algorithms to process and analyze the acquired images to extract key information such as the position, shape, and size of the product. These algorithms can identify the characteristics of the product, calculate the exact position of the grasping point, and control the grasping mechanism for precise grasping. Solve technical problems: When the product moves at high speed, the visual system needs to be able to stably and accurately obtain the position information of the product. This requires the use of high-speed image processing technology and real-time tracking algorithms to ensure that the time difference between image acquisition, processing, analysis and control is as small as possible to achieve real-time and accurate positioning and grasping. Adapt to diverse product lines: Due to the wide variety of products on the assembly line, the shapes and sizes are different, so the visual positioning and grasping system also needs to have high flexibility and adaptability. This requires the system to be able to automatically adjust and optimize according to different product characteristics to ensure stable and reliable positioning and grasping under various circumstances. Consider practical factors: In practical applications, it is also necessary to consider the effects of lighting conditions, reflections on the product surface, occlusion and other factors on the quality and accuracy of image acquisition. Make appropriate adjustments and optimizations according to specific circumstances to ensure the stability and reliability of the system. Acquiring images of production line products is a complex process involving hardware equipment, software algorithms and actual application adjustments. It is necessary to comprehensively consider multiple factors to ensure that the collected image information can accurately reflect the quality of the product and support the automation and intelligent management of the production line.

[0033] Step S300, inputting sensor data and sheet material image data into a sheet material quality detection model; The sheet metal quality detection model is an artificial intelligence algorithm that uses big data and deep learning technology to simulate human thinking and creativity. It can process multimodal data such as images, text, audio, etc. to predict future events or generate new content. This model usually has a large number of parameters and can show intelligence in different fields and tasks, such as generating high-quality text, images, and audio and video content. The sheet metal quality detection model can produce valuable answers that meet the actual needs of users through pre-training, and improve the naturalness and fluency of conversations. In addition, these models can also learn autonomously without manual annotation, and become an important research direction for future prediction models. With the increase in data scale, the improvement of computing power, and the innovation of algorithms, sheet metal quality detection models will face challenges in large-scale data processing, multimodal data fusion, explanatory prediction models, autonomous learning, and ethics and privacy. The sheet metal quality detection model has a wide range of applications. They can process large amounts of data from various data sources through big data technology and algorithms, so as to capture market trends and corporate performance more comprehensively. ‌These models are able to identify and analyze complex relationships and patterns, provide more accurate predictions, and avoid biases and misjudgments in human intuition. ‌In addition, sheet metal quality inspection models usually have good interpretability, which helps users better understand the prediction results. ‌Sheet metal quality inspection models are an algorithm that uses big data and deep learning technology to simulate human thinking and creativity. They can demonstrate intelligence in different fields and tasks, and provide powerful tools for future prediction and generation. ‌

[0034] Step S400: the sheet material quality detection model outputs a sheet material quality judgment result.

[0035] The quality judgment standards of sheet materials mainly include the following aspects: Size and geometry requirements: The length, width, thickness and other dimensional parameters of the sheet should meet the standard requirements to ensure the accuracy and consistency of its geometric shape. Physical properties: Including the density, strength, hardness, bending performance, impact performance, elastic modulus, etc. of the sheet, evaluated through experimental tests such as tensile test, impact test, hardness test, etc. Chemical properties: The chemical composition, solubility, corrosion resistance, etc. of the sheet are evaluated through water quality test, corrosion test, solubility test, etc. Surface quality: Including the flatness, finish, unevenness, defects (such as cracks, breakage, bubbles, etc.), color difference, etc. of the sheet surface, evaluated through visual observation, measurement by measuring instruments, optical microscope inspection, etc. Environmental protection requirements: Environmental protection indicators of the board, such as formaldehyde emission, volatile organic compound content, heavy metal content, etc., are evaluated through indoor air quality testing, chemical analysis, etc. Fire performance: The combustion performance and flame retardant performance of the board, to ensure its performance in the event of a fire, are evaluated through combustion experiments, flame retardant experiments, etc. Surface treatment and coating evaluation: For coated or surface-treated boards, the adhesion, corrosion resistance, wear resistance, appearance quality, etc. of the coating are evaluated through coating peeling experiments, salt spray tests, appearance inspections, etc.

[0036] The sheet material quality judgment standard is intended to ensure that its physical, chemical, environmental and safety performances meet the requirements to meet the needs of different application fields for sheet materials. The size and geometric requirements are mainly judged by data from sheet material images. Physical properties, chemical properties, surface quality, environmental protection requirements, fire performance, surface treatment and coating evaluation are mainly judged by sensor data simulated from the digital twin workshop. The present invention integrates the image data of the sheet material and the sensor data of the digital twin workshop to conduct a comprehensive evaluation of the sheet material quality, which can greatly improve the accuracy of the evaluation. In addition, it is more convenient to obtain sensor data from the digital twin workshop than to obtain data by placing sensors in the real workshop, and it is not easily affected by the external environment and cause inaccurate sensor data, which further improves the accuracy and efficiency of sheet material quality detection.

[0037] Preferably, reference Figure 2 and Figure 5 , step S300, before inputting the sensor data and the image data into the sheet material quality detection model, it also includes step S230, constructing the sheet material quality detection model, specifically: Step S231, constructing the input layer, hidden layer, feedback receiving layer and output layer of the sheet material quality detection model; Input layer: In a computer control system, the input layer is the interface between the computer control system and the outside world, and is responsible for receiving signals from sensors, operators or other input devices. These signals may be continuous (such as temperature, pressure, etc.) or discrete (such as switch status). The main task of the input layer is to convert these raw signals into digital signals that can be processed by the computer, which is usually achieved through an analog-to-digital converter (ADC). In a neural network, the input layer is the first layer of the neural network, and its function is to convert the input data into a format that can be processed inside the neural network. The number of neurons in the input layer depends on the number of features of the input data. In an embodiment of the present invention, the input layer only memorizes the input samples, so the number of nodes in the input layer is the same as the number of attributes of the input samples; In a neural network, a hidden layer is one or more layers located between the input layer and the output layer. Its function is to convert the input data into a higher-level feature representation. The number of neurons and the way they are connected are different in each hidden layer, depending on the specific neural network architecture. The number of hidden layers is an important parameter in the design of the neural network structure, which directly affects the performance and training time of the neural network. In the embodiment of the present invention, the hidden layer is used to activate the input, and the hidden layer outputs to the feedback receiving layer and the output layer. The feedback layer adjusts the control strategy based on the information fed back by the output results to achieve more precise control. This feedback mechanism is a key part in many control systems, ensuring that the system can make corresponding adjustments based on the actual operating conditions. In the embodiment of the present invention, the feedback layer memorizes and stores the output of the hidden layer, and the output layer obtains the final calculation result of the network.

[0038] Step S232, setting the connection weight between the input layer and the hidden layer; Each connection between the input layer and the hidden layer corresponds to a real number, called the connection weight. The function of the connection weight is to add a bias term to the input signal of the neuron to enhance or suppress the strength of the input signal. By adjusting the weight, the activation mode and response degree of the neuron can be changed. In addition, the connection weight can also be used as a feature extractor to improve the performance and generalization ability of the model by learning the features and patterns in the data.

[0039] Step S233, setting the hidden layer node bias and activation function; The activation function is a function that runs on the neurons of the artificial neural network and is responsible for mapping the input of the neuron to the output. The activation function introduces nonlinear factors to the neurons, allowing the neural network to arbitrarily approximate any nonlinear function, so that the neural network can be applied to many nonlinear models.

[0040] Step S234, setting the trend characteristic factor of the feedback receiving layer; In the embodiment of the present invention, the trend characteristic factor represents the data change rate per unit time, and can help the feedback receiving layer improve the memory and forgetting functions.

[0041] Step S235, setting the output weight between each feedback receiving layer and the hidden layer.

[0042] The hidden layer uses a linear or nonlinear function as its transfer function. The output layer performs linear weighting. The feedback layer connects the feedback within or between layers to achieve the purpose of memory. The output weight is used to optimize the association between each feedback layer and the hidden layer, thereby improving the accuracy of the sheet metal quality detection model.

[0043] Preferably, step S234, setting the trend characteristic factor of the feedback receiving layer includes: The trend characteristic factor of the feedback connection layer is expressed as:

[0044] in, is the trend characteristic factor of the i-th feedback connection layer, and n is the number of attributes of the current feedback connection layer;

[0045] Where t represents the change per unit time, is the output at the current moment, g is the memory factor of the sheet metal quality detection model, and k is the memory sample of the sheet metal quality detection model.

[0046] The present invention uses a sliding window to extract data features. Sliding window data extraction is a flow control technology, which is mainly used to improve the data transmission efficiency in network communication. Sliding window technology processes data by maintaining a fixed-size window and sliding the window continuously. It defines two pointers, one pointing to the starting position of the window and the other pointing to the end position of the window, and then continuously moves the two pointers, reads the content of the window, and performs necessary operations. This technology is suitable for solving problems that need to be found or calculated in continuous substrings or subarrays, such as finding the longest non-repeating character substring, finding the minimum window containing the target substring in the string, etc. The application of sliding windows is not limited to the data link layer and the transport layer, it is also widely used in data analysis, time series data processing and other fields. In the field of data analysis, sliding window technology can perform window operations on time series data to obtain various statistical indicators, so as to better understand the trends and characteristics of the data. This technology observes the data in the window by sliding a fixed-size window on the data sequence. The window moves one by one, each time moving a fixed step size until the entire data sequence is covered. Sliding window data extraction is a technique for processing data by maintaining a fixed-size window and sliding it over the data. It performs data analysis or processing by continuously moving the window position and reading the window content. It is applicable to a variety of scenarios, including but not limited to network communication, string processing, and data analysis. Preferably, step S235, setting the output weight between each feedback receiving layer and the hidden layer includes: The influence factor of the output weight between each feedback receiving layer and the hidden layer is between 0 and 1, expressed as:

[0047] The trend characteristic factor is used to correct the influencing factor of the output weight between each feedback receiving layer and the hidden layer:

[0048] The output weight of the feedback receiving layer is expressed as: .

[0049] Preferably, after setting the output weight between each feedback receiving layer and the hidden layer in step S235, the method further includes step S236 of correcting the output of the hidden layer, specifically: Linearly add the feedback receiving layer to the current input, that is, ; The corrected hidden layer output matrix is ​​expressed as:

[0050] in, is the input of the sheet metal quality inspection model at the current moment, is the output weight of the receiving layer at the current moment, Feedback the output weight of the receiving layer at the current moment.

[0051] Preferably, after obtaining the image data of the production workshop in step S200, the method further comprises step S210 of correcting the sheet material image data, specifically: The image data is subjected to distortion correction, and the coordinates of the image points after radial distortion correction are expressed as:

[0052] in, is the abscissa after radial correction, is the ordinate after radial correction, is the horizontal coordinate before correction, is the ordinate before correction, is the radial distortion coefficient; Radial distortion correction is a series of image processing operations on the distorted images acquired by the camera to remove the distortion in the image. This usually involves using an appropriate parametric model to simulate image distortion, calculating the parameters of the model, and then using the model to remove the distortion generated during the camera imaging process. The purpose of radial distortion correction is to facilitate subsequent processing related to the field of computer vision, such as spatial positioning, target detection and tracking algorithms. This technology has extremely wide application needs in many computer vision-related fields such as video surveillance, virtual reality, robot navigation, military aiming, television editing, and medical image analysis.

[0053] There are two main forms of radial distortion: barrel distortion and pincushion distortion. Barrel distortion is characterized by the image pixels diverging toward the center of the image, while pincushion distortion is the opposite, causing the edges of the image to shrink inward. Both distortions are caused by the fact that light is more bent away from the center of the lens than near the center. Radial distortion correction calculates and adjusts the image parameters to make the corrected image more consistent with the actual scene, improving the image quality and accuracy.

[0054] The image data is subjected to distortion correction, and the coordinates of the image points after tangential distortion correction are expressed as:

[0055] in, is the horizontal coordinate after tangential correction, is the ordinate after tangential correction, is the tangential distortion coefficient.

[0056] Tangential distortion correction is a step-by-step calibration method that uses an optimization algorithm to further improve the calibration accuracy by considering distortion factors. This method first uses a direct linear transformation method or a perspective transformation matrix method to solve the camera parameters, and then uses the obtained parameters as the initial values, and further considers distortion factors, especially tangential distortion, and optimizes these parameters through a nonlinear optimization algorithm to achieve the purpose of correction. Tangential distortion correction mainly targets the problem of inaccurate angles between the lens and the imaging plane, that is, the lens is not completely parallel to the image plane, which may occur during the camera assembly process. This distortion will lead to a decrease in image quality, so specific algorithms and techniques are needed to correct this distortion in order to improve the accuracy and clarity of the imaging.

[0057] Preferably, after obtaining the image data of the production workshop in step S200, the method further comprises step S220 of denoising the sheet material image data, specifically: For pixels in the sheet image , filter the image with a 3×3 template and get the current pixel after filtering It is expressed as:

[0058] After processing Sort the pixels in ascending order to generate a monotonically increasing two-dimensional sequence. Let ϑ be a two-dimensional template, and the median filter output is: .

[0059] Image denoising refers to the process of reducing noise in digital images, aiming to restore the original clear image from the image disturbed by noise. This process involves identifying and removing noise in the image, which may be caused by various factors, such as optical factors, atmospheric factors, artificial factors, and technical factors. The goal of image denoising is to improve the image quality while retaining the important features of the original image as much as possible. In image processing, denoising is an extremely important step, which includes two main methods: spatial domain denoising and transform domain denoising. Spatial domain denoising directly operates on the grayscale of the image, while transform domain denoising modifies the transform coefficient values ​​of the image in a certain transform domain of the image. Spatial domain denoising methods mainly include mean filtering, median filtering, and Wiener filtering, which show different performances in removing different types of noise (such as Gaussian noise, salt and pepper noise, and Poisson noise). For example, median filtering is more effective in filtering out impulse noise (such as salt and pepper noise), which appears as random white or black spots in the image. In addition, denoising methods also include bilateral filtering, which aims to remove noise from the image while avoiding introducing new noise or causing edge blur. Denoising technology has a wide range of applications, including but not limited to improving image quality, enhancing image readability and visibility, and improving the accuracy of subsequent image analysis.

[0060] The main purpose of image filtering and denoising is to reduce or eliminate the noise in the image by applying specific filtering techniques, such as Gaussian filtering, median filtering, etc., while maintaining the original features and details of the image as much as possible, such as edges, textures, etc., so that the processed image is more suitable for further analysis or display. Gaussian filtering partially overcomes the defects of noise suppression, but cannot completely overcome it because it does not consider the difference in pixel values ​​and has a low weight on edge information. Median filtering is a nonlinear method of removing noise, which can remove noise and protect the edges of the image. Its implementation principle is to replace the value of a point in a digital image with the median value of the values ​​of each point in a region where the point is located, so as to remove noise while maintaining the edge information of the image. Image filtering and denoising technology uses different filtering methods to reduce image noise while retaining the important features of the image as much as possible, providing a better foundation for subsequent image processing and analysis.

[0061] Preferably, reference Figure 3 , step S300, before inputting the sensor data and the sheet material image data into the sheet material quality detection model, it also includes step S240, training the sheet material quality detection model, specifically: Step S241, extracting an adaptive dictionary of sample data by a K-SVD method, including: Step S2411, setting K-SVD algorithm parameters, the number of iterations and the sparsity of the OMP algorithm during the iteration process, initializing the adaptive dictionary and the number of atoms in the adaptive dictionary; Step S2412, using the OMP algorithm to solve the coefficient matrix; ‌OMP algorithm, full name Orthogonal Matching Pursuit, is an algorithm widely used in the field of signal processing and compressed sensing. It is particularly suitable for recovering information from sparse signals. The working principle of the OMP algorithm mainly includes: Initialization: At the beginning, the residual is set to the difference between the measurement vector and the projection, and the solution vector is initialized to empty. Iteration process: In each iteration, the algorithm calculates the inner product of the current residual and the column vector in the dictionary, and selects the corresponding atom with the largest inner product. Then, the index of the atom is added to the set of selected atoms. Solution vector update: The atoms in the selected atom set are used to calculate the least squares solution to update the solution vector. Residual update: The residual is updated using the sub-dictionary composed of the selected atoms. Stop criterion: If the stopping criterion is met (such as the residual energy is lower than a certain threshold or the predetermined number of iterations is reached), the iteration ends; otherwise, return to the second step to continue the iteration. In this way, the OMP algorithm can effectively reconstruct the original sparse signal from a small number of observations.

[0062] Step S2413, randomly arrange the dictionary atomic numbers, denoted as reperm, and initialize the counter j=1; Step S2414, let h = rperm (j), find out the non-zero elements in the coefficient matrix, and record the positions of the non-zero elements as the hth atom of the random permutation dictionary; Step S2415, if there is no non-zero element, calculate the error matrix and record the column where the maximum error is located; Step S2416, normalizing the column where the maximum error is located as the atom of the adaptive dictionary; Step S2417, iteratively updating the counter until the preset requirement is met; Iterative updating is to continuously adjust the parameters in the network to minimize the prediction error or maximize the prediction accuracy so that the model can better fit the training data. This updating process is implemented through optimization algorithms such as gradient descent. Each update is based on the prediction error of the current model, and the gradient is calculated through the back-propagation algorithm. Then the network parameters are adjusted according to the calculated gradient, in order to improve the performance of the model in the next iteration. The benefits of iterative updating include: Improving model performance: By continuously adjusting the network parameters, the sheet metal detection model can gradually learn more accurate feature representations, thereby improving the prediction performance of the model. Each iterative update helps the model better adapt to the training data, thereby improving the generalization ability of the model. Adapting to data changes: In practical applications, the data set may change, such as the addition of new samples or changes in sample distribution. Through iterative updating, the sheet metal detection model can adapt to these changes, keeping the performance of the model unaffected, and further improving the learning efficiency and performance of the sheet metal detection model. The iterative update of the sheet metal detection model is an indispensable step in training deep learning models by continuously adjusting the network parameters to improve the performance of the model and adapt to data changes. Step S242, sparsely encode the sample data using orthogonal matching pursuit according to the adaptive dictionary, and use the sparse code as the input of the sheet material quality detection model.

[0063] The dictionary learned by the K-SVD learning algorithm has better sparsity than other dictionaries. It can accurately represent the sparse features of data with fewer coefficients and can effectively compress massive operation monitoring data. Compared with the time domain sample signal, the frequency domain signal can effectively eliminate the influence of the time shift of fault impact between samples of the same type, and the robustness of fault mode recognition is better. Therefore, its frequency domain data is used for processing. The process of using the K-SVD algorithm to obtain the input of the sheet metal quality detection model is as follows: all data in the database are converted into frequency domain to obtain its frequency domain data; the frequency domain database is divided into data set 1 and data set 2, so that both data sets contain each fault type evenly; data set 1 is used as the sample data of the K-SVD algorithm to learn the dictionary; based on the dictionary, the OMP algorithm is used to obtain the sparse coefficient matrices of data set 1 and data set 2 as the training set and test set, and labels are assigned according to the fault type to obtain the input of the sheet metal quality detection model.

[0064] Example 2 The hardware sheet material production process quality inspection system based on image processing includes: The sensor data acquisition module is used to obtain the sensor data generated by the digital twin workshop; An image data acquisition module, used for acquiring sheet material image data; A data transmission module, used for inputting sensor data and sheet metal image data into a sheet metal quality detection model; The data processing module is used for the sheet material quality detection model to output the sheet material quality judgment result.

[0065] Example 3 An electronic device, characterized in that it includes: a processor and a memory, the memory is used to store computer program code, the computer program code includes computer instructions, and when the processor executes the computer instructions, the electronic device executes a quality detection method for a metal sheet production process based on image processing.

[0066] refer to Figure 4 , the electronic device 2 includes a processor 21, a memory 22, an input device 23, and an output device 24. The processor 21, the memory 22, the input device 23, and the output device 24 are coupled via a connector, and the connector includes various interfaces, transmission lines, or buses, etc., which are not limited in the embodiments of the present invention. It should be understood that in various embodiments of the present invention, coupling refers to mutual connection in a specific manner, including direct connection or indirect connection through other devices, for example, through various interfaces, transmission lines, buses, etc.

[0067] The processor 21 may be one or more graphics processing units (GPUs). When the processor 21 is a GPU, the GPU may be a single-core GPU or a multi-core GPU. Optionally, the processor 21 may be a processor group consisting of multiple GPUs, and the multiple processors are coupled to each other via one or more buses. Optionally, the processor may also be other types of processors, etc., which are not limited in the embodiments of the present invention.

[0068] The memory 22 can be used to store computer program instructions and various computer program codes including program codes for executing the scheme of the present invention. Optionally, the memory includes but is not limited to random access memory (RAM), read-only memory (ROM), erasable programmable read only memory (EPROM), or portable read only memory (CD-ROM), which is used for related instructions and data.

[0069] The input device 23 is used to input data and / or signals, and the output device 24 is used to output data and / or signals. The output device 24 and the input device 23 may be independent devices or an integrated device.

[0070] The present invention adopts a combination of image data acquired by machine vision and sensor data simulated by a digital twin workshop to detect the quality of sheet metal from multiple aspects, which can greatly improve the accuracy of sheet metal quality detection; the sensor data acquired by the present invention comes from the simulation of the digital twin workshop rather than from sensors installed in the actual workshop, which can greatly improve the efficiency and accuracy of data acquisition, save detection costs, and improve detection efficiency; the present invention adopts a sheet metal quality detection model to process sheet metal image data and sensor data, which has higher accuracy than ordinary manual judgment and simple artificial intelligence judgment models, and the model can be continuously improved through trend characteristic factors, and is suitable for various sheet metal quality detection scenarios.

[0071] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A quality inspection method for the production process of metal sheet materials based on image processing, characterized in that: include: Acquire sensor data generated by the digital twin workshop; Acquire sheet material image data; Inputting the sensor data and the sheet material image data into a sheet material quality detection model; The sheet material quality detection model outputs a sheet material quality judgment result.

2. The method for detecting quality of metal sheet production process based on image processing according to claim 1 is characterized in that: Before inputting the sensor data and the image data into the sheet material quality detection model, the sheet material quality detection model is constructed, specifically: Constructing the input layer, hidden layer, feedback receiving layer and output layer of the sheet material quality detection model; Set the connection weights between the input layer and the hidden layer; Set the hidden layer node bias and activation function; Set the trend characteristic factor of the feedback receiving layer; Set the output weights between each feedback receiving layer and the hidden layer.

3. The method for detecting quality of metal sheet production process based on image processing according to claim 2 is characterized in that: The trend characteristic factors for setting the feedback receiving layer include: The trend characteristic factor of the feedback connection layer is expressed as: in, is the trend characteristic factor of the i-th feedback connection layer, and n is the number of attributes of the current feedback connection layer; Where t represents the change per unit time, is the output at the current moment, g is the memory factor of the sheet metal quality detection model, and k is the memory sample of the sheet metal quality detection model.

4. The method for detecting quality of metal sheet production process based on image processing according to claim 3 is characterized in that: The step of setting the output weight between each feedback receiving layer and the hidden layer includes: The influence factor of the output weight between each feedback receiving layer and the hidden layer is between 0 and 1, expressed as: The trend characteristic factor is used to correct the influencing factor of the output weight between each feedback receiving layer and the hidden layer: The output weight of the feedback receiving layer is expressed as: 。 5. The method for detecting quality of metal sheet production process based on image processing according to claim 4 is characterized in that: After setting the output weights between each feedback receiving layer and the hidden layer, the output of the hidden layer is further modified, specifically: Linearly add the feedback receiving layer to the current input, that is, ; The corrected hidden layer output matrix is ​​expressed as: in, is the input of the sheet metal quality inspection model at the current moment, is the output weight of the receiving layer at the current moment, Feedback the output weight of the receiving layer at the current moment.

6. The method for quality inspection of metal sheet production process based on image processing according to claim 1 is characterized in that: After acquiring the image data of the production workshop, the image data of the sheet metal is corrected, specifically: The image data is subjected to distortion correction, and the coordinates of the image points after radial distortion correction are expressed as: in, is the horizontal coordinate after radial correction, is the ordinate after radial correction, is the horizontal coordinate before correction, is the ordinate before correction, is the radial distortion coefficient; The image data is subjected to distortion correction, and the coordinates of the image points after tangential distortion correction are expressed as: in, is the horizontal coordinate after tangential correction, is the ordinate after tangential correction, is the tangential distortion coefficient.

7. The method for detecting quality of metal sheet production process based on image processing according to claim 6 is characterized in that: After acquiring the image data of the production workshop, the process further includes denoising the sheet material image data, specifically: For pixels in the sheet image , filter the image with a 3×3 template and get the current pixel after filtering It is expressed as: After processing Sort the pixels in ascending order to generate a monotonically increasing two-dimensional sequence. Let ϑ be a two-dimensional template, and the median filter output is: 。 8. The method for quality inspection of metal sheet production process based on image processing according to claim 1 is characterized in that: Before inputting the sensor data and the sheet material image data into the sheet material quality detection model, the sheet material quality detection model is trained, specifically: Extracting an adaptive dictionary of sample data by using a K-SVD method, including: setting K-SVD algorithm parameters, the number of iterations and the sparsity of the OMP algorithm during the iteration process, initializing the adaptive dictionary and the number of atoms in the adaptive dictionary; The OMP algorithm is used to solve the coefficient matrix; Randomize dictionary atomic numbers; Find the non-zero elements in the coefficient matrix and record the positions of the non-zero elements; If there are no non-zero elements, calculate the error matrix and record the column with the maximum error; Normalize the column with the maximum error as the atom of the adaptive dictionary; Iterate and update until the preset requirements are met; The sample data is sparsely encoded using orthogonal matching pursuit according to the adaptive dictionary, and the sparse encoding is used as the input of the sheet metal quality detection model.

9. The hardware sheet material production process quality inspection system based on image processing is characterized by: include: The sensor data acquisition module is used to obtain the sensor data generated by the digital twin workshop; An image data acquisition module, used for acquiring sheet material image data; A data transmission module, used for inputting the sensor data and the sheet material image data into a sheet material quality detection model; The data processing module is used for the sheet material quality detection model to output the sheet material quality judgment result.

10. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store computer program codes, wherein the computer program codes include computer instructions. When the processor executes the computer instructions, the electronic device executes the metal sheet production process quality detection method based on image processing as described in any one of claims 1 to 8.

Images