A micro-service-based silicon steel sheet visual quality detection method and experimental system

Abstract

The application discloses a kind of silicon steel sheet visual quality detection method and experimental system based on microservice, including the following steps;Step 1, build remote service management platform based on microservice, meet the user's custom storage needs through database interaction API, namely design corresponding data storage structure and convenient and fast data interaction API;Step 2, offline acquisition industrial area camera's checkerboard image and defect silicon steel sheet sample image, calculate the row and column scale factor between pixel and actual size;Step 3, offline training model and deployment model;Using the method of deep neural network learning, build PyTorch deep learning framework, use YOLOV7 model to carry out visual quality detection;Step 4, online silicon steel sheet visual quality detection experiment;Step 5, visual quality detection result display.The application carries out visual quality detection experiment by machine vision software, with high degree of automation, wide measurement range, fast, high precision, the characteristics of accurate detection.

Description

Technical Field

[0001] This invention relates to the field of visual quality inspection technology for silicon steel sheets, and specifically to a microservice-based visual quality inspection method and experimental system for silicon steel sheets. Background Technology

[0002] Silicon steel sheets are a soft magnetic alloy of silicon and iron with extremely low carbon content. They possess characteristics such as high permeability, low coercivity, and high resistivity, and are mainly used in the cores of various transformers, motors, and generators. They are widely used in the electrical, communication, power, and machinery industries. With the rapid development of these industries, the dimensional accuracy requirements for silicon steel sheets in industrial production are becoming increasingly stringent. However, factors such as the production environment and manufacturing processes often lead to various types of defects on the surface of silicon steel sheets, directly affecting their quality, safety, and performance. In terms of dimensional measurement, misalignment during finished product stacking affects core performance. The current industrial method of manually aligning and reading measurements using a measuring platform is inefficient, costly, and requires highly experienced and skilled workers, making it difficult to meet the accuracy and efficiency requirements of modern industrial production.

[0003] In surface defect detection, traditional manual visual inspection suffers from high labor costs, low efficiency, strong subjectivity, and limitations. While machine learning methods, which have become increasingly popular in recent years, have achieved good results in defect classification, the significant differences between different target recognition tasks mean that it is often difficult to directly transfer an algorithm model to another task. New algorithms need to be designed to complete new detection tasks, resulting in a large workload. Manual feature extraction methods cannot meet the needs of different tasks.

[0004] In recent years, industrial configuration software has been widely used in actual production environments and various scientific research projects both at home and abroad, achieving good social benefits. At the same time, many domestic companies have also launched a number of small machine vision software programs. Various software programs are flourishing, but their functions are chaotic, resulting in a certain waste of resources.

[0005] Traditional monolithic architectures are still widely used in industry. They are easy to deploy, but any change can have far-reaching consequences. Even a small change to an application or deep learning model requires the entire monolithic application to be rebuilt and redeployed. Microservices, on the other hand, are a distributed architecture framework for building applications. They solve many problems faced by traditional software development, such as: high code duplication, large and difficult-to-maintain codebase, inability to iterate quickly, high testing costs, poor scalability, poor reliability, and high dependencies between modules. Summary of the Invention

[0006] To overcome the shortcomings of the existing technologies, this invention proposes a microservice-based visual quality inspection method and experimental system for silicon steel sheets. The system utilizes a platform architecture that enables remote connection between a remote microserver and the machine vision software MicroVT. The visual quality inspection algorithm model is deployed on the microservice, and the relevant algorithm libraries are integrated into the configuration vision software. Visual quality inspection experiments are conducted using the machine vision software. This method features high automation, a wide measurement range, high speed, high accuracy, and precise detection, meeting the experimental requirements of the machine vision field and possessing the potential for practical application in industrial applications.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A microservice-based visual quality inspection method for silicon steel sheets includes the following steps;

[0009] Step 1: Build a remote service management platform based on microservices. Use Spring Cloud as the main microservice framework, use the Sidecar component to encapsulate and communicate with remote services, select MySQL database as the platform, and establish the connection between heterogeneous service configuration information and user interaction data; realize data management of each heterogeneous service, and meet the user's custom storage needs through database interaction API, that is, design the corresponding data storage structure and convenient and fast data interaction API.

[0010] Step 2: Offline acquisition of checkerboard images and defective silicon steel sheet sample images from industrial area array cameras; corner detection for checkerboard images; calculation of intrinsic and extrinsic parameter matrices; correction of the checkerboard; corner detection for the corrected checkerboard; calculation of the row and column scaling factor between pixels and actual size.

[0011] Step 3: Offline model training and deployment; using deep neural network learning methods, build the PyTorch deep learning framework and use the YOLOv7 model for visual quality detection;

[0012] Step 4: Conduct visual quality inspection experiments on the online silicon steel sheets;

[0013] Step 5: Visual quality test results are displayed.

[0014] The construction of the remote service management platform for microservices in Step 1 consists of the following steps:

[0015] Step 1.1: Image acquisition is performed using an industrial area scan camera and a ring light source; the acquired images are used as input for subsequent visual quality inspection. The industrial area scan camera is used to acquire images through machine vision configuration software, and the ring light source is used to provide illumination. It is stable and suitable for detecting defects on highly reflective or uneven surfaces.

[0016] Step 1.2: The remote unsupervised deep model invocation module enables users to initiate unsupervised deep model invocations to remote servers through simple operations on the software side; the remote unsupervised deep model invocation module obtains the user's instruction information and accurately forwards the instructions to the server to execute the corresponding heterogeneous service program; at the same time, the remote unsupervised deep model invocation module needs to meet the high load capacity, scalability and availability of the entire system and meet the load pressure brought by multiple user accesses;

[0017] Step 1.3: The data storage module stores the image data obtained in Step 1.1 and the instruction information obtained in Step 1.2, which are transmitted from the user to the remote end. It also stores various types of data generated during the operation of the remote service. The data interaction module needs to provide API data interfaces for both the user end and the remote end, so that various types of data in the storage model can be accessed and obtained by the user and the remote service.

[0018] Step 2 specifically involves:

[0019] The checkerboard pattern is captured using the area scan camera installed on the visual quality inspection platform to obtain an initial calibration photo set. Then, the corner points of each checkerboard pattern photo are detected sequentially. The pixel coordinates of the detected corner points in the image are then paired with the corresponding corner point coordinates in the world coordinate system, and the intrinsic parameter matrix and distortion coefficient matrix are solved using the least squares method. The offline parameters are written into an ini file, which includes the camera calibration parameters of the industrial area scan camera 2, coordinate transformation relationships, and the row and column scaling factors between pixels and actual size.

[0020] In step 3, the defects detected by visual quality inspection include six types of defects: scratches, oxidized oil stains, spots, dents and bumps, curling, and wrinkles; a deep neural network learning method is used to distinguish and identify the different types of defects.

[0021] The defective silicon steel sheet sample images obtained in step 2 are used as the defect sample dataset. The collected sample dataset is augmented by operations such as random rotation, scaling, horizontal or vertical flipping, and changing the image brightness, saturation, and contrast to improve the model's generalization ability and make it more robust. The sample dataset is divided into training set, validation set, and test set in a 3:1:1 ratio, and the defects in the training set are labeled using the labelimg annotation tool. The offline trained model is deployed on the remote microservice platform in step 1 for online visual quality inspection of silicon steel sheets.

[0022] Step 4 specifically involves conducting an experiment using machine vision configuration software. A high-resolution industrial area scan camera is used to acquire images of the silicon steel sheet under test on the silicon steel sheet visual inspection platform, obtaining complete and clear images of the silicon steel sheet. These image data are then uploaded to the database of the remote microservice. The acquired silicon steel sheet images are preprocessed using image processing algorithms integrated into MicroVT. A silicon steel sheet visual quality detection algorithm is then used to accurately locate the four corner points of the silicon steel sheet. Combined with the acquired offline parameters, the coordinates of each corner point are calibrated and distances are calculated. Finally, the dimensions of the four sides of the silicon steel sheet are calculated online. Simultaneously, the machine vision software sends a request to the microservice for online monitoring, reads the data uploaded to the database, uses the model trained in step 3 for inference, saves the results to the database, and then the microservice sends the results back to MicroVT.

[0023] In step 4, the visual quality detection algorithm is divided into a size measurement part and a defect detection part;

[0024] The algorithm for dimensional measurement is encapsulated. In online mode, the offline parameters are first read from the ini file written in step 2, then the images of silicon steel sheets acquired online by the array camera are processed, and finally the relevant dimensional information of the silicon steel sheets is calculated by combining the offline parameters.

[0025] The specific process for the size measurement is as follows:

[0026] Step 1: Image preprocessing;

[0027] (1) Grayscale conversion; The high-resolution image acquired in step 4 is converted to grayscale.

[0028] (2) Filtering and noise reduction; Gaussian filtering method is selected for noise reduction. A 5×5 normal kernel is used to assign different weights to pixels at different positions, and the weighted average value is used as the pixel value of the center pixel; Gaussian filtering uses the Gaussian kernel function to smooth the image;

[0029] (3) Threshold segmentation; The Otsu method is selected to perform threshold segmentation on the preprocessed grayscale image, dividing the grayscale values... Figure 2 Value-based;

[0030] The Otsu method consists of the following four steps:

[0031] First, a grayscale histogram is plotted. The image to be processed is converted to grayscale, and the number of pixels at each grayscale level is counted. Then, the inter-class variance is calculated. For each possible threshold, the image is divided into two classes (foreground and background) based on the grayscale histogram, and the inter-class variance is calculated. The inter-class variance represents the degree of difference between different classes, and the largest inter-class variance corresponds to the optimal threshold. Next, the optimal threshold is found. All possible thresholds are iterated, the inter-class variance is calculated, and the threshold that maximizes the inter-class variance is selected as the optimal threshold. Finally, the optimal threshold is applied. The original image is thresholded using the optimal threshold, dividing the pixels into foreground and background categories.

[0032] (4) Edge extraction; After threshold segmentation is completed, the Canny operator is used to extract the edges;

[0033] Step 2: Hough Line Feature Detection; Based on the features extracted from the measurement object, perform line detection; use the Hough line detection method to extract line features; the specific method is as follows:

[0034] (1) Divide the preprocessed and reduced image into four parts: upper left, lower left, upper right, and lower right. Each part of the image after cutting has only two edges in the horizontal and vertical directions. Perform two Hough transformations in the horizontal direction (-45°~45°) and the vertical direction (55°~-125°) to find two edges. After finding two straight lines, the two straight lines determine an intersection point. Use this intersection point as the coarse extraction corner point for the second-level Hough detection.

[0035] (2) The obtained corner points are transformed to the unreduced image through coordinate transformation. In the ROC region near the coarse corner points, horizontal and vertical straight lines are detected by Hough detection. The second-level Hough detection operation is performed in the four parts of the image to find the four precise corner points respectively.

[0036] (3) Perform coordinate transformation to transform it into the unsegmented original image to obtain the final accurate corner pixel coordinates used to calculate the size;

[0037] Step 3: Dimension Calculation; After obtaining the four precise corner points of the workpiece (top left, bottom left, top right, and bottom right), use the pixel coordinates of the four corner points and the row and column scaling factor calculated by the camera calibration part to calculate the Euclidean distance between the pixels, which is the side length of the silicon steel panel.

[0038]

[0039] Where Δd is the actual distance between the two corner points;

[0040] The defect detection section is as follows:

[0041] The defect detection model is deployed as follows: After image acquisition, the data interaction API locates the corresponding data storage model based on the remote service information provided by the user, and stores the acquired images for later use. Next, the user sends an execution command to the remote service from the software. This command is received, recognized, and forwarded by the remote model invocation unit, accurately extracting the corresponding image sequence from the data storage unit and successfully invoking the target heterogeneous service deployed on the remote server. The image sequence is then input into the end-to-end visual quality detection depth model invocation program. A multi-task convolutional algorithm based on a dichroic reflectance model for joint specular detection and removal effectively handles spatially varying specular highlights.

[0042] Inference is performed using the ONNX runtime inference engine. Various types of data generated during remote program execution can be transferred to the corresponding location in the data storage unit for easy access and viewing by the user. The detection results output by the depth model are automatically transferred to the target folder on the software side and displayed through the software's embedded visualization module. The detection results include selecting defective areas and labeling the defect category and confidence level on the original image. The model adopts a server-side deployment scheme; the model calling program is encapsulated as a dynamically loadable DLL (Dynamic Link Library) on the Windows platform, which can then be imported into the machine vision configuration software for use.

[0043] Step 5 specifically involves: the dimensional measurement result is calculated in step 4, and the defect detection result, obtained from step 4, is sent from the microservice to MicroVT. In the acquired image under test, the defective area is selected, the defect type and confidence level are labeled, and finally, the visual quality inspection result of the silicon steel sheet is displayed in the MicroVT display area of ​​the machine vision configuration software. This includes the dimensional measurement result of the silicon steel sheet, the defect area, the defect type and confidence level, which facilitates viewing the real-time status of the inspected workpiece and performing subsequent processing, thus realizing a complete visual quality inspection process.

[0044] A microservice-based visual quality inspection experimental system for silicon steel sheets includes a visual inspection experimental platform. A ring light source and an industrial area scan camera are mounted at the top center of the platform. When a silicon steel sheet is placed, the industrial area scan camera acquires image data of the silicon steel sheet. The ring light source provides sufficient illumination to ensure that the industrial area scan camera captures bright and clear images. A PC is equipped with the machine vision configuration software MicroVT, which receives the acquired silicon steel sheet images online and performs visual quality inspection of the silicon steel sheet using the software algorithm of MicroVT. Finally, the visual quality inspection results are displayed online in real time through a display module above MicroVT.

[0045] The beneficial effects of this invention are:

[0046] This invention builds a remote service management platform based on microservices, using MySQL as the platform to achieve data management for various heterogeneous services. The platform architecture enables remote connection between a remote microserver and the machine vision software MicroVT. The visual quality inspection algorithm for silicon steel sheets is deployed on the microserver, realizing microservice-based visual quality inspection of silicon steel sheets. Its specific advantages are as follows:

[0047] First, it adopts a distributed microservice architecture. This solves many problems faced by traditional software development, such as high code duplication, large and difficult-to-maintain codebases, inability to iterate quickly, high testing costs, poor scalability, poor reliability, and high dependencies between modules. It achieves modular componentization and deep model componentization, enabling effective orchestration and management of multiple experimental cases, providing greater flexibility, and allowing development using the best and most suitable programming languages ​​and tools. This allows for targeted solutions to specific problems, and represents an innovation in traditional visual quality inspection methods and experimental systems.

[0048] Second: The machine vision configuration software MicroVT is an independently developed machine vision software with independent intellectual property rights. MicroVT modularizes various functions such as camera operation, image reading, file operation, threshold segmentation, image filtering, size measurement, and model calling. It has the function of customizing, combining and expanding vision algorithms, which can be used in different experimental contents. It can interact with remote microservices, and is also responsible for realizing human-computer interaction. It also opens up the independent programming interface of the algorithm library for diverse practical application scenarios.

[0049] Third: The visual quality inspection algorithm for silicon steel sheets adopts a multi-scale approach, which avoids processing high-resolution images and reduces the time required for size measurement.

[0050] Fourth: The visual quality inspection algorithm for silicon steel sheets employs a multi-task convolutional algorithm based on a dichroic reflectance model for joint specular detection and removal. This effectively handles spatially varying specular highlights and improves the image's resistance to lighting interference. It can effectively solve the specular problem on the surface of silicon steel sheets with metallic luster and strong reflective properties in bright-field environments, thus improving the accuracy of surface visual quality inspection. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the experimental system for visual quality inspection of silicon steel sheets based on microservices, as described in this invention.

[0052] Figure 2 This is a flowchart of the microservice call process of the present invention.

[0053] Figure 3This is a flowchart illustrating the implementation of the visual quality detection method of the present invention on a microservice platform.

[0054] Figure 4 This is an example diagram of online dimensional measurement of silicon steel sheets according to the present invention.

[0055] Figure 5 The following are example diagrams illustrating different types of defective silicon steel sheets according to the present invention.

[0056] Figure 6 This is a diagram showing the detection results of the software interface of the present invention. Detailed Implementation

[0057] The present invention will now be described in further detail with reference to the accompanying drawings.

[0058] like Figure 1 As shown, the present invention provides a microservice-based visual quality inspection experimental system for silicon steel sheets, comprising a visual inspection experimental platform 1, an industrial area scan camera 2, a ring light source 3, machine vision configuration software MicroVT, a PC 4, and a silicon steel sheet for inspection 5.

[0059] The visual inspection experimental platform 1 is equipped with a ring light source 3 and an industrial area scan camera 2 at the top center. When the silicon steel sheet 5 is placed, the industrial area scan camera 2 acquires image data of the silicon steel sheet 5. The ring light source 3 is used to provide sufficient illumination to ensure that the industrial area scan camera 2 captures bright and clear images. The PC 4 is equipped with machine vision configuration software MicroVT, which is used to receive the acquired images of the silicon steel sheet 5 online and perform visual quality inspection of the silicon steel sheet 5 through the software algorithm of the machine vision configuration software MicroVT. Finally, the visual quality inspection results are displayed online in real time through the display module above MicroVT.

[0060] A microservice-based visual quality inspection method for silicon steel sheets includes the following steps;

[0061] The entire online visual quality inspection of silicon steel sheets includes four main stages: building a remote microservice platform, configuring the offline environment, packaging the visual quality inspection algorithm and deploying the model, and online inspection.

[0062] 1. Build a remote microservice platform;

[0063] A remote service management platform based on microservice technology was built, using Spring Cloud as the main microservice framework. The Sidecar component was used to encapsulate and communicate with remote services, and MySQL was selected as the platform. A connection relationship was established between heterogeneous service configuration information and user interaction data, thereby realizing data management of various heterogeneous services. A series of database interaction APIs were developed to meet users' custom storage needs. Corresponding data storage structures and convenient and fast data interaction APIs were designed, thus building the communication foundation for the managed call platform of remote services and solving the technical problem of deploying visual quality detection algorithms as remote services in machine vision platforms.

[0064] like Figure 2 This is a flowchart illustrating the microservice call process. Users send call commands to remote target services via the MicroVT API. These commands are received and forwarded by Feign under Spring Cloud Netflix. After recognizing the routing information in the command, Feign forwards it to the proxy container Sidecar using a load balancing strategy. Within the Sidecar, the heterogeneous application connecting to the microservice obtains the target service's route name on the web server. Through the communication strategy between Sidecars, the known route name is used to route to the remote heterogeneous application calling the microservice on the web server. The data storage information corresponding to this route path is retrieved from the MySQL database, allowing data to be retrieved from a specific table. Finally, the retrieved information is used as input data to start running the heterogeneous application. Its runtime data and results are persisted in the MySQL database for user viewing.

[0065] The main work involves building the algorithm framework and visualization platform. The algorithm is required to perform visual quality detection of the target object, and the visualization platform must be able to meet user interaction requirements and display the visual quality detection results, such as... Figure 3 The document demonstrates the overall implementation process of the visual quality inspection method on a microservice platform:

[0066] First, images are acquired through a visual quality inspection platform. After image acquisition, the data interaction API locates the corresponding data storage model based on the remote service information provided by the user, storing the acquired images for later use. Next, the user sends an execution command to the remote service from the software. This command is received, recognized, and forwarded by the remote model invocation unit, accurately extracting the corresponding image sequence from the data storage unit and successfully invoking the target heterogeneous service deployed on the remote server. The image sequence is then passed as input data to the end-to-end visual quality inspection depth model invocation program. Various types of data generated during the remote program execution can be transmitted to the corresponding location in the data storage unit for later viewing by the user. The depth model's output detection results are automatically transferred to the target folder on the software side and can be displayed through the software's embedded visualization module.

[0067] 2. Offline configuration environment. The offline configuration environment includes offline camera calibration and offline model training.

[0068] Offline acquisition of checkerboard images from an industrial area scan camera 2, checkerboard corner detection, calculation of intrinsic and extrinsic parameter matrices, checkerboard correction, corner detection on the corrected checkerboard, and calculation of the row and column scaling factor between pixels and actual size; This involves using the industrial area scan camera 2 mounted on a machine vision platform to capture checkerboard photos, obtaining an initial calibration photo set; then sequentially detecting the corners of each checkerboard photo; finally, pairing the pixel coordinates of the detected corners in the image with the corresponding corner coordinates in the world coordinate system, and using the least squares method to solve for the intrinsic parameter matrix and distortion coefficient matrix.

[0069] The offline calibration method for the industrial area scan camera 2 is as follows:

[0070] (1) Coordinate system transformation. First, one of the purposes of camera calibration is to establish the correspondence between objects from the three-dimensional world to points on the pixel plane. Therefore, four coordinate systems need to be defined: World coordinate system (unit: mm); X W Y W Z W Camera coordinate system (unit: mm): X C X C X C Image coordinate system (unit: mm): x, y and pixel coordinate system (unit: pixels): u, v;

[0071] Secondly, the intrinsic and extrinsic parameters of the industrial area scan camera 2 need to be determined. The intrinsic parameters of the industrial area scan camera 2 represent its optical or geometric parameters, including focal length, scale factor, and distortion parameters. The extrinsic parameters represent the camera's position relative to the external world coordinate system, and are divided into translation and rotation parameters. The intrinsic parameter matrix M is a matrix that directly connects the pixel coordinate system and the camera coordinate system. Because the values ​​within the matrix are only related to the camera's intrinsic parameters and do not change with the object's position, it is called the intrinsic parameter matrix.

[0072]

[0073] Assuming the pixel coordinates of the optical center O of the pixel coordinate plane are (u0, v0), the intrinsic parameter matrix M can be written as:

[0074]

[0075] In the formula: f x —Pixel value per millimeter along the x-axis / mm·pixels - 1; f y —Pixel value per millimeter in the y-axis direction / mm·pixels-1.

[0076] The extrinsic parameter matrix is ​​the matrix that links the world coordinate system and the camera coordinate system, as shown in equation (3-3). The extrinsic parameter matrix is ​​the same for all pixels in a photograph, but it changes whenever the calibration plate moves. In other words, the extrinsic parameter matrix is ​​different depending on the position of the calibration plate.

[0077]

[0078] In the formula: R—rotation matrix; T—translation vector; [R|T]—external parameter matrix.

[0079] Then, the world coordinate system is transformed into the camera coordinate system through rigid body transformation, then projected onto the image coordinate system through perspective, and finally transformed into the pixel coordinate system through a second transformation. It is known that the transformation factor for projecting the object from the world system to the pixel system is the camera's intrinsic and extrinsic parameter matrices.

[0080] Finally, given that the intrinsic parameter matrix connects the pixel coordinate system and the camera coordinate system, while the extrinsic parameter matrix connects the world coordinate system and the camera coordinate system, we can obtain the transformation relationship from the world coordinate system to the pixel coordinate system:

[0081]

[0082] (2) Calculation of row and column scaling factors. First, corner points are detected on the checkerboard image of the calibration set to obtain the pixel coordinates of the corner points. For any two corner points, the pixel coordinates are known, and the distance between the two points in the world frame can be calculated according to the size of the checkerboard. Therefore, the distance relationship equation between the two points can be listed.

[0083]

[0084] In the formula: Δd—world system distance between the two corner points; Δu—pixel series coordinate difference between the two corner points; s r — Row scaling factor; Δv — Difference in pixel row coordinates between two corner points; s c — List the scaling factor.

[0085] Δd, Δu, and Δv are known; there are two unknowns: s r and s c To find the two unknowns, we need to use three corner points to set up two equations and solve them. To reduce the error, we choose three corner points with relatively large distances: the top left corner (corner point 1), the top right corner (corner point 2), and the bottom left corner (corner point 3).

[0086] Using the coordinates of corner points 1, 2, and 3, we can derive the equation:

[0087]

[0088]

[0089] In the formula: Δd ij —World-system distance between corner points i and j; Δx ij —The difference in pixel column coordinates between corner points i and j; Δy ij — The difference in pixel row coordinates between corner points i and j.

[0090] The formula for calculating the row-column scaling factor is obtained:

[0091]

[0092]

[0093] Write the offline parameters into the ini file. The offline parameters include the camera calibration parameters of the area scan camera, coordinate transformation relationships, and the row and column scaling factors between pixels and actual size.

[0094] The model is trained offline. For the defect detection part of the visual quality inspection of silicon steel sheets, a deep learning algorithm is applied to detect, identify, and confirm the defect category.

[0095] The specific process of model training is as follows:

[0096] (1) Data Preprocessing. The defect images used in this study came from two sources: one was images of silicon steel sheet defects from actual production workshops provided by industrial sites, and the other was images taken in the laboratory. Data augmentation was performed on the collected sample datasets, including random rotation, scaling, horizontal or vertical flipping, and adjustments to image brightness, saturation, and contrast to improve the model's generalization ability and robustness. The dataset was divided into training, validation, and test sets in a 3:1:1 ratio, and the defects in the training set were labeled using the labelimg annotation tool.

[0097] (2) Model Selection. The PyTorch framework was chosen for model development. YOLOv5 and EfficientDet were used as control groups to examine the superiority of YOLOv7. The accuracy, recall, and mAP of the three models were compared and analyzed. Observation of the experimental results showed that YOLOv7's detection performance is superior to YOLOv5 and EfficientDet, and YOLOv7 has the advantages of fast convergence and detection speed. The choice of YOLOv7 was a trade-off between model training time and training accuracy. GPUs were used to accelerate model training. During training, important parameters such as epoch, batch size, and learning rate were continuously adjusted to maximize defect detection accuracy.

[0098] (3) Model training. The YOLOv7 training process mainly includes two stages: forward propagation and backward propagation.

[0099] In the forward propagation phase, the network's weight parameters are first initialized. Their values ​​cannot all be zero; otherwise, during the first forward propagation, all hidden layer activation values ​​will be identical, leading to identical weight updates during backpropagation and a lack of differentiation among hidden neurons. Initialization determines the starting point for model training. Good initialization helps address the vanishing and exploding gradient problems, thereby accelerating model convergence and improving performance. Next, the raw data enters the network through the input layer, propagating forward through convolutional layers, pooling layers, upsampling, and concatenation. The extracted features are then classified, and the input to each layer and unit throughout the process is calculated. Finally, the error between the network's output and the target value is calculated.

[0100] In the backpropagation phase, the loss function is first used to calculate the error between the network's actual output value and the expected target value, and this error is propagated in the reverse direction of the network, calculating the contribution of each weight to the model error. Starting from the last layer (output layer), the error of each layer and the partial derivatives of the parameters (weights and biases) of each layer are calculated. The error gradient can be calculated in the next step using the chain rule. Next, based on the error gradient and the learning rate, each parameter is updated to reduce the error. The learning rate controls the step size of each parameter update, and is generally chosen to be a very small value to ensure model stability. Finally, the above steps are repeated until the model converges; that is, when the model's error decreases to the expected level, training is considered complete. The model is then deployed on a remote microservice platform.

[0101] 3. Visual quality detection algorithm encapsulation and model deployment.

[0102] The visual quality detection algorithm of this invention is divided into a size measurement part and a defect detection part.

[0103] The dimensional measurement algorithm is encapsulated. In online mode, offline parameters are first read from the ini file, then the images of silicon steel sheets acquired online by the array camera are processed, and finally, the relevant dimensional information of the silicon steel sheets can be calculated efficiently, quickly, and accurately by combining the offline parameters. The specific process is as follows:

[0104] Step 1: Image Preprocessing. Preprocessing refers to a series of preprocessing operations performed on the original image to reduce noise, enhance image features, and provide better input for subsequent processing steps.

[0105] (1) Grayscale Conversion. The high-resolution images acquired are converted to grayscale. The processed images do not lose important edge contour information, and the image processing requires less computing resources and is faster. Therefore, this invention first performs grayscale conversion to improve the processing speed of the entire system.

[0106] (2) Noise Reduction Filtering. This invention selects Gaussian filtering for noise reduction. A 5×5 normal kernel is used to assign different weights to pixels at different locations, and the weighted average value is used as the pixel value of the center pixel. Gaussian filtering uses a Gaussian kernel function to smooth the image. Gaussian filtering can effectively remove Gaussian noise and preserve the edge information of the image during the smoothing process. After Gaussian filtering, the burrs around the silicon steel plate and the defects on the surface of the metal plate are blurred, which lays the groundwork for edge detection and improves the accuracy of edge detection.

[0107] (3) Threshold segmentation. The Otsu method is selected to perform threshold segmentation on the preprocessed grayscale image, dividing the grayscale values... Figure 2 Value-based.

[0108] The Otsu method can be divided into the following four steps:

[0109] First, a grayscale histogram is calculated. The image to be processed is converted to grayscale, and the number of pixels at each grayscale level is counted. Then, the inter-class variance is calculated. For each possible threshold, the image is divided into two classes (foreground and background) based on the grayscale histogram, and the inter-class variance is calculated. The inter-class variance represents the degree of difference between different classes, and the largest inter-class variance corresponds to the optimal threshold. Next, the optimal threshold is found. All possible thresholds are iterated, the inter-class variance is calculated, and the threshold that maximizes the inter-class variance is selected as the optimal threshold. Finally, the optimal threshold is applied. The original image is thresholded using the optimal threshold, dividing the pixels into foreground and background categories. The advantage of Otsu's method is its simplicity and ease of implementation, and it performs well for many image segmentation problems.

[0110] (4) Edge Extraction. After thresholding, it is necessary to further extract the edges of the binary image. Edge extraction utilizes the gray-level abrupt changes of the edges as features for segmentation. This invention uses the Canny operator to extract edges. The Canny operator has advantages in edge detection such as low error rate, accurate localization, multi-threshold processing, high sensitivity, and suppression of non-maximum values. The edges extracted using this method not only have good continuity and accuracy, but also highlight important features in the image.

[0111] Step Two: Hough Line Feature Detection. This invention targets a quadrilateral silicon steel plate, so the features to be extracted are the four sides of the silicon steel plate, requiring line detection. This study uses the Hough line detection method to extract line features. The specific method is as follows:

[0112] (1) The preprocessed and reduced image is divided into four parts: upper left, lower left, upper right, and lower right. Each part of the image after segmentation has only two edges in the horizontal and vertical directions. Therefore, when searching for straight lines, it is not necessary to traverse 0 to 180°. It is only necessary to perform two Hough transforms in the horizontal direction (-45° to 45°) and the vertical direction (55° to -125°) to find the two edges. After finding the two straight lines, the two straight lines can determine an intersection point. This intersection point is used as the coarse corner extraction point for the second-level Hough detection.

[0113] (2) The obtained corner points are transformed to the unreduced image by coordinate transformation. Horizontal and vertical straight lines are detected by Hough detection in the ROC region near the coarse corner points. The second-level Hough detection operation is performed in the four parts of the image in sequence to find the four precise corner points.

[0114] (3) The obtained precise corner pixel coordinates are relative to the ROC region of each part. The coordinate transformation is performed to transform them into the unsegmented original image to obtain the final precise corner pixel coordinates used to calculate the size.

[0115] Step 3: Dimension Calculation. After obtaining the four precise corner points of the workpiece (top left, bottom left, top right, and bottom right), use the pixel coordinates of these four corner points, combined with the row and column scaling factors calculated from the camera calibration, to calculate the Euclidean distance between the pixels, which is the side length of the silicon steel panel.

[0116]

[0117] Where Δd is the actual distance between the two corner points.

[0118] The dimensional measurement algorithm is developed based on C++ and OpenCV libraries. It is packaged into a DLL dynamic link library that can be dynamically loaded under the Windows platform and then imported into machine vision configuration software for use.

[0119] Deployment of the defect detection model.

[0120] After image acquisition, the data interaction API locates the corresponding data storage model based on the remote service information provided by the user, storing the acquired images for later use. Next, the user sends an execution command to the remote service from the software. This command is received, recognized, and forwarded by the remote model invocation unit, accurately extracting the corresponding image sequence from the data storage unit and successfully invoking the target heterogeneous service deployed on the remote server. The image sequence is then passed as input data to the end-to-end visual quality detection depth model invocation program. A multi-task convolutional algorithm based on a dichroic reflectance model for joint specular detection and removal effectively handles spatially varying specular highlights while preserving shadows, resolving the specular highlight problem on the surface of the acquired silicon steel sheet image in a bright-field environment. Inference is performed using the ONNX runtime inference engine. Various types of data generated during remote program execution can be passed to the corresponding locations in the data storage unit for easy viewing by the user. The depth model output detection results are automatically transferred to the target folder on the software side and can be displayed through the software's embedded visualization module. The detection results include selecting the defective area and labeling the defect category and confidence level on the original image. The model adopts a server-side deployment scheme, and the model calling program is packaged as a dynamically loadable DLL dynamic link library under the Windows platform, which can be imported into machine vision configuration software for use.

[0121] 4. Online detection. For example... Figure 4 , 5Examples of online dimensional measurement and defect detection for silicon steel sheets are shown below. By importing images of silicon steel sheets captured on-site by a field-array camera, the silicon steel sheets are inspected using a pre-packaged visual quality inspection algorithm library in the machine vision configuration software. The inspection results are displayed in the display area of ​​the machine vision configuration software, as shown below. Figure 6 As shown, the displayed content includes dimensional measurement results, a bounding box of the defect area, the defect category, and the confidence level.

[0122] MicroVT, a large-scale industrial machine vision configuration software, is a software system platform that combines machine vision and digital image processing. It not only meets the requirements of teaching experiments in the field of machine vision but also has the potential for practical application in industrial fields, possessing a certain degree of originality in the domestic machine vision software field. The microservice platform adopted in this invention is a platform architecture that enables remote connection between deep learning models and machine vision software.

[0123] In summary, this invention addresses the problem of a scarcity of relatively mature machine vision quality inspection experimental systems and a severe lack of complete machine vision experimental cases, including software, platforms, and algorithms, in various educational institutions. Utilizing microservice technology and a distributed architecture framework, it achieves modular componentization, enabling effective orchestration and management of multiple experimental cases, providing greater flexibility, and allowing development using optimal and suitable programming languages ​​and tools. This allows for targeted solutions to specific problems, innovating traditional visual quality inspection methods and systems. The above descriptions are merely specific embodiments of this invention and are intended to explain the principles of the invention, not to limit the scope of protection of the invention. Based on this explanation, those skilled in the art can readily conceive of other specific embodiments of the invention without creative effort, and these structures will all fall within the scope of protection of this invention.

Claims

1. A microservice-based visual quality inspection method for silicon steel sheets, characterized in that, Includes the following steps; Step 1: Build a remote service management platform based on microservices. Spring Cloud is used as the main microservice framework. The Sidecar component is used to complete the encapsulation and communication of remote services. MySQL database is selected as the platform, and the connection relationship between heterogeneous service configuration information and user interaction data is established. To achieve data management for various heterogeneous services, and to meet users’ custom storage needs through database interaction APIs, namely, to design corresponding data storage structures and convenient and fast data interaction APIs; Step 2: Offline acquisition of checkerboard images and defective silicon steel sheet sample images from industrial area array camera (2), corner detection for checkerboard images, calculation of internal and external parameter matrices, correction of checkerboard, corner detection for the corrected checkerboard, and calculation of the row and column scaling factor between pixels and actual size. Step 3: Offline model training and deployment; using deep neural network learning methods, build the PyTorch deep learning framework and use the YOLOv7 model for visual quality detection; Step 4: Conduct visual quality inspection experiments on the online silicon steel sheets; Step 5: Visual quality test results are displayed.

2. The method for visual quality inspection of silicon steel sheets based on microservices according to claim 1, characterized in that, The construction of the remote service management platform for microservices in step 1 consists of the following steps: Step 1.1: Image acquisition is performed using an industrial area scan camera (2) and a ring light source (3); the acquired images are used for subsequent visual quality inspection, wherein the industrial area scan camera (2) is used to acquire images through machine vision configuration software, and the ring light source (3) is used to provide illumination; Step 1.2: Obtain user instruction information through the remote unsupervised deep model calling module, and accurately forward the instructions to the server to execute the corresponding heterogeneous service program; Step 1.3: The data storage module stores the image data obtained in Step 1.1 and the instruction information obtained in Step 1.2, which are transmitted from the user to the remote end. It also stores various types of data generated during the operation of the remote service. The data interaction module needs to provide API data interfaces for both the user end and the remote end, so that various types of data in the storage model can be accessed and obtained by the user and the remote service.

3. The method for visual quality inspection of silicon steel sheets based on microservices according to claim 1, characterized in that, Step 2 specifically involves: Using the industrial area scan camera (2) installed on the visual quality inspection platform, chessboard photos were taken to obtain an initial calibration photo set; then, the corner points of each chessboard photo were detected in turn; then, the pixel coordinates of the detected corner points in the image were paired with the corresponding corner coordinates in the world coordinate system, and the intrinsic parameter matrix and distortion coefficient matrix were solved using the least squares method; the offline parameters were written into the ini file, and the offline parameters included the camera calibration parameters of the industrial area scan camera (2), coordinate transformation relationship, and row and column scaling factor between pixels and actual size.

4. The microservice-based visual quality inspection method for silicon steel sheets according to claim 3, characterized in that, In step 3, the defects detected by visual quality inspection include six types of defects: scratches, oxidized oil stains, spots, dents and bumps, curling, and wrinkles; a deep neural network learning method is used to distinguish and identify the different types of defects. The defective silicon steel sheet sample images obtained in step 2 are used as the defect sample dataset. The collected sample dataset is augmented by random rotation, scaling, horizontal or vertical flipping, and changes in image brightness, saturation, and contrast. The sample dataset is divided into training set, validation set, and test set. The defects in the training set are labeled using the labelimg annotation tool. The offline trained model is deployed on the remote microservice platform in step 1 for online visual quality inspection of silicon steel sheets.

5. The microservice-based visual quality inspection method for silicon steel sheets according to claim 4, characterized in that, Step 4 specifically involves: conducting an experiment using machine vision configuration software, calling a high-resolution industrial area array camera (2) to acquire images of the silicon steel sheet to be tested on the silicon steel sheet visual inspection platform, obtaining complete and clear images of the silicon steel sheet, and uploading the image data to the database of the remote microservice; then using the image processing algorithm integrated on MicroVT to preprocess the acquired silicon steel sheet images, and then using the silicon steel sheet visual quality detection algorithm to detect and accurately locate the four corner points of the silicon steel sheet; combining the acquired offline parameters to calibrate the coordinates and calculate the distance of each corner point, and finally calculate the size data of the four sides of the silicon steel sheet online; at the same time, using machine vision software to send a request to the microservice to perform online listening service, read the data uploaded to the database, use the model trained in step 3 for inference, save the results to the database, and then the microservice sends the results to MicroVT.

6. The method for visual quality inspection of silicon steel sheets based on microservices according to claim 5, characterized in that, In step 4, the visual quality detection algorithm is divided into a size measurement part and a defect detection part; The algorithm for dimensional measurement is encapsulated. In online mode, the offline parameters are first read from the ini file written in step 2, then the images of silicon steel sheets acquired online by the array camera are processed, and finally the relevant dimensional information of the silicon steel sheets is calculated by combining the offline parameters.

7. The microservice-based visual quality inspection method for silicon steel sheets according to claim 6, characterized in that, The specific process for the size measurement is as follows: Step 1: Image preprocessing; (1) Grayscale conversion; The high-resolution image acquired in step 4 is converted to grayscale. (2) Filtering and noise reduction; Gaussian filtering method is selected for noise reduction. A 5×5 normal kernel is used to assign different weights to pixels at different positions, and the weighted average value is used as the pixel value of the center pixel; Gaussian filtering uses the Gaussian kernel function to smooth the image; (3) Threshold segmentation; The Otsu method is selected to perform threshold segmentation on the grayscale image obtained after preprocessing, and the grayscale image is binarized. (4) Edge extraction; After threshold segmentation is completed, the Canny operator is used to extract the edges; Step 2: Hough Line Feature Detection; Based on the features extracted from the measurement object, perform line detection; use the Hough line detection method to extract line features; the specific method is as follows: (1) Divide the pre-processed and reduced image into four parts: upper left, lower left, upper right, and lower right. Each part of the image after cutting has only two sides in the horizontal and vertical directions. Perform two Hough transformations in the horizontal and vertical directions respectively to find two sides. After finding two straight lines, the two straight lines determine an intersection point. Use this intersection point as the coarse extraction corner point for the second-level Hough detection. (2) The obtained corner points are transformed to the unreduced image through coordinate transformation. In the ROC region near the coarse corner points, horizontal and vertical straight lines are detected by Hough detection. The second-level Hough detection operation is performed in the four parts of the image to find the four precise corner points respectively. (3) Perform coordinate transformation to transform it into the unsegmented original image to obtain the final accurate corner pixel coordinates used to calculate the size; Step 3: Dimension Calculation; After obtaining the four precise corner points of the workpiece (top left, bottom left, top right, and bottom right), the Euclidean distance between the pixels is calculated using the pixel coordinates of the four corner points and the row and column scaling factor calculated by the camera calibration part. This is the side length of the silicon steel panel. Where Δd is the actual distance between the two corner points; The Otsu method in step one consists of the following four steps: First, a grayscale histogram is plotted. The image to be processed is converted to grayscale, and the number of pixels at each grayscale level is counted. Then, the inter-class variance is calculated. For each possible threshold, the image is divided into two classes based on the grayscale histogram, and the inter-class variance is calculated. The inter-class variance represents the degree of difference between different classes, and the largest inter-class variance corresponds to the optimal threshold. Next, the optimal threshold is found. All possible thresholds are iterated, the inter-class variance is calculated, and the threshold that maximizes the inter-class variance is selected as the optimal threshold. Finally, the optimal threshold is applied. The original image is thresholded using the optimal threshold, dividing the pixels into foreground and background categories.

8. The method for visual quality inspection of silicon steel sheets based on microservices according to claim 6, characterized in that, The defect detection section is as follows: Defect detection model deployment: After image acquisition is completed, the data interaction API will find the corresponding data storage model based on the remote service information provided by the user, and store the acquired images for later use. Then, the user sends the execution command of the remote service from the software. This command will be received, recognized and forwarded by the remote model calling unit, so as to accurately extract the corresponding image sequence from the corresponding data storage unit and successfully call the target heterogeneous service deployed on the remote server, and pass the image sequence as input data to the end-to-end visual quality detection depth model calling program. A multi-task convolutional algorithm based on a dichroic reflection model for joint specular detection and removal is used to effectively handle spatially varied specular highlights. Inference is performed using the ONNX runtime inference engine. All types of data generated during the execution of the remote program can be transferred to the corresponding location in the data storage unit for easy reading and viewing by the user later. The detection results output by the depth model are automatically transmitted to the target folder on the software side and displayed through the software's embedded visualization module. The detection results include selecting the defective area and annotating the defect category and confidence level on the original image. The model adopts a server-side deployment scheme, and the model calling program is packaged into a dynamically loadable DLL dynamic link library under the Windows platform, which can be imported into the machine vision configuration software for use.

9. The microservice-based visual quality inspection method for silicon steel sheets according to claim 6, characterized in that, Step 5 specifically involves: the dimensional measurement result is calculated in step 4, and the defect detection result, obtained from step 4, is sent from the microservice to MicroVT. In the acquired image under test, the defective area is selected, the defect type and confidence level are labeled, and finally, the visual quality inspection result of the silicon steel sheet is displayed in the MicroVT display area of ​​the machine vision configuration software. This includes the dimensional measurement result of the silicon steel sheet, the defect area, the defect type and confidence level, which facilitates viewing the real-time status of the inspected workpiece and performing subsequent processing, thus realizing a complete visual quality inspection process.

10. A visual quality inspection experimental system for implementing the microservice-based visual quality inspection method for silicon steel sheets according to any one of claims 1-9, characterized in that, The system includes a visual inspection experimental platform (1), which is equipped with a ring light source (3) and an industrial area array camera (2) in the center. When a silicon steel sheet (5) is placed, the industrial area array camera (2) acquires the image data of the silicon steel sheet (5). The ring light source (3) is used to provide sufficient illumination to ensure that the industrial area array camera (2) captures a bright and clear image. The PC (4) is equipped with the machine vision configuration software MicroVT, which is used to receive the collected images of the silicon steel sheet (5) online and to perform visual quality inspection of the silicon steel sheet (5) through the software algorithm of the machine vision configuration software MicroVT. Finally, the visual quality inspection results are displayed online in real time through the display module above MicroVT.

Images