A method and device for correcting lateral jitter of high-speed wire images

Abstract

The present invention relates to the field of intelligent manufacturing technology, and in particular to a method and device for correcting lateral jitter of a high-speed wire rod image. The method comprises: obtaining a high-speed wire rod training sample, the high-speed wire rod training sample including a high-speed wire rod sample image and a corresponding true value boundary point position; training a boundary recognition model to be trained based on the high-speed wire rod sample image and the corresponding true value boundary point position to obtain a trained boundary recognition model; obtaining a high-speed wire rod original image, inputting the high-speed wire rod original image into the trained boundary recognition model to obtain boundary line position information on the left and right sides of the high-speed wire rod original image; and performing jitter correction on the high-speed wire rod original image based on the boundary line position information on the left and right sides of the high-speed wire rod original image. By adopting the present invention, the lateral jitter of the high-speed wire rod original image can be processed, so that the processed high-speed wire rod side view image is more stable and clear.

Description

Technical Field

[0001] The present invention relates to the field of intelligent manufacturing technology, and in particular to a method and device for correcting lateral jitter of a high-speed wire image. Background Art

[0002] In the field of steel production, wire rod is the product with the smallest cross-sectional dimensions among hot-rolled steel sections. Depending on the rolling mill, wire rod can be divided into two types: high-speed wire rod (high-speed wire) and ordinary wire rod (ordinary wire). High-speed wire rod has the characteristics of fast production speed and high furnace temperature, but traditional manual quality supervision methods often have difficulty in accurately detecting defects. In recent years, deep learning technology has become a mainstream method in the field of computer vision due to its powerful feature extraction capabilities. The use of deep learning methods for online detection of steel defects has the advantages of monitoring implementation, good stability, fast speed, and high accuracy. However, the accuracy of the defect detection method based on deep learning is closely related to the reliability of the data. Therefore, clear and reliable data can greatly improve the accuracy of detection.

[0003] High-speed wire production workshops typically have high temperatures and strong vibration environments, which leads to image information redundancy and noise interference in images taken with high-speed cameras. Among them, lateral jitter in the image causes severe defect distortion, which seriously affects the accuracy of defect detection and imaging effects. Summary of the Invention

[0004] In order to solve the problem that high-speed wire rod images have severe lateral jitter and background noise, which cannot meet the requirements of subsequent defect detection, the embodiment of the present invention provides a method and device for correcting lateral jitter of high-speed wire rod images. The technical solution is as follows:

[0005] In one aspect, a method for correcting lateral jitter of a high-speed wire image is provided. The method is implemented by an electronic device and includes:

[0006] S1. Obtain a high-speed wire rod training sample, wherein the high-speed wire rod training sample includes a high-speed wire rod sample image and a corresponding true value boundary point position;

[0007] S2. Training the boundary recognition model to be trained based on the high-speed wire sample image and the corresponding true boundary point positions to obtain a trained boundary recognition model;

[0008] S3. Obtain an original image of the high-speed wire rod, input the original image of the high-speed wire rod into the trained boundary recognition model, and obtain the boundary line position information on the left and right sides of the original image of the high-speed wire rod;

[0009] S4. Performing jitter correction on the high-speed wire rod original image according to the boundary line position information on the left and right sides of the high-speed wire rod original image.

[0010] Optionally, the boundary recognition model includes a feature extraction module, a classification recognition branch module, and a regression recognition branch module;

[0011] The classification and recognition branch module includes three fully connected layers;

[0012] The regression recognition branch module includes multi-layer convolutional layer and pooling layer processing.

[0013] Optionally, the step S2 of training a boundary recognition model to be trained based on the high-speed wire sample image and the corresponding true boundary point positions to obtain a trained boundary recognition model includes:

[0014] S21, inputting the high-speed wire sample image into a feature extraction module of a boundary recognition model to be trained to obtain feature information of the high-speed wire sample image;

[0015] S22, input the feature information of the high-speed wire sample image into the classification and recognition branch module to be trained, and perform calculations through three fully connected layers. The last fully connected layer outputs a discrimination result, and the classification result is calculated by applying the following formula (1) of the mean square error (MSE) loss function:

[0016]

[0017] Among them, k represents the number of pixels in each row, y′ i represents the predicted label of the i-th pixel, y i represents the true label of the i-th pixel;

[0018] S23, inputting the feature information of the high-speed wire sample image into the regression recognition branch module to be trained, outputting the predicted boundary point position of the high-speed wire sample image, and performing loss calculation on the regression recognition result according to the predicted boundary point coordinates, the true boundary point position corresponding to the high-speed wire sample image, and the following formula (2) of the OKS loss function;

[0019]

[0020] Among them, x left Represents the left boundary horizontal coordinate of the predicted boundary point position, x right Represents the right boundary horizontal coordinate of the predicted boundary point position, x gt_left Represents the left boundary horizontal coordinate of the true boundary point position, x gt_right Represents the right boundary abscissa of the true boundary point position, L foreground The length that represents the prospects of high-speed wire.

[0021] Optionally, the step S3 of inputting the high-speed wire rod original image into a trained boundary recognition model to obtain boundary line position information on the left and right sides of the high-speed wire rod original image includes:

[0022] S31, setting i=1, N is the total number of pixel rows of the original image of the high-speed wire rod;

[0023] S32, reading the pixels of the i-th row of the high-speed wire rod original image, obtaining the boundary point position results of the i-th row, connecting the pixels of the i-th row and the boundary point position results of the i-th row, inputting the connection result into the feature extraction module, and obtaining feature information of the pixels of the i-th row;

[0024] S33, inputting the feature information of the pixels in the i-th row into the classification and recognition branch module to obtain a determination result of whether the pixels in the i-th row contain steel;

[0025] S34, inputting the feature information of the pixels in the i-th row into the regression recognition branch module to obtain the position of the boundary point to be determined in the i-th row;

[0026] S35, determining the position of the boundary point in the i-th row according to the judgment result of whether the pixels in the i-th row contain steel and the position of the boundary point to be determined in the i-th row;

[0027] S36. Compare i with N. If i is greater than or equal to N, go to S37. If i is less than N, then i=i+1 and go to S32.

[0028] S37. Determine the boundary line position information on the left and right sides of the high-speed wire rod original image based on the boundary point position information of each row.

[0029] Optionally, when i=1, the boundary point position result of the i-1th row is set to 0.

[0030] Optionally, the step S35 of determining the position of the boundary points in the i-th row according to the judgment result of whether the i-th row of pixels contains steel and the positions of the boundary points to be determined in the i-th row includes:

[0031] If the result of the judgment of whether the pixels in the i-th row contain steel is that there is steel, the position of the boundary point to be determined in the i-th row is determined as the result of the boundary point position in the i-th row;

[0032] If the result of determining whether the pixels in the i-th row contain steel is no steel, the position of the boundary point to be determined in the i-th row is determined to be invalid, and the result of the boundary point position in the i-th row is determined to be 0.

[0033] Optionally, the feature extraction module is feature pyramid extraction, or ResNet-50 feature extraction network, or a combination of feature pyramid extraction and ResNet-50 feature extraction network.

[0034] On the other hand, a high-speed wire rod image lateral jitter correction device is provided, which is applied to a high-speed wire rod image lateral jitter correction method, and the device includes:

[0035] An acquisition module is used to acquire a high-speed wire rod training sample, wherein the high-speed wire rod training sample includes a high-speed wire rod sample image and a corresponding true value boundary point position;

[0036] A training module is used to train the boundary recognition model to be trained based on the high-speed wire sample image and the corresponding true value boundary point position to obtain a trained boundary recognition model;

[0037] A recognition module is used to obtain an original image of the high-speed wire rod, input the original image of the high-speed wire rod into a trained boundary recognition model, and obtain the position information of the boundary lines on the left and right sides of the original image of the high-speed wire rod;

[0038] The correction module is used to perform jitter correction on the high-speed wire rod original image according to the boundary line position information on the left and right sides of the high-speed wire rod original image.

[0039] Optionally, the boundary recognition model includes a feature extraction module, a classification recognition branch module, and a regression recognition branch module;

[0040] The classification and recognition branch module includes three fully connected layers;

[0041] The regression recognition branch module includes multi-layer convolutional layer and pooling layer processing.

[0042] Optionally, the training module is used to:

[0043] S21, inputting the high-speed wire sample image into a feature extraction module of a boundary recognition model to be trained to obtain feature information of the high-speed wire sample image;

[0044] S22, input the feature information of the high-speed wire sample image into the classification and recognition branch module to be trained, and perform calculations through three fully connected layers. The last fully connected layer outputs a discrimination result, and the classification result is calculated by applying the following formula (1) of the mean square error (MSE) loss function:

[0045]

[0046] Among them, k represents the number of pixels in each row, y′ i represents the predicted label of the i-th pixel, yi represents the true label of the i-th pixel;

[0047] S23, inputting the feature information of the high-speed wire sample image into the regression recognition branch module to be trained, outputting the predicted boundary point position of the high-speed wire sample image, and performing loss calculation on the regression recognition result according to the predicted boundary point coordinates, the true boundary point position corresponding to the high-speed wire sample image, and the following formula (2) of the OKS loss function;

[0048]

[0049] Among them, x left Represents the left boundary horizontal coordinate of the predicted boundary point position, x right Represents the right boundary horizontal coordinate of the predicted boundary point position, x gt_left Represents the left boundary horizontal coordinate of the true boundary point position, x gt_right Represents the right boundary abscissa of the true boundary point position, L foreground The length that represents the prospects of high-speed wire.

[0050] On the other hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the above-mentioned high-speed wire image lateral jitter correction method.

[0051] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement the above-mentioned high-speed wire image lateral jitter correction method.

[0052] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0053] In an embodiment of the present invention, a high-speed wire rod training sample is obtained, comprising a high-speed wire rod sample image and the corresponding true boundary point locations. A boundary recognition model to be trained is trained based on the high-speed wire rod sample image and the corresponding true boundary point locations to obtain a trained boundary recognition model. An original high-speed wire rod image is obtained and input into the trained boundary recognition model to obtain information on the boundary line positions on the left and right sides of the original high-speed wire rod image. Based on the boundary line position information on the left and right sides of the original high-speed wire rod image, the original high-speed wire rod image is subjected to jitter correction. The processed image effectively removes lateral jitter, making the processed high-speed wire rod side view image more stable and clear. During the lateral jitter correction process, the harsh background noise in the production environment can also be eliminated, making the corrected image cleaner and more accurate. By removing lateral jitter and reducing background noise, the processed image becomes lightweight, convenient for storage and transmission, and can improve the subsequent defect detection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0055] Figure 1 This is a flow chart of a method for correcting lateral jitter of a high-speed wire image provided by an embodiment of the present invention;

[0056] Figure 2 This is a structural diagram of a boundary recognition model training process provided by an embodiment of the present invention;

[0057] Figure 3 This is a schematic diagram of an original image of a high-speed wire provided by an embodiment of the present invention;

[0058] Figure 4 This is a schematic diagram of the left and right boundary lines of a high-speed wire rod original image provided by an embodiment of the present invention;

[0059] Figure 5 is a schematic diagram of a corrected image provided by an embodiment of the present invention;

[0060] Figure 6 This is a block diagram of a high-speed wire image lateral jitter correction device provided by an embodiment of the present invention;

[0061] Figure 7 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0063] An embodiment of the present invention provides a method for correcting lateral jitter in high-speed wire rod images. The method can be implemented by an electronic device, such as a terminal or a server, equipped with a boundary recognition model. The boundary recognition model comprises at least three components: a feature extraction module, a classification recognition branch module, and a regression recognition branch module. The classification recognition branch module can include at least three fully connected layers, and the regression recognition branch module can include multiple convolutional and pooling layers.

[0064] like Figure 1 The flowchart of the method for correcting the horizontal jitter of a high-speed wire image is shown. The processing flow of the method may include the following steps:

[0065] S1. Obtain a high-speed wire training sample, where the high-speed wire training sample includes a high-speed wire sample image and a corresponding true value boundary point position.

[0066] The true value boundary point position corresponding to the high-speed wire sample image may be the boundary point coordinate value corresponding to each row of pixels of the high-speed wire sample image.

[0067] S2. The boundary recognition model to be trained is trained according to the high-speed wire sample image and the corresponding true value boundary point position to obtain a trained boundary recognition model.

[0068] Alternatively, as Figure 2 In the network structure shown, the feature extraction module can use a 5-layer feature pyramid for feature extraction. The training process of S2 can specifically include the following steps S21-S23:

[0069] S21 . Input the high-speed wire sample image into a feature extraction module of a boundary recognition model to be trained to obtain feature information of the high-speed wire sample image.

[0070] In a feasible implementation, the current row of pixels of the high-speed wire sample image is read, and the predicted boundary point position of the previous row of pixels of the current row is obtained. Both are a row array. The current row of pixels and the predicted boundary point position of the previous row are connected to obtain the connected array.

[0071] The concatenated array is input layer by layer into the 5-layer feature pyramid of the feature extraction module. The output size of each layer of the 5-layer feature pyramid is W represents the width of a row of pixels. The results of each of the last four layers are convolved with 1*1, and the four convolution results are upsampled respectively. The upsampled results are connected, and the size of the feature information obtained is

[0072] Among them, the downsampling operation is the maximum pooling with a kernel size of 1×2 and a stride of 2, and the upsampling operation can be any one of the interpolation methods such as nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation. The kernel size of the convolution operation and deconvolution operation is 1×3. The deconvolution operation can double the size of the feature map and reduce the number of feature maps by 1. In this embodiment, all convolution and deconvolution operations are connected to the linear rectification function (ReLU) by default. During the convolution and deconvolution operations, the feature map boundaries are padded with 0.

[0073] Among them, the 5-layer feature pyramid of the feature extraction module can be feature pyramid extraction, or ResNet-50 feature extraction network, or a combination of feature pyramid extraction and ResNet-50 feature extraction network.

[0074] S22. Input the feature information of the high-speed wire sample image into the classification and recognition branch module to be trained. After calculation through three layers of fully connected layers, the last layer of fully connected layers outputs a discrimination result. The loss of the classification result is calculated by applying the following formula (1) of the mean square error (MSE) loss function:

[0075]

[0076] Among them, k represents the number of pixels in each row, y′ i represents the predicted label of the i-th pixel, y i represents the true label of the i-th pixel.

[0077] In a feasible implementation, the output judgment result may be 0 or 1, where 0 represents the absence of steel and 1 represents the presence of steel.

[0078] S23, inputting the feature information of the high-speed wire sample image into the regression recognition branch module to be trained, outputting the predicted boundary point position of the high-speed wire sample image, and performing loss calculation on the regression recognition result according to the predicted boundary point coordinates, the true boundary point position corresponding to the high-speed wire sample image, and the following formula (2) of the OKS loss function;

[0079]

[0080] Among them, x left Represents the left boundary horizontal coordinate of the predicted boundary point position, x right Represents the right boundary horizontal coordinate of the predicted boundary point position, x gt_leftRepresents the left boundary horizontal coordinate of the true boundary point position, x gt_right Represents the right boundary abscissa of the true boundary point position, L foreground The length that represents the prospects of high-speed wire.

[0081] In one feasible implementation, the feature information output by the feature extraction module is processed through multiple convolutional and pooling layers, outputting a 1*W vector (W is the pixel width of each row), where the value at each position represents the probability of being a boundary. The OKS (Object Keypoint Similarity) loss function is used to calculate the difference between the predicted vector and the true value vector of the actual boundary. The loss is then propagated back to the network through gradient backpropagation for parameter optimization and training.

[0082] It should be noted that the calculation method of the loss function provided above does not constitute a limitation of the present invention. In actual operation, other loss functions can be used to train the network.

[0083] S3. Obtain an original image of the high-speed wire rod, input the original image of the high-speed wire rod into the trained boundary recognition model, and obtain boundary line position information on the left and right sides of the original image of the high-speed wire rod.

[0084] The original image of the high-speed wire refers to the image taken from the side of the wire when it is in motion. Due to the small diameter of the high-speed wire, the gap between the wire and the guide groove, and the harsh production environment, the high-speed wire image often has serious lateral jitter and a lot of background redundant noise information. Figure 3 Shown is an original image of a high-speed wire rod.

[0085] Optionally, S3 of identifying the boundary point position information on the left and right sides of the high-speed wire rod original image through the boundary recognition model may include S31-S37:

[0086] S31 , setting i=1, and N being the total number of pixel rows of the original image of the high-speed wire rod.

[0087] S32. Read the pixels of the i-th row of the original image of the high-speed wire rod, obtain the boundary point position results of the i-1-th row, connect the pixels of the i-th row and the boundary point position results of the i-1-th row, input the connection result into the feature extraction module, and obtain the feature information of the pixels of the i-th row.

[0088] When i=1, the boundary point position result of the i-1th row is set to 0.

[0089] S33. Input the feature information of the pixels in the i-th row into the classification and recognition branch module to obtain a determination result of whether the pixels in the i-th row contain steel.

[0090] S34. Input the feature information of the pixels in the i-th row into the regression recognition branch module to obtain the position of the boundary point to be determined in the i-th row.

[0091] The position of the boundary point to be determined in the i-th row may be a row array encoded with a one-hot code, and specifically, the value of the position that is not a boundary point is 0, and the value of the position that is a boundary point is 1.

[0092] S35. Determine the position of the boundary point in the i-th row according to the judgment result of whether the pixels in the i-th row contain steel and the position of the boundary point to be determined in the i-th row.

[0093] In a feasible implementation, the output result of the classification recognition branch module is used to guide the validity of the output result of the regression recognition branch module. The specific judgment method can be: if the judgment result of whether the pixels in the i-th row contain steel is that there is steel, then the position of the boundary point to be determined in the i-th row is determined as the boundary point position result of the i-th row; if the judgment result of whether the pixels in the i-th row contain steel is that there is no steel, then the position of the boundary point to be determined in the i-th row is determined to be invalid, and the boundary point position result of the i-th row is determined to be 0.

[0094] S36. Compare i with N. If i is greater than or equal to N, go to S37. If i is less than N, then i=i+1, go to S32.

[0095] S37. Determine the boundary line position information on the left and right sides of the high-speed wire rod original image based on the boundary point position information of each row.

[0096] In a feasible implementation, the boundary point position information of each row is spliced ​​together to obtain the boundary line position information of the high-speed wire rod original image, which can be as follows: Figure 4 shown.

[0097] S4. According to the position information of the boundary lines on the left and right sides of the high-speed wire rod original image, the high-speed wire rod original image is subjected to jitter correction.

[0098] In a feasible implementation, the jitter correction of the high-speed wire rod original image is performed based on the boundary line position information, with the goal of aligning the boundary points of each row, and the pixel values ​​of each row are adjusted horizontally. The corrected image can be as follows: Figure 5 shown.

[0099] In an embodiment of the present invention, a high-speed wire rod training sample is obtained, comprising a high-speed wire rod sample image and the corresponding true boundary point locations. A boundary recognition model to be trained is trained based on the high-speed wire rod sample image and the corresponding true boundary point locations to obtain a trained boundary recognition model. An original high-speed wire rod image is obtained and input into the trained boundary recognition model to obtain information on the boundary line positions on the left and right sides of the original high-speed wire rod image. Based on the boundary line position information on the left and right sides of the original high-speed wire rod image, the original high-speed wire rod image is subjected to jitter correction. The processed image effectively removes lateral jitter, making the processed high-speed wire rod side view image more stable and clear. During the lateral jitter correction process, the harsh background noise in the production environment can also be eliminated, making the corrected image cleaner and more accurate. By removing lateral jitter and reducing background noise, the processed image becomes lightweight, convenient for storage and transmission, and can improve the subsequent defect detection effect.

[0100] Figure 6 This is a block diagram of a high-speed wire image lateral shake correction device according to an exemplary embodiment, which is used in a high-speed wire image lateral shake correction method. Figure 6 The device includes an acquisition module 610, a training module 620, a recognition module 630, and a correction module 640, wherein:

[0101] An acquisition module 610 is configured to acquire a high-speed wire rod training sample, wherein the high-speed wire rod training sample includes a high-speed wire rod sample image and a corresponding ground-truth boundary point position;

[0102] A training module 620 is configured to train a boundary recognition model to be trained based on the high-speed wire sample image and the corresponding true boundary point positions to obtain a trained boundary recognition model;

[0103] The recognition module 630 is used to obtain an original image of the high-speed wire rod, input the original image of the high-speed wire rod into the trained boundary recognition model, and obtain the position information of the boundary lines on the left and right sides of the original image of the high-speed wire rod;

[0104] The correction module 640 is configured to perform jitter correction on the high-speed wire rod original image according to the boundary line position information on the left and right sides of the high-speed wire rod original image.

[0105] Optionally, the boundary recognition model includes a feature extraction module, a classification recognition branch module, and a regression recognition branch module;

[0106] The classification and recognition branch module includes three fully connected layers;

[0107] The regression recognition branch module includes multi-layer convolutional layer and pooling layer processing.

[0108] Optionally, the training module 620 is configured to:

[0109] S21, inputting the high-speed wire sample image into a feature extraction module of a boundary recognition model to be trained to obtain feature information of the high-speed wire sample image;

[0110] S22, input the feature information of the high-speed wire sample image into the classification and recognition branch module to be trained, and perform calculations through three fully connected layers. The last fully connected layer outputs a discrimination result, and the classification result is calculated by applying the following formula (1) of the mean square error (MSE) loss function:

[0111]

[0112] Among them, k represents the number of pixels in each row, y′ i represents the predicted label of the i-th pixel, y i represents the true label of the i-th pixel;

[0113] S23, inputting the feature information of the high-speed wire sample image into the regression recognition branch module to be trained, outputting the predicted boundary point position of the high-speed wire sample image, and performing loss calculation on the regression recognition result according to the predicted boundary point coordinates, the true boundary point position corresponding to the high-speed wire sample image, and the following formula (2) of the OKS loss function;

[0114]

[0115] Among them, x left Represents the left boundary horizontal coordinate of the predicted boundary point position, x right Represents the right boundary horizontal coordinate of the predicted boundary point position, x gt_left Represents the left boundary horizontal coordinate of the true boundary point position, x gt_right Represents the right boundary abscissa of the true boundary point position, L foreground The length that represents the prospects of high-speed wire.

[0116] Optionally, the identification module 630 is configured to:

[0117] S31, setting i=1, N is the total number of pixel rows of the original image of the high-speed wire rod;

[0118] S32, reading the pixels of the i-th row of the high-speed wire rod original image, obtaining the boundary point position results of the i-th row, connecting the pixels of the i-th row and the boundary point position results of the i-th row, inputting the connection result into the feature extraction module, and obtaining feature information of the pixels of the i-th row;

[0119] S33, inputting the feature information of the pixels in the i-th row into the classification and recognition branch module to obtain a determination result of whether the pixels in the i-th row contain steel;

[0120] S34, inputting the feature information of the pixels in the i-th row into the regression recognition branch module to obtain the position of the boundary point to be determined in the i-th row;

[0121] S35, determining the position of the boundary point in the i-th row according to the judgment result of whether the pixels in the i-th row contain steel and the position of the boundary point to be determined in the i-th row;

[0122] S36. Compare i with N. If i is greater than or equal to N, go to S37. If i is less than N, then i=i+1 and go to S32.

[0123] S37. Determine the boundary line position information on the left and right sides of the high-speed wire rod original image based on the boundary point position information of each row.

[0124] Optionally, when i=1, the boundary point position result of the i-1th row is set to 0.

[0125] Optionally, the identification module 630 is further configured to:

[0126] If the result of the judgment of whether the pixels in the i-th row contain steel is that there is steel, the position of the boundary point to be determined in the i-th row is determined as the result of the boundary point position in the i-th row;

[0127] If the result of determining whether the pixels in the i-th row contain steel is no steel, the position of the boundary point to be determined in the i-th row is determined to be invalid, and the result of the boundary point position in the i-th row is determined to be 0.

[0128] Optionally, the feature extraction module is feature pyramid extraction, or ResNet-50 feature extraction network, or a combination of feature pyramid extraction and ResNet-50 feature extraction network.

[0129] In an embodiment of the present invention, a high-speed wire rod training sample is obtained, comprising a high-speed wire rod sample image and the corresponding true boundary point locations. A boundary recognition model to be trained is trained based on the high-speed wire rod sample image and the corresponding true boundary point locations to obtain a trained boundary recognition model. An original high-speed wire rod image is obtained and input into the trained boundary recognition model to obtain information on the boundary line positions on the left and right sides of the original high-speed wire rod image. Based on the boundary line position information on the left and right sides of the original high-speed wire rod image, the original high-speed wire rod image is subjected to jitter correction. The processed image effectively removes lateral jitter, making the processed high-speed wire rod side view image more stable and clear. During the lateral jitter correction process, the harsh background noise in the production environment can also be eliminated, making the corrected image cleaner and more accurate. By removing lateral jitter and reducing background noise, the processed image becomes lightweight, convenient for storage and transmission, and can improve the subsequent defect detection effect.

[0130] Figure 7 1 is a schematic diagram of the structure of an electronic device 700 provided in an embodiment of the present invention. The electronic device 700 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 701 and one or more memories 702, wherein the memory 702 stores at least one instruction, and the at least one instruction is loaded and executed by the processor 701 to implement the steps of the above-mentioned Chinese text spelling checking method.

[0131] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions, which can be executed by a processor in a terminal to perform the above-mentioned Chinese text spelling check method. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0132] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0133] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for correcting lateral jitter of a high-speed wire image, characterized in that: The method comprises: S1. Obtain a high-speed wire rod training sample, wherein the high-speed wire rod training sample includes a high-speed wire rod sample image and a corresponding true value boundary point position; S2. Training the boundary recognition model to be trained based on the high-speed wire sample image and the corresponding true boundary point positions to obtain a trained boundary recognition model; Wherein, the S2 includes: S21, inputting the high-speed wire sample image into a feature extraction module of a boundary recognition model to be trained to obtain feature information of the high-speed wire sample image; S22, input the feature information of the high-speed wire sample image into the classification and recognition branch module to be trained, and perform calculations through three fully connected layers. The last fully connected layer outputs a discrimination result, and the classification result is calculated by applying the following formula (1) of the mean square error (MSE) loss function: Among them, k represents the number of pixels in each row, y i ′ represents the predicted label of the i-th pixel, y i represents the true label of the i-th pixel; S23, inputting the feature information of the high-speed wire sample image into the regression recognition branch module to be trained, outputting the predicted boundary point position of the high-speed wire sample image, and performing loss calculation on the regression recognition result according to the predicted boundary point coordinates, the true boundary point position corresponding to the high-speed wire sample image, and the following formula (2) of the OKS loss function; Among them, x left Represents the left boundary horizontal coordinate of the predicted boundary point position, x right Represents the right boundary horizontal coordinate of the predicted boundary point position, x gt_left Represents the left boundary horizontal coordinate of the true boundary point position, x gt_right Represents the right boundary abscissa of the true boundary point position, L foreground The length that represents the prospects of high-speed wire; S3. Obtain an original image of the high-speed wire rod, input the original image of the high-speed wire rod into a trained boundary recognition model, and obtain boundary line position information on the left and right sides of the original image of the high-speed wire rod; Wherein, the S3 includes: S31, setting i=1, N is the total number of pixel rows of the original image of the high-speed wire rod; S32, reading the pixels of the i-th row of the high-speed wire rod original image, obtaining the boundary point position results of the i-th row, connecting the pixels of the i-th row and the boundary point position results of the i-th row, inputting the connection result into the feature extraction module, and obtaining feature information of the pixels of the i-th row; S33, inputting the feature information of the pixels in the i-th row into the classification and recognition branch module to obtain a determination result of whether the pixels in the i-th row contain steel; S34, inputting the feature information of the pixels in the i-th row into the regression recognition branch module to obtain the position of the boundary point to be determined in the i-th row; S35, determining the position of the boundary point in the i-th row according to the judgment result of whether the pixels in the i-th row contain steel and the position of the boundary point to be determined in the i-th row; S36. Compare i with N. If i is greater than or equal to N, go to S37. If i is less than N, then i=i+1 and go to S32. S37, determining the boundary line position information on the left and right sides of the high-speed wire rod original image based on the boundary point position information of each row; S4. Performing jitter correction on the high-speed wire rod original image according to the boundary line position information on the left and right sides of the high-speed wire rod original image.

2. The method according to claim 1, characterized in that The boundary recognition model includes a feature extraction module, a classification recognition branch module and a regression recognition branch module; The classification and recognition branch module includes three fully connected layers; The regression recognition branch module includes multi-layer convolutional layer and pooling layer processing.

3. The method according to claim 1, characterized in that When i=1, the boundary point position result of the i-1th row is set to 0.

4. The method according to claim 1, wherein The step S35 of determining the position of the boundary point in the i-th row according to the judgment result of whether the i-th row of pixels contains steel and the position of the boundary point to be determined in the i-th row includes: If the result of the judgment of whether the pixels in the i-th row contain steel is that there is steel, the position of the boundary point to be determined in the i-th row is determined as the result of the boundary point position in the i-th row; If the result of determining whether the pixels in the i-th row contain steel is no steel, the position of the boundary point to be determined in the i-th row is determined to be invalid, and the result of the boundary point position in the i-th row is determined to be 0.

5. The method according to claim 2, characterized in that The feature extraction module is feature pyramid extraction, or ResNet-50 feature extraction network, or a combination of feature pyramid extraction and ResNet-50 feature extraction network.

6. A high-speed wire image lateral jitter correction device, characterized in that: The device comprises: An acquisition module is used to acquire a high-speed wire rod training sample, wherein the high-speed wire rod training sample includes a high-speed wire rod sample image and a corresponding true value boundary point position; A training module is used to train the boundary recognition model to be trained based on the high-speed wire sample image and the corresponding true value boundary point position to obtain a trained boundary recognition model; Wherein, the training module is used to: S21, inputting the high-speed wire sample image into a feature extraction module of a boundary recognition model to be trained to obtain feature information of the high-speed wire sample image; S22, input the feature information of the high-speed wire sample image into the classification and recognition branch module to be trained, and perform calculations through three fully connected layers. The last fully connected layer outputs a discrimination result, and the classification result is calculated by applying the following formula (1) of the mean square error (MSE) loss function: Among them, k represents the number of pixels in each row, y i ′ represents the predicted label of the i-th pixel, y i represents the true label of the i-th pixel; S23, inputting the feature information of the high-speed wire sample image into the regression recognition branch module to be trained, outputting the predicted boundary point position of the high-speed wire sample image, and performing loss calculation on the regression recognition result according to the predicted boundary point coordinates, the true boundary point position corresponding to the high-speed wire sample image, and the following formula (2) of the OKS loss function; Among them, x left Represents the left boundary horizontal coordinate of the predicted boundary point position, x right Represents the right boundary horizontal coordinate of the predicted boundary point position, x gt_left Represents the left boundary horizontal coordinate of the true boundary point position, x gt_right Represents the right boundary abscissa of the true boundary point position, L foreground The length that represents the prospects of high-speed wire; A recognition module is used to obtain an original image of the high-speed wire rod, input the original image of the high-speed wire rod into a trained boundary recognition model, and obtain the position information of the boundary lines on the left and right sides of the original image of the high-speed wire rod; Optionally, the identification module is used to: S31, setting i=1, N is the total number of pixel rows of the original image of the high-speed wire rod; S32, reading the pixels of the i-th row of the high-speed wire rod original image, obtaining the boundary point position results of the i-th row, connecting the pixels of the i-th row and the boundary point position results of the i-th row, inputting the connection result into the feature extraction module, and obtaining feature information of the pixels of the i-th row; S33, inputting the feature information of the pixels in the i-th row into the classification and recognition branch module to obtain a determination result of whether the pixels in the i-th row contain steel; S34, inputting the feature information of the pixels in the i-th row into the regression recognition branch module to obtain the position of the boundary point to be determined in the i-th row; S35, determining the position of the boundary point in the i-th row according to the judgment result of whether the pixels in the i-th row contain steel and the position of the boundary point to be determined in the i-th row; S36. Compare i with N. If i is greater than or equal to N, go to S37. If i is less than N, then i=i+1 and go to S32. S37, determining the boundary line position information on the left and right sides of the high-speed wire rod original image based on the boundary point position information of each row; The correction module is used to perform jitter correction on the high-speed wire rod original image according to the boundary line position information on the left and right sides of the high-speed wire rod original image.

7. The device according to claim 6, characterized in that The boundary recognition model includes a feature extraction module, a classification recognition branch module and a regression recognition branch module; The classification and recognition branch module includes three fully connected layers; The regression recognition branch module includes multi-layer convolutional layer and pooling layer processing.

Images