Linear array camera image stretching correction method and device

Abstract

The invention provides a line-scan digital camera image stretching correction method and device, and the method comprises the steps: obtaining a to-be-corrected metal surface image based on a line-scan digital camera; the to-be-corrected metal surface image is a non-uniformly stretched image; performing image slicing on the to-be-corrected metal surface image to obtain a plurality of to-be-corrected slices; based on the trained image classification model, identifying the plurality of to-be-corrected slices to obtain a stretch rate corresponding to each to-be-corrected slice; based on the stretch rate corresponding to each to-be-corrected slice, interpolation stretch correction is carried out on the corresponding to-be-corrected slice, and a plurality of corrected slices are obtained; and splicing the plurality of corrected slices to obtain a stretched and corrected metal surface image corresponding to the to-be-corrected metal surface image. According to the method, the problem of image stretching of the linear array camera caused by uneven speed can be solved, the generalization is good, and the efficiency is high.

Description

Technical Field

[0001] The present invention relates to the field of intelligent manufacturing technology, and in particular to a linear array camera image stretching correction method and device. Background Art

[0002] At present, in the field of steel plate production, metal surface defect detection systems based on industrial camera imaging have been widely used. Industrial cameras are mainly divided into linear array cameras and area array cameras. Linear array cameras have the advantages of fast acquisition speed, high resolution, and continuous acquisition. Therefore, the linear array camera is fixed on the upper and lower surfaces of the steel plate for data acquisition. However, due to the uneven movement speed of the steel plate, the camera's acquisition at a fixed frame rate will not match the speed of the steel plate, resulting in the captured metal image being stretched in the direction of movement. Image stretching brings great difficulties to subsequent defect target detection and automated defect processing. Summary of the invention

[0003] The purpose of the present invention is to provide a method and device for image stretching correction of a line array camera in order to solve at least one of the above technical problems.

[0004] In a first aspect, an embodiment of the present invention provides a linear array camera image stretching correction method, comprising: based on a linear array camera, acquiring a metal surface image to be corrected; the metal surface image to be corrected is an image that is non-uniformly stretched; slicing the metal surface image to be corrected to obtain a plurality of slices to be corrected; based on a trained image classification model, identifying the plurality of slices to be corrected respectively to obtain a stretching rate corresponding to each slice to be corrected; based on the stretching rate corresponding to each slice to be corrected, interpolating and stretching-correcting the corresponding slices to be corrected respectively to obtain a plurality of corrected slices; splicing the plurality of corrected slices to obtain a stretched and corrected metal surface image corresponding to the metal surface image to be corrected.

[0005] Furthermore, it also includes: training a preset image classification model to obtain the trained image classification model; the preset image classification model is a deep learning model.

[0006] Furthermore, the preset image classification model is trained to obtain the trained image classification model, including: selecting data evenly distributed in the steel image as original data; performing image preprocessing on the original data to obtain the original image; the image preprocessing includes: removing background interference and mirroring; stretching the original image at different magnifications to obtain image data corresponding to different stretching rates; obtaining a training set and a verification set based on the image data corresponding to the different stretching rates; training the preset image classification model based on the training set to obtain the trained image classification model; and verifying the trained image classification model based on the verification set.

[0007] Furthermore, based on the image data corresponding to the different stretching rates, a training set and a validation set are obtained, including: performing image slicing on the image data corresponding to the different stretching rates respectively to obtain image slice sets corresponding to the different stretching rates; the size of the image slices in the image slice set is the same as the size of the slice to be corrected; and the image slice set is divided according to a preset ratio to obtain the training set and the validation set.

[0008] Furthermore, the preset image classification model includes a ResNet-18 network model.

[0009] In a second aspect, an embodiment of the present invention further provides a linear array camera image stretching and correction device, comprising: an acquisition module, a slicing module, an identification module, a correction module and a splicing module; wherein the acquisition module is used to acquire a metal surface image to be corrected based on a linear array camera; the metal surface image to be corrected is an image that is non-uniformly stretched; the slicing module is used to slice the metal surface image to be corrected to obtain a plurality of slices to be corrected; the identification module is used to identify the plurality of slices to be corrected based on a trained image classification model to obtain a stretching rate corresponding to each slice to be corrected; the correction module is used to interpolate and stretch the corresponding slices to be corrected based on the stretching rate corresponding to each slice to be corrected to obtain a plurality of corrected slices; the splicing module is used to splice the plurality of corrected slices to obtain a stretched and corrected metal surface image corresponding to the metal surface image to be corrected.

[0010] Furthermore, it also includes: a training module, used to train a preset image classification model to obtain the trained image classification model; the preset image classification model is a deep learning model.

[0011] Furthermore, the training module is also used to: select data evenly distributed in the steel image as original data; perform image preprocessing on the original data to obtain the original image; the image preprocessing includes: removing background interference and mirroring; stretching the original image at different magnifications to obtain image data corresponding to different stretching rates; obtaining a training set and a verification set based on the image data corresponding to different stretching rates; training a preset image classification model based on the training set to obtain the trained image classification model; and verifying the trained image classification model based on the verification set.

[0012] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.

[0013] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described in the first aspect above is implemented.

[0014] The present invention provides a linear array camera image stretch correction method and device. Considering that the stretch rate will change quickly due to the change of speed, the original image is sliced ​​and then the stretch rate of the slice is detected in a small range, which can effectively handle the situation of fast speed change. The present invention can handle the problem of linear array camera image stretching caused by uneven speed, and has good generalization and high efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 A flowchart of a linear camera image stretching correction method provided by an embodiment of the present invention;

[0017] Figure 2 A flowchart for training a preset image classification model provided by an embodiment of the present invention;

[0018] Figure 3 A uniform original image of a linear array camera on a steel plate surface provided by an embodiment of the present invention;

[0019] Figure 4 A metal surface image to be corrected provided by an embodiment of the present invention;

[0020] Figure 5 A corrected metal surface image provided by an embodiment of the present invention;

[0021] Figure 6 A schematic diagram of a linear array camera image stretching correction device provided by an embodiment of the present invention;

[0022] Figure 7 A schematic diagram of another linear array camera image stretching correction device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] Unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one", "one" or "the" do not indicate a quantitative limitation, but indicate the existence of at least one. Words such as "include" or "comprise" mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0025] It should be noted that the terms "up", "down", "left", "right", "front", "back", etc. used in the present invention are only used to indicate relative position relationships. When the absolute position of the object being described changes, the relative position relationship may also change accordingly.

[0026] In the metal surface image defect recognition and automatic processing system, due to the complex production environment and rapid speed changes, the metal image has problems such as underexposure and stretching distortion. Taking the above problems into consideration, the present invention proposes a linear array camera image stretching correction and device, which uses a deep learning-based image classification algorithm to detect the stretching rate of a small-scale stretched image, and then interpolates and stretches the image according to the obtained stretching rate to achieve stretched restoration, effectively dealing with the problem of metal surface image stretching distortion caused by speed mismatch, and improving the accuracy and efficiency of metal surface defect detection and automatic defect processing. The above-mentioned linear array camera image stretching correction and device are described in detail below.

[0027] Embodiment 1

[0028] Figure 1 FIG. 1 is a flow chart of a linear array camera image stretching correction method provided according to an embodiment of the present invention. Figure 1 As shown, the method specifically comprises the following steps:

[0029] Step S102, based on the line array camera, obtaining the metal surface image to be corrected; the metal surface image to be corrected is an image that is non-uniformly stretched.

[0030] Optionally, the metal to be corrected comprises steel plate.

[0031] Step S104, slicing the metal surface image to be corrected to obtain a plurality of slices to be corrected.

[0032] Step S106: Based on the trained image classification model, multiple slices to be corrected are identified respectively to obtain the stretching rate corresponding to each slice to be corrected.

[0033] Step S108 , based on the stretching rate corresponding to each slice to be corrected, interpolation stretching correction is performed on the corresponding slice to be corrected, to obtain a plurality of corrected slices.

[0034] Step S110 , splicing a plurality of corrected slices to obtain a stretched and corrected metal surface image corresponding to the metal surface image to be corrected.

[0035] Specifically, the method provided by an embodiment of the present invention also includes: training a preset image classification model to obtain a trained image classification model; wherein the preset image classification model is a deep learning model.

[0036] Preferably, in an embodiment of the present invention, the preset image classification model includes a ResNet-18 network model.

[0037] Figure 2 FIG. 1 is a flowchart of training a preset image classification model according to an embodiment of the present invention. Figure 2 As shown, the following steps are included:

[0038] Step S201, selecting uniformly distributed data in the steel passing image as original data.

[0039] Step S202, performing image preprocessing on the original data to obtain an original image; the image preprocessing includes: removing background interference and mirroring.

[0040] For example, Figure 3 is a uniform original image of a steel plate surface linear array camera provided by an embodiment of the present invention. Figure 3 As shown, the uniformly distributed data in the steel image is selected as the original data, and simple image preprocessing such as removing background interference and mirroring is performed, and finally the original image with a height of 2048 and a width of 1024 is obtained.

[0041] Step S203, stretching the original image at different magnifications to obtain image data corresponding to the different stretching ratios.

[0042] Specifically, the uniform original image is stretched at different ratios according to the direction of movement. For example, the ratios of x1, x2, x3, x4, x5, ... x10, x15, x20, x30, x40, x50, x100, x200, and x500 are used respectively, and finally image data corresponding to 18 types of different stretching ratios are obtained.

[0043] Step S204: obtaining a training set and a validation set based on the image data corresponding to different stretching rates.

[0044] Specifically, image data corresponding to different stretching ratios are sliced ​​to obtain image slice sets corresponding to different stretching ratios; the size of the image slices in the image slice set is the same as the size of the slice to be corrected;

[0045] The image slice set is divided into a training set and a validation set according to a preset ratio.

[0046] In an embodiment of the present invention, in order to ensure that the image scale in the data set is consistent, image slices are performed from each type of image according to the size of the slice to be corrected, for example, the image slices are performed with a height of 256 pixels and a constant width. The original image is an image captured by a linear array camera with 2048 lines. Since target detection processing is performed later, it is inconvenient to directly capture and store 256 lines during acquisition. At the same time, considering the imbalance problem between classes in deep learning, the same number of small images are obtained from each class and placed in different folders. The folders are named as stretch multiples to construct a data set in ImageNet format. It is divided into a training set and a verification machine in a ratio of 8:2.

[0047] Step S205: training the preset image classification model based on the training set to obtain a trained image classification model.

[0048] Specifically, based on the image classification method in deep learning, the data set is trained to obtain an image stretch rate recognition model (i.e., image classification model). Specifically, the training set is sent to the ResNet-18 network model for image classification training; the classification result is calculated by applying the cross entropy loss function; the gradient calculation is completed by iteration through back propagation; the optimizer selects AdamW, the learning rate is 1e-3, and the learning rate decays to 1e-5, and the network model parameters are updated to minimize the loss function value. All batches are trained in a loop, and all data are iterated once, which is called an epoch. When the set number of iterations is reached, the training is completed and the model is saved.

[0049] Step S206: verify the trained image classification model based on the verification set.

[0050] After obtaining the trained image classification model, obtain the metal surface image to be corrected, for example, Figure 4is a metal surface image to be corrected provided by an embodiment of the present invention, such as Figure 4 As shown in the figure, the metal surface image to be corrected is the original image taken by the linear array camera and non-uniformly stretched. Then, the original image is sliced ​​into images of the same size as the image slice set according to 256 lines, and input into the trained image classification model for recognition, and the category of the small image, i.e., the stretching rate, is obtained;

[0051] Then, interpolation and stretch correction are performed on each slice according to its stretching rate. Finally, multiple small images are stitched together in the original order to obtain the stretch correction image of the original stretch image, as shown in Figure 5 As shown, the correction of the stretched image is completed.

[0052] It can be seen from the above description that the embodiment of the present invention provides a linear camera image stretching correction method. Compared with the prior art, the advantages and positive effects of the present invention are:

[0053] According to the imaging principle of the linear array camera, the present invention constructs the original image into a reasonable training data set without any manual frame marking; the original image is sliced, and considering that the stretching rate will change rapidly due to the change in speed, a method of detecting the stretching rate in a small range by slicing 256 lines is adopted, which can effectively handle the situation of fast speed change.

[0054] Embodiment 2

[0055] Figure 6 Schematic diagram of a linear array camera image stretching correction device provided according to an embodiment of the present invention. Figure 6 As shown, the device includes: an acquisition module 10, a slicing module 20, a recognition module 30, a correction module 40 and a stitching module 50.

[0056] Specifically, the acquisition module 10 is used to acquire the metal surface image to be corrected based on a line array camera; the metal surface image to be corrected is an image that is non-uniformly stretched.

[0057] The slicing module 20 is used to slice the metal surface image to be corrected to obtain a plurality of slices to be corrected.

[0058] The recognition module 30 is used to recognize the multiple slices to be corrected respectively based on the trained image classification model to obtain the stretching rate corresponding to each slice to be corrected.

[0059] The correction module 40 is used to perform interpolation stretching correction on the corresponding slices to be corrected based on the stretching rate corresponding to each slice to be corrected, so as to obtain a plurality of corrected slices.

[0060] The stitching module 50 is used to stitch a plurality of corrected slices to obtain a stretched and corrected metal surface image corresponding to the metal surface image to be corrected.

[0061] Figure 7 FIG. 1 is a schematic diagram of another linear array camera image stretching correction device provided according to an embodiment of the present invention. Figure 7 As shown, it also includes: a training module 60 and a defect target detection module 70.

[0062] Specifically, the training module 60 is used to train a preset image classification model to obtain a trained image classification model; the preset image classification model is a deep learning model.

[0063] The defect target detection module 70 is used to test the defect recognition accuracy in the image after stretching correction.

[0064] Specifically, the training module 60 is also used for:

[0065] Select the data evenly distributed in the steel image as the original data;

[0066] Perform image preprocessing on the original data to obtain the original image; image preprocessing includes: removing background interference and mirroring;

[0067] The original image is stretched at different magnifications to obtain image data corresponding to different stretching ratios;

[0068] Based on the image data corresponding to different stretching rates, a training set and a validation set are obtained; specifically, image slices are respectively performed on the image data corresponding to different stretching rates to obtain image slice sets corresponding to different stretching rates; the size of the image slices in the image slice set is the same as the size of the slice to be corrected; the image slice set is divided according to a preset ratio to obtain the training set and the validation set;

[0069] Train the preset image classification model based on the training set to obtain a trained image classification model;

[0070] Based on the validation set, the trained image classification model is verified.

[0071] From the above description, it can be seen that an embodiment of the present invention also provides a linear array camera image stretching correction device, which is aimed at sites with large environmental constraints and is less affected by factors such as constant changes in metal speed and severe image deformation. It has a fast correction speed and good correction performance and robustness. It can solve the problem of linear array camera metal surface image stretching, improve the accuracy of subsequent defect detection and the positioning accuracy of automatic control defect processing, and provide a solution for efficient intelligent manufacturing.

[0072] An embodiment of the present invention further provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method in the first embodiment when executing the computer program.

[0073] An embodiment of the present invention further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed by a processor, the method in the first embodiment is implemented.

[0074] There are a few points to note:

[0075] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention. Other structures may refer to the general design.

[0076] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0077] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0078] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A linear array camera image stretching correction method, characterized in that: include: Based on the linear array camera, the image of the metal surface to be corrected is obtained; The metal surface image to be corrected is an image that is non-uniformly stretched; Slicing the metal surface image to be corrected to obtain a plurality of slices to be corrected; Based on the trained image classification model, the plurality of slices to be corrected are respectively identified to obtain the stretching rate corresponding to each slice to be corrected; Based on the stretching rate corresponding to each slice to be corrected, interpolation stretching correction is performed on the corresponding slice to be corrected to obtain a plurality of corrected slices; The multiple corrected slices are spliced ​​to obtain a stretched and corrected metal surface image corresponding to the metal surface image to be corrected.

2. The method according to claim 1, characterized in that Also includes: The preset image classification model is trained to obtain the trained image classification model; the preset image classification model is a deep learning model.

3. The method according to claim 2, characterized in that Training a preset image classification model to obtain the trained image classification model includes: Select the data evenly distributed in the steel image as the original data; Performing image preprocessing on the original data to obtain an original image; the image preprocessing includes: removing background interference and mirroring; Stretching the original image at different magnifications to obtain image data corresponding to the different stretching ratios; Based on the image data corresponding to the different stretching rates, a training set and a validation set are obtained; Training a preset image classification model based on the training set to obtain the trained image classification model; Based on the verification set, the trained image classification model is verified.

4. The method according to claim 3, characterized in that Based on the image data corresponding to the different stretching rates, a training set and a validation set are obtained, including: Slicing the image data corresponding to different stretching rates respectively to obtain image slice sets corresponding to different stretching rates; the size of the image slices in the image slice set is the same as the size of the slice to be corrected; The image slice set is divided into the training set and the verification set according to a preset ratio.

5. The method according to claim 2, characterized in that: The preset image classification model includes a ResNet-18 network model.

6. A linear array camera image stretching correction device, characterized in that: include: Acquisition module, slicing module, recognition module, correction module and splicing module; among them, The acquisition module is used to acquire the metal surface image to be corrected based on the linear array camera; the metal surface image to be corrected is an image that is non-uniformly stretched; The slicing module is used to slice the metal surface image to be corrected to obtain a plurality of slices to be corrected; The recognition module is used to recognize the multiple slices to be corrected respectively based on the trained image classification model to obtain the stretching rate corresponding to each slice to be corrected; The correction module is used to perform interpolation stretching correction on the corresponding slices to be corrected based on the stretching rate corresponding to each slice to be corrected, so as to obtain a plurality of corrected slices; The stitching module is used to stitch the multiple corrected slices to obtain the stretched and corrected metal surface image corresponding to the metal surface image to be corrected.

7. The device according to claim 6, characterized in that Also includes: A training module, used to train a preset image classification model to obtain the trained image classification model; The preset image classification model is a deep learning model.

8. The device according to claim 7, characterized in that The training module is also used for: Select the data evenly distributed in the steel image as the original data; Performing image preprocessing on the original data to obtain an original image; the image preprocessing includes: removing background interference and mirroring; Stretching the original image at different magnifications to obtain image data corresponding to the different stretching ratios; Based on the image data corresponding to the different stretching rates, a training set and a validation set are obtained; Training a preset image classification model based on the training set to obtain the trained image classification model; Based on the verification set, the trained image classification model is verified.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 5 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images