A crystal flower detection method, device, electronic device and storage medium

Abstract

The present invention provides a crystal flower detection method, device, electronic device and storage medium. The method includes: obtaining multiple crystal flower images to be tested and detection light source-camera optical axis angle information of a coated metal plate to be tested, performing a first image reconstruction based on the detection light source-camera optical axis angle information and the multiple crystal flower images to be tested to obtain a reflectivity information image to be tested, performing detection convolution on the reflectivity information image to be tested with multiple preset two-dimensional texture filters to obtain multiple target feature texture vector images, inputting all the target feature texture vector images into a crystal flower classification model to perform crystal flower size classification to obtain a target crystal flower size category, calculating energy feature quantities of multiple grayscale co-occurrence matrices of the reflectivity information image to be tested based on multiple preset scanning directions, obtaining a feature energy mean value based on all the energy feature quantities to determine the uniformity of crystal flower texture distribution; and improving the accuracy and intelligent efficiency of crystal flower detection by the target crystal flower size category and the uniformity of crystal flower texture distribution.

Description

Technical Field

[0001] The present application relates to the field of intelligent metallurgical technology, and in particular to a crystal flower detection method, device, electronic equipment and storage medium. Background Art

[0002] Hot-dip galvanizing is a widely used steel surface coating technology, effectively addressing metal corrosion. However, during hot-dip galvanizing, visible dendritic grains called "spangles" appear on the surface. The size and uniformity of the spangles significantly impact corrosion resistance. Currently, spangle grading relies primarily on manual experience. However, as market demands for galvanized product quality continue to rise, manual experience is no longer sufficient for accurate, intelligent, and efficient spangle detection. Summary of the Invention

[0003] The present invention provides a zinc flower detection method, device, electronic equipment and storage medium to solve the technical problems of low accuracy and low intelligent efficiency of zinc flower detection.

[0004] In one embodiment of the present application, the present application provides a crystal flower detection method, comprising: obtaining a plurality of crystal flower images to be tested and detection light source-camera optical axis angle information of the coated metal plate to be tested, and performing a first image reconstruction based on the detection light source-camera optical axis angle information and the plurality of crystal flower images to be tested to obtain a reflectivity information image to be tested, wherein the plurality of crystal flower images to be tested are collected by a camera module when the metal plate to be tested is successively irradiated by light source modules of different preset detection directions; the reflectivity information image to be tested is respectively convolved with a plurality of preset two-dimensional texture filters to obtain a plurality of target feature texture vector images; and all the The target feature texture vector image is input into the crystal flower classification model for crystal flower size classification to obtain a target crystal flower size category; based on multiple preset scanning directions, the energy feature quantities corresponding to multiple grayscale co-occurrence matrices of the reflectivity information image to be measured are calculated, and the feature energy mean value obtained according to all the energy feature quantities is used to determine the uniformity of the crystal flower texture distribution, and the target crystal flower size category and the uniformity of the crystal flower texture distribution are used as the crystal flower detection result of the coated metal plate to be measured; wherein, the crystal flower classification model is obtained by training multiple initial crystal flower images and the crystal flower size classification labels corresponding to each of the initial crystal flower images.

[0005] In one embodiment of the present application, before obtaining multiple crystal flower images to be tested and detecting light source-camera optical axis angle information of the coated metal plate to be tested, the method also includes: obtaining multiple initial crystal flower images of multiple size categories, training light source-camera optical axis angle information and crystal flower size classification labels corresponding to each of the initial crystal flower images, and the multiple initial crystal flower images of a size category crystal flower are collected by the camera module when the light source modules of different preset training light source directions successively irradiate the training coated metal plate; a second image is reconstructed according to the training light source-camera optical axis angle information and the multiple initial crystal flower images of the crystal flower of the size category to obtain a training reflectivity information image; the training reflectivity information image is respectively convolved with multiple preset two-dimensional texture filters to obtain multiple training feature texture vector images; the crystal flower size classification label of the crystal flower of the size category and each of the training feature texture vector images of the crystal flower of the size category are used as a training sample, and the training sample is input into the initial classification model for classification training to obtain a crystal flower classification model.

[0006] In one embodiment of the present application, the training reflectivity information image is respectively trained and convolved with multiple preset two-dimensional texture filters to obtain multiple training feature texture vector images, including: the training reflectivity information image is respectively trained and convolved with each of the preset two-dimensional texture filters to obtain multiple training feature texture filter images; each of the training feature texture filter images is locally averaged filtered by a preset smoothing filter to obtain multiple training feature texture vector images.

[0007] In one embodiment of the present application, a second image reconstruction is performed on the multiple initial crystal flower images of the crystal flower of the one size category according to the training light source-camera optical axis angle information to obtain a training reflectivity information image, including: a second image reconstruction is performed on the multiple initial crystal flower images of the crystal flower of the one size category according to the training light source-camera optical axis angle information and a preset photometric stereo equation to obtain a training reflectivity information image.

[0008] In one embodiment of the present application, energy feature quantities corresponding to multiple grayscale co-occurrence matrices of the reflectivity information image to be measured are calculated based on multiple preset scanning directions, and the characteristic energy mean value obtained based on all the energy feature quantities is used to determine the uniformity of the crystal flower texture distribution: multiple grayscale co-occurrence matrices of the reflectivity information image to be measured in different preset scanning directions are calculated; the energy feature quantity of each grayscale co-occurrence matrix is ​​determined based on each of the grayscale co-occurrence matrices; each of the energy feature quantities is averaged to obtain the characteristic energy mean value; if the characteristic energy mean value is greater than or equal to the preset energy mean value, the uniformity of the crystal flower texture distribution is determined to be uniform; if the characteristic energy mean value is less than the preset energy mean value, the uniformity of the crystal flower texture distribution is determined to be uneven; wherein, the preset scanning directions include horizontal direction, vertical direction, right diagonal direction and left diagonal direction.

[0009] In one embodiment of the present application, a method for determining a preset two-dimensional texture filter includes: generating multiple one-dimensional texture filters based on preset texture features; performing texture convolution on two different one-dimensional texture filters or two identical one-dimensional texture filters to obtain a preset two-dimensional texture filter; wherein each of the one-dimensional texture filters includes a one-dimensional grayscale texture filter, a one-dimensional edge texture filter, a one-dimensional ripple texture filter, a one-dimensional speckle texture filter, and a one-dimensional oscillation texture filter.

[0010] In one embodiment of the present application, the present application provides a crystal flower detection device, including: an image acquisition module, used to obtain multiple crystal flower images to be tested and detection light source-camera optical axis angle information of the coated metal plate to be tested, and perform a first image reconstruction based on the detection light source-camera optical axis angle information and each of the crystal flower images to be tested to obtain a reflectivity information image to be tested, and each of the crystal flower images to be tested is obtained by successively irradiating the coated metal plate to be tested by light source modules in different directions of a preset detection light source direction; a texture filtering module, used to perform detection convolution on the reflectivity information image to be tested with multiple preset two-dimensional texture filters respectively to obtain multiple target feature texture vector images; a crystal flower classification module, Used to input each of the target feature texture vector images into a crystal flower classification model for crystal flower size classification to obtain a target crystal flower size category; a uniformity determination module is used to calculate the energy feature quantities corresponding to multiple grayscale co-occurrence matrices of the reflectivity information image to be measured based on multiple preset scanning directions, and determine the uniformity of the crystal flower texture distribution according to the feature energy mean obtained by averaging each of the energy feature quantities, so as to use the target crystal flower size category and the crystal flower texture distribution uniformity as the crystal flower detection result of the coated metal plate to be measured; wherein, the crystal flower classification model is obtained by training multiple initial crystal flower images and the crystal flower size classification labels corresponding to each of the initial crystal flower images.

[0011] In one embodiment of the present application, the crystal flower detection device also includes an image capturing module; the image capturing module includes a coated metal plate to be tested, a camera module and at least three light source modules; the camera optical axis of the camera module is perpendicular to the surface to be tested of the coated metal plate to be tested; each of the light source modules is installed around the camera module according to a preset detection light source direction, and is used to successively illuminate the coated metal plate to be tested; the camera module is used to capture each crystal flower image to be tested that is successively illuminated by each of the light source modules on the coated metal plate to be tested.

[0012] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device implements the device operation status diagnosis method as described in any of the above embodiments.

[0013] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor of a computer, the computer executes the device operation status diagnosis method described in any of the above embodiments.

[0014] The beneficial effects of the present invention are as follows: the present invention provides a crystal flower detection method, device, electronic device and storage medium. The present invention reconstructs a first image of multiple crystal flower images to be tested obtained by successively irradiating the coated metal plate to be tested with light source modules in different directions to obtain a reflectivity information image to be tested, thereby enhancing the boundary texture of the crystal flower and improving the detection accuracy. The target crystal flower size classification of the coated metal plate to be tested is obtained by extracting multiple categories of texture features from the reflectivity information image to be tested and then inputting them into a crystal flower classification model for crystal flower size classification. The texture distribution uniformity is determined by calculating the characteristic energy mean of multiple grayscale co-occurrence matrices of the reflectivity information image to be tested in multiple preset scanning directions. The target crystal flower size classification and the crystal flower texture distribution uniformity are used as the crystal flower detection results of the coated metal plate to be tested, thereby improving the accuracy and intelligent efficiency of crystal flower detection.

[0015] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are incorporated into and constitute a part of the specification, illustrating embodiments consistent with the present application and, together with the specification, serving to explain the principles of the present application. It is obvious that the drawings described below are merely some embodiments of the present application, and a person of ordinary skill in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0017] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied;

[0018] Figure 2 A schematic diagram of a process for detecting a crystal flower according to an embodiment of the present application is shown;

[0019] Figure 3 shows a side view of a photometric stereo method according to one embodiment of the present application;

[0020] Figure 4 shows a top view of a photometric stereo method according to one embodiment of the present application;

[0021] Figure 5 Shows reflectivity information images to be measured corresponding to multiple spangle images to be measured according to one embodiment of the present application;

[0022] Figure 6 FIG. 1 shows edge texture features in a target feature texture vector image according to an embodiment of the present application;

[0023] Figure 7 shows an oscillating texture feature in a target feature texture vector image according to one embodiment of the present application;

[0024] Figure 8 shows a spot texture feature in a target feature texture vector image according to one embodiment of the present application;

[0025] Figure 9 shows the grayscale edge texture features in the target feature texture vector image according to one embodiment of the present application;

[0026] Figure 10 A schematic diagram of the implementation process of a zinc spangle detection method according to an embodiment of the present application is shown;

[0027] Figure 11 A block diagram of a crystal flower detection device according to an embodiment of the present application is shown;

[0028] Figure 12 A schematic diagram of the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] The following describes the embodiments of the present application through specific examples. Those skilled in the art can easily understand the other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0030] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application. Therefore, the illustrations only show components related to the present application and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0031] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0032] See also Figure 1 , Figure 1 Schematic diagram showing an exemplary system architecture to which the technical solution of the embodiment of the present application can be applied. Figure 1 As shown, the system architecture may include an image capturing module 101 and a computer device 102. The image capturing module 101 is used to capture multiple images of crystal flowers to be tested on the coated metal plate to be tested, and the computer device 102 is used to detect crystal flowers by determining the target crystal flower size category and the uniformity of crystal flower texture distribution.

[0033] Exemplarily, after the computer device 102 obtains multiple crystal flower images to be tested and the detection light source-camera optical axis angle information of the coated metal plate to be tested, it performs a first image reconstruction based on the detection light source-camera optical axis angle information and the multiple crystal flower images to be tested to obtain a reflectivity information image to be tested, and the multiple crystal flower images to be tested are collected by the camera module when the light source modules of different preset detection directions successively illuminate the metal plate to be tested; the reflectivity information images to be tested are respectively convolved with multiple preset two-dimensional texture filters to obtain multiple target feature texture vector images; all target feature texture vector images are input into the crystal flower classification model for crystal flower size classification to obtain a target crystal flower size category; based on multiple preset scanning directions, the energy feature quantities corresponding to multiple grayscale co-occurrence matrices of the reflectivity information images to be tested are calculated, and the feature energy mean value obtained based on all energy feature quantities is used to determine the uniformity of the crystal flower texture distribution, and the target crystal flower size category and the uniformity of the crystal flower texture distribution are used as the crystal flower detection result of the coated metal plate to be tested; wherein, the crystal flower classification model is obtained by training multiple initial crystal flower images and the crystal flower size classification labels corresponding to each initial crystal flower image.

[0034] In related technologies, manual experience cannot meet the requirements of accurate, intelligent and efficient detection of crystal flowers.

[0035] In order to solve the above technical problems, the present application provides a crystal flower detection method, device, electronic device and storage medium. The implementation details of the technical solution of the embodiment of the present application are elaborated in detail below.

[0036] See also Figure 2 , Figure 2 FIG. 1 shows a flow chart of a method for detecting a crystal flower according to an embodiment of the present application. Figure 2 As shown, in an exemplary embodiment, the information method includes at least steps S210 to S240, which are described in detail as follows:

[0037] Step S210, obtain multiple crystal flower images to be tested and detection light source-camera optical axis angle information of the coated metal plate to be tested, and perform a first image reconstruction based on the detection light source-camera optical axis angle information and the multiple crystal flower images to be tested to obtain a reflectivity information image to be tested.

[0038] Among them, the multiple crystal flower images to be tested are collected by the camera module when the light source modules with different preset detection directions successively illuminate the metal plate to be tested.

[0039] In one embodiment of the present application, the crystal flower is a crystal formed on the surface of a metal coating such as zinc plating or tin plating, and the crystal flower includes the structure of zinc flower.

[0040] In one embodiment of the present application, before obtaining multiple crystal flower images to be tested of the coated metal plate to be tested and detecting the light source-camera optical axis angle information, the method also includes: obtaining multiple initial crystal flower images of multiple size categories of crystal flowers, training light source-camera optical axis angle information and crystal flower size classification labels corresponding to each initial crystal flower image, and the multiple initial crystal flower images of a size category of crystal flowers are collected by the camera module when the light source modules of different preset training light source directions successively irradiate the training coated metal plate; a second image is reconstructed according to the training light source-camera optical axis angle information and the multiple initial crystal flower images of a size category of crystal flowers to obtain a training reflectivity information image; the training reflectivity information image is respectively convolved with multiple preset two-dimensional texture filters to obtain multiple training feature texture vector images; the crystal flower size classification label of a size category of crystal flowers and each training feature texture vector image of a size category of crystal flowers are used as a training sample, and the training sample is input into the initial classification model for classification training to obtain a crystal flower classification model.

[0041] In one embodiment of the present application, if the crystal flower is a zinc flower, the size categories of the crystal flower include large zinc flowers, small zinc flowers, and no zinc flowers. There is a corresponding relationship between the size category of the crystal flower and the crystal flower size classification label.

[0042] In one embodiment of the present application, the preset training light source directions include training illumination angles for a light source module. Based on the principles of photometric stereo, at least three light source modules are provided. Multiple initial crystal flower images are captured by sequentially illuminating the same area of ​​a training coated metal plate with light source modules having different preset training light source directions. This reduces the effect of reflections from the coated metal material on the initial crystal flower images, thereby enhancing the boundary texture of the crystal flower and improving the accuracy of crystal flower detection.

[0043] In one embodiment of the present application, see Figure 3 , Figure 3 FIG. 1 shows a side view of a photometric stereo method according to an embodiment of the present application. Figure 3 As shown, 301 is the target coated metal plate, 302 is the camera module, and 303 is the target light source. The target coated metal plate 301 includes the coated metal plate to be tested and the training coated metal plate. In the side view, the first angle between the light source module and the camera optical axis of the camera module is recorded as Slant, and the angle range of Slant includes 30° to 60°. Figure 4 , Figure 4 FIG. 1 shows a top view of a photometric stereo method according to an embodiment of the present application. Figure 4As shown, each light source module is evenly distributed around the target coated metal plate. This embodiment uses four light source modules. The second angle between each light source module and the center of the target coated metal plate is Tilts, and Tilts are 0°, 90°, 180°, and 270°, respectively. Tilts can be evenly or unevenly distributed depending on the number of light source modules. This application does not impose any restrictions on the value of the second angle.

[0044] In one embodiment of the present application, the training illumination angle includes a first angle and a second angle, and the training light source-camera optical axis angle information is included in the first angle and the second angle.

[0045] In one embodiment of the present application, a second image reconstruction is performed based on the training light source-camera optical axis angle information and multiple initial crystal flower images of a size category to obtain a training reflectivity information image, including: performing a second image reconstruction on multiple initial crystal flower images of a size category of crystal flowers based on the training light source-camera optical axis angle information and a preset photometric stereo equation to obtain a training reflectivity information image.

[0046] In one embodiment of the present application, the preset photometric stereo equation is as follows:

[0047] I (x,y) =ρ (x,y) *N*L formula (1)

[0048] Among them, I (x,y) is the pixel value of the first image at point (x, y), ρ (x,y) is the pixel value of the second image at point (x, y), L is the unit vector of the light source direction of the target light source, N is the surface unit normal vector of the second image, the first image includes the initial crystal flower image and one of the crystal flower images to be measured, the second image includes one of the reflectivity information image to be measured and the training reflectivity information image, and the first image is acquired by the camera module when the target light source illuminates the target coated metal plate.

[0049] In one embodiment of the present application, the light source direction unit vector is as follows:

[0050] L=(a,b,c) Formula (2)

[0051] Wherein, L is the light source direction unit vector, a is the first component of the light source direction unit vector, b is the second component of the light source direction unit vector, and c is the third component of the light source direction unit vector.

[0052] In one embodiment of the present application, the first component of the light source direction unit vector is as follows:

[0053] a=cos(Slant)*sin(Tilts) Formula (3)

[0054] Where a is the first component of the light source direction unit vector, Slant is the first angle between the target light source and the camera optical axis, and Tilts is the second angle between the target light source and the target coated metal plate.

[0055] In one embodiment of the present application, the first component of the light source direction unit vector is as follows:

[0056] b=sin(Slant)*sin(Tilts) Formula (4)

[0057] Where b is the second component of the light source direction unit vector, Slant is the first angle between the target light source and the camera optical axis, and Tilts is the second angle between the target light source and the target coated metal plate.

[0058] In one embodiment of the present application, the first component of the light source direction unit vector is as follows:

[0059] c=cos(Tilts) Formula (5)

[0060] Wherein, c is the third component of the light source direction unit vector, and Tilts is the second angle between the target light source and the target coated metal plate.

[0061] In one embodiment of the present application, the equation system is solved by at least three first images to obtain the pixel value ρ of the second image at the point (x, y) (x,y) , the equations are as follows:

[0062]

[0063] Where I1(x,y) is the pixel value of the first image at point (x,y), I2(x,y) is the pixel value of the second image at point (x,y), I3(x,y) is the pixel value of the third image at point (x,y), and ρ (x,y) is the pixel value of the second image at point (x, y), L1 is the light source direction unit vector of the first target light source, L2 is the light source direction unit vector of the second target light source, L3 is the light source direction unit vector of the third target light source, and N is the surface unit normal vector of the second image.

[0064] In one embodiment of the present application, a method for determining a preset two-dimensional texture filter includes: generating multiple one-dimensional texture filters based on preset texture features; performing texture convolution on two different one-dimensional texture filters or two identical one-dimensional texture filters to obtain a preset two-dimensional texture filter; wherein each dimensional texture filter includes a one-dimensional grayscale texture filter, a one-dimensional edge texture filter, a one-dimensional ripple texture filter, a one-dimensional spot texture filter, and a one-dimensional oscillation texture filter.

[0065] In one embodiment of the present application, the preset texture features include grayscale features, edge features, ripple features, spot features, and oscillation features, and various one-dimensional texture filters include: one-dimensional grayscale texture filter L = [1 4 6 4 1]; one-dimensional edge texture filter E = [-1-2 0 2 1]; one-dimensional ripple texture filter W = [-1 2 0-2 1]; one-dimensional spot texture filter S = [-1 0 2 0-1]; one-dimensional oscillation texture filter R = [1-4 6-4 1]. This is only an example, and the present application does not impose any restrictions on the specific values ​​of the one-dimensional texture filters.

[0066] In one embodiment of the present application, multiple two-dimensional texture filters are generated by performing texture convolution between one-dimensional texture filters. A two-dimensional grayscale texture filter LS is obtained by performing texture convolution with a one-dimensional speckle texture filter. LS is shown as follows:

[0067]

[0068] Among them, LS is a two-dimensional grayscale speckle texture filter.

[0069] In one embodiment of the present application, the training reflectivity information image is trained and convolved with multiple preset two-dimensional texture filters to obtain multiple training feature texture vector images, including: training and convolving the training reflectivity information image with each preset two-dimensional texture filter to obtain multiple training feature texture filter images; and locally averaging filtering each training feature texture filter image through a preset smoothing filter to obtain multiple training feature texture vector images.

[0070] In one embodiment of the present application, the preset smoothing filter includes but is not limited to a Gaussian filter or a mean filter.

[0071] In one embodiment of the present application, an initial classification model is generated based on a multilayer perceptron (MLP) neural network classifier. The initial classification model includes a feature vector preprocessing training function and an output layer activation function. A training feature texture vector image is a feature vector. The feature vector preprocessing training function is obtained by principal component analysis of each training feature texture vector image. The output value of the activation function ranges from 0 to 1.

[0072] In one embodiment of the present application, the activation function is as follows:

[0073]

[0074] Among them, f(x) is the output value of multiple feature vectors, x i is the eigenvalue of the i-th eigenvector, and k is the number of eigenvectors.

[0075] In one embodiment of the present application, a preset training light source direction is used as a preset detection light source direction or a new preset detection light source direction is set. The preset detection light source direction includes a detection illumination angle of a light source module. The detection illumination angle includes a first angle and a second angle. The detection light source-camera optical axis angle information includes a first angle and a second angle.

[0076] In one embodiment of the present application, see Figure 5 , Figure 5 The figure shows the reflectivity information images to be measured corresponding to the multiple spangle images to be measured according to one embodiment of the present application. Figure 5 As shown, a first image reconstruction is performed on a plurality of zinc spangle images to be measured according to the detection light source-camera optical axis angle information and a preset photometric stereo equation to obtain a reflectivity information image to be measured.

[0077] Step S220 , performing detection convolution on the reflectivity information image to be measured and a plurality of preset two-dimensional texture filters respectively to obtain a plurality of target feature texture vector images.

[0078] In one embodiment of the present application, see Figure 6 , Figure 6 FIG. 4 shows edge texture features in a target feature texture vector image according to an embodiment of the present application. Figure 6 As shown, Figure 6 The target feature texture vector image is obtained by convolving the reflectivity information image to be measured with the two-dimensional edge texture filter. Figure 6 The edge texture features of zinc spangles in the target feature texture vector image are shown. The two-dimensional edge texture filter is obtained by texture convolution of two one-dimensional edge texture filters.

[0079] In one embodiment of the present application, see Figure 7 , Figure 7 FIG. 4 shows an oscillating texture feature in a target feature texture vector image according to an embodiment of the present application. Figure 7 As shown, Figure 7 The target feature texture vector image is obtained by convolving the reflectivity information image to be measured with the two-dimensional oscillating texture filter. Figure 7 The oscillating texture features of zinc spangles in the target feature texture vector image are shown. The two-dimensional oscillating texture filter is obtained by texture convolution of two one-dimensional oscillating texture filters.

[0080] In one embodiment of the present application, see Figure 8 , Figure 8 FIG. 4 shows a spot texture feature in a target feature texture vector image according to an embodiment of the present application. Figure 8 As shown, Figure 8The target feature texture vector image is obtained by convolving the reflectivity information image to be measured with the two-dimensional speckle texture filter. Figure 8 The speckle texture features of zinc spangles in the target feature texture vector image are shown. The two-dimensional speckle texture filter is obtained by texture convolution of two one-dimensional speckle texture filters.

[0081] In one embodiment of the present application, see Figure 9 , Figure 9 FIG. 1 shows the grayscale edge texture features in the target feature texture vector image according to an embodiment of the present application. Figure 9 As shown, Figure 9 The target feature texture vector image is obtained by convolving the reflectivity information image to be measured with the two-dimensional grayscale edge texture filter. Figure 9 The grayscale edge texture features of zinc spangles in the target feature texture vector image are shown. The two-dimensional speckle texture filter is obtained by texture convolution of a one-dimensional grayscale texture filter and a one-dimensional edge texture filter.

[0082] In one embodiment of the present application, the detection reflectivity information image is detected and convolved with multiple preset two-dimensional texture filters to obtain multiple target feature texture vector images, including: detecting and convolving the detection reflectivity information image with each preset two-dimensional texture filter to obtain multiple feature texture filter images to be filtered; and locally averaging filtering each feature texture filter image to be filtered through a preset smoothing filter to obtain multiple target feature texture vector images.

[0083] Step S230: Input all target feature texture vector images into the crystal flower classification model to perform crystal flower size classification to obtain the target crystal flower size category.

[0084] In one embodiment of the present application, each target feature texture vector image is input into a spangle classification model, which then performs spangle size classification to obtain a target spangle size category for the coated metal plate to be tested. If the spangles are zinc spangles, the target spangle size category includes large zinc spangles, small zinc spangles, and no zinc spangles.

[0085] Step S240, based on multiple preset scanning directions, calculates the energy feature quantities corresponding to multiple grayscale co-occurrence matrices of the reflectivity information image to be tested, and obtains the characteristic energy mean value based on all energy feature quantities to determine the uniformity of the crystal flower texture distribution, and uses the target crystal flower size category and the uniformity of the crystal flower texture distribution as the crystal flower detection result of the coated metal plate to be tested.

[0086] In one embodiment of the present application, energy feature quantities corresponding to multiple grayscale co-occurrence matrices of the reflectivity information image to be measured are calculated based on multiple preset scanning directions, and the characteristic energy mean value obtained based on all energy feature quantities is used to determine the uniformity of the crystal flower texture distribution, including: calculating multiple grayscale co-occurrence matrices of the reflectivity information image to be measured in different preset scanning directions; determining the energy feature quantity of each grayscale co-occurrence matrix based on each grayscale co-occurrence matrix; averaging each energy feature quantity to obtain the characteristic energy mean value; if the characteristic energy mean value is greater than or equal to the preset energy mean value, the uniformity of the crystal flower texture distribution is determined to be uniform; if the characteristic energy mean value is less than the preset energy mean value, the uniformity of the crystal flower texture distribution is determined to be uneven; wherein, the preset scanning directions include horizontal direction, vertical direction, right diagonal direction and left diagonal direction.

[0087] In one embodiment of the present application, multiple grayscale co-occurrence matrices corresponding to the reflectivity information image to be measured in the horizontal direction, vertical direction, right diagonal direction and left diagonal direction are calculated, and the ASM (angular second moment) energy of each grayscale co-occurrence matrix, that is, the energy characteristic quantity, is calculated to evaluate the uniformity of the crystal flower texture distribution and the texture coarseness of the reflectivity information image to be measured based on the energy characteristic quantity. A large mean value of the characteristic energy indicates that the texture is regularly changing and relatively stable, that is, the uniformity of the crystal flower texture distribution is uniform.

[0088] In one embodiment of the present application, the horizontal direction is 0°, the vertical direction is 45°, the right diagonal direction is 90°, and the left diagonal direction is 135°.

[0089] In one embodiment of the present application, see Figure 10 , Figure 10 FIG. 1 shows a schematic diagram of the implementation process of the zinc spangle detection method according to an embodiment of the present application. Figure 10As shown, the zinc spangle detection method includes a zinc spangle classification model training process and a zinc spangle classification model detection process. The zinc spangle classification model training process includes step S1011 to obtain multiple initial zinc spangle images of multiple size categories of zinc spangles: obtain multiple initial zinc spangle images of multiple size categories of zinc spangles, training light source-camera optical axis angle information and zinc spangle size classification labels corresponding to each initial zinc spangle image; step S1012 to reconstruct the training reflectivity information image by using the preset photometric stereo method: perform a second image reconstruction on multiple initial zinc spangle images of a size category of zinc spangles according to the training light source-camera optical axis angle information and the preset photometric stereo equation to obtain a training reflectivity information image; step S1013 to perform training convolution with multiple preset two-dimensional texture filters to generate multiple training feature texture filter images: convolve the training reflectivity information image with each preset two-dimensional texture filter respectively. Convolution is performed to obtain multiple training feature texture filter images; step S1014 performs local average filtering with a preset smoothing filter to generate multiple training feature texture vector images: each training feature texture filter image is locally average filtered through a preset smoothing filter to obtain multiple training feature texture vector images; step S1015 creates an initial classification model and trains to obtain a zinc flower classification model: the initial classification model is generated based on an MLP neural network classifier, the initial classification model includes a feature vector preprocessing training function and an activation function of the output layer, and the zinc flower size classification label of a size category zinc flower and each training feature texture vector image of a size category zinc flower are used as a training sample, and a training sample is input into the initial classification model for classification training to obtain a zinc flower classification model.The detection process of applying the zinc spangle classification model includes step S1021 obtaining multiple zinc spangle images to be tested of the coated metal plate to be tested: obtaining multiple zinc spangle images to be tested of the coated metal plate to be tested and the detection light source-camera optical axis angle information; step S1022 reconstructing the reflectivity information image to be tested by the preset photometric stereo method: performing a first image reconstruction based on the detection light source-camera optical axis angle information and multiple zinc spangle images to be tested to obtain the reflectivity information image to be tested; step S1023 performing detection convolution with multiple preset two-dimensional texture filters to generate multiple feature texture filter images to be filtered: performing detection convolution on the detection reflectivity information image and each preset two-dimensional texture filter respectively to obtain multiple feature texture filter images to be filtered; step S1024 performing local average filtering with a smoothing filter to generate multiple detection feature texture filters. Step S1025: Classify the spangle size by the spangle classification model to obtain the target spangle size category. Input all target feature texture vector images into the spangle classification model to classify the spangle size and obtain the target spangle size category. Step S1026: Calculate the uniformity of the spangle texture distribution of the reflectivity information image to be measured. Calculate the energy feature quantities corresponding to the multiple grayscale co-occurrence matrices of the reflectivity information image to be measured based on multiple preset scanning directions, and determine the uniformity of the spangle texture distribution based on the feature energy mean obtained from all the energy feature quantities. The target spangle size category and the uniformity of the spangle texture distribution are used as the spangle detection results of the coated metal plate to be measured.

[0090] See also Figure 11 , Figure 11 A block diagram of a crystal flower detection device according to an embodiment of the present application is shown. The device can be applied to Figure 1 The implementation environment shown is specifically configured in the computer device 102. The apparatus may also be applicable to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the apparatus is applicable.

[0091] like Figure 11 As shown, a crystal flower detection device 1100 according to an embodiment of the present application includes: an image acquisition module 1101, a texture filtering module 1102, a crystal flower classification module 1103 and a uniformity determination module 1104.

[0092] An image acquisition module 1101 is configured to acquire a plurality of crystal flower images to be measured and detection light source-camera optical axis angle information of the coated metal plate to be measured, and perform a first image reconstruction based on the detection light source-camera optical axis angle information and the plurality of crystal flower images to be measured to obtain a reflectivity information image to be measured, wherein the plurality of crystal flower images to be measured are collected by a camera module when the metal plate to be measured is successively irradiated by light source modules in different preset detection directions;

[0093] The texture filtering module 1102 is used to perform detection convolution on the reflectivity information image to be measured and a plurality of preset two-dimensional texture filters to obtain a plurality of target feature texture vector images;

[0094] The crystal flower classification module 1103 is used to input all target feature texture vector images into the crystal flower classification model to perform crystal flower size classification and obtain the target crystal flower size category;

[0095] A uniformity determination module 1104 is configured to calculate energy feature quantities corresponding to multiple gray-level co-occurrence matrices of the reflectivity information image to be measured based on multiple preset scanning directions, and determine the uniformity of the crystal flower texture distribution based on the characteristic energy mean obtained from all the energy feature quantities. The target crystal flower size category and the uniformity of the crystal flower texture distribution are used as the crystal flower detection result of the coated metal plate to be measured;

[0096] Among them, the crystal flower classification model is obtained by training multiple initial crystal flower images and the crystal flower size classification labels corresponding to each initial crystal flower image.

[0097] The crystal flower detection device according to one embodiment of the present application further includes: an image capture module;

[0098] The image capture module includes a coated metal plate to be tested, a camera module and at least three light source modules;

[0099] The camera optical axis of the camera module is perpendicular to the surface to be tested of the coated metal plate to be tested;

[0100] Each light source module is installed around the camera module according to different preset detection light source directions, and is used to successively illuminate the coated metal plate to be tested;

[0101] The camera module is used to capture the image of each crystal flower to be tested that is successively illuminated by each light source module on the coated metal plate to be tested.

[0102] It should be noted that the crystal flower detection device provided in the above embodiment and the crystal flower detection method provided in the above embodiment are based on the same concept. The specific manner in which each module and unit performs operations has been described in detail in the method embodiment and will not be repeated here. In actual applications, the crystal flower detection device provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0103] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, which, when executed by one or more processors, enables the electronic device to implement the crystal flower detection method provided in the above-mentioned embodiments.

[0104] See also Figure 12 , Figure 12 The following is a schematic diagram showing the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present application. Figure 12 The computer system 1200 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0105] like Figure 12 As shown, the computer system 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1202 or the program loaded from the storage part 1208 to the random access memory (RAM) 1203, such as executing the method in the above embodiment. Various programs and data required for system operation are also stored in the RAM 1203. The CPU 1201, ROM 1202 and RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0106] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, and the like; an output section 1207 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and speakers; a storage section 1208 including a hard disk; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. Removable media 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 1210 as needed, so that computer programs read from the removable media can be installed in the storage section 1208 as needed.

[0107] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1209, and / or installed from a removable medium 1211. When the computer program is executed by the central processing unit (CPU) 1201, the various functions defined in the system of the present application are executed.

[0108] It should be noted that the computer-readable medium shown in the embodiments of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable computer program. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0109] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. Among them, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0110] The units involved in the embodiments described in the present application can be implemented by software or by hardware, and the units described can also be set in a processor. The names of these units do not constitute a limitation on the units themselves under certain circumstances. Therefore, the technical solution according to the embodiment of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiment of the present application.

[0111] Another aspect of the present application provides a computer-readable storage medium storing a computer program. When executed by a computer processor, the computer program causes the computer to perform the crystal flower detection methods provided in the aforementioned embodiments. The computer-readable storage medium may be included in the electronic device described in the aforementioned embodiments, or may exist independently and not be incorporated into the electronic device.

[0112] In the above embodiments, unless otherwise specified, the use of serial numbers such as "first" and "second" to describe common objects only indicates that they refer to different instances of the same object, rather than indicating that the objects being described must adopt a given order, whether in time, space, sorting or any other way.

[0113] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, any equivalent modifications or alterations accomplished by a person of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.

Claims

1. A crystal flower detection method, characterized in that: The method comprises: Acquire multiple crystal flower images to be tested and detection light source-camera optical axis angle information of the coated metal plate to be tested, and perform a first image reconstruction based on the detection light source-camera optical axis angle information and the multiple crystal flower images to be tested to obtain a reflectivity information image to be tested, wherein the multiple crystal flower images to be tested are collected by a camera module when the metal plate to be tested is successively irradiated by light source modules of different preset detection directions; Performing detection convolution on the reflectivity information image to be measured and a plurality of preset two-dimensional texture filters respectively to obtain a plurality of target feature texture vector images; Inputting all the target feature texture vector images into a crystal flower classification model to perform crystal flower size classification to obtain a target crystal flower size category; Calculate the energy feature quantities corresponding to multiple gray-level co-occurrence matrices of the reflectivity information image to be measured based on multiple preset scanning directions, and determine the uniformity of the crystal flower texture distribution according to the characteristic energy mean value obtained from all the energy feature quantities, and use the target crystal flower size category and the uniformity of the crystal flower texture distribution as the crystal flower detection result of the coated metal plate to be measured; The crystal flower classification model is obtained by training multiple initial crystal flower images and crystal flower size classification labels corresponding to each of the initial crystal flower images.

2. The crystal flower detection method according to claim 1, wherein Before obtaining a plurality of crystal flower images to be tested of the coated metal plate to be tested and detecting light source-camera optical axis angle information, the method further includes: Acquire multiple initial crystal flower images of multiple size categories of crystal flowers, training light source-camera optical axis angle information, and crystal flower size classification labels corresponding to each of the initial crystal flower images, wherein the multiple initial crystal flower images of a size category of crystal flowers are collected by a camera module when the training coated metal plate is successively illuminated by light source modules with different preset training light source directions; Performing a second image reconstruction based on the training light source-camera optical axis angle information and the multiple initial crystal flower images of the crystal flower of the one size category to obtain a training reflectivity information image; Performing training convolution on the training reflectivity information image and a plurality of preset two-dimensional texture filters respectively to obtain a plurality of training feature texture vector images; The crystal flower size classification label of the crystal flower of the size category and each of the training feature texture vector images of the crystal flower of the size category are used as a training sample, and the training sample is input into the initial classification model for classification training to obtain a crystal flower classification model.

3. The crystal flower detection method according to claim 2, wherein The training reflectivity information image is respectively subjected to training convolution with a plurality of preset two-dimensional texture filters to obtain a plurality of training feature texture vector images, including: Performing training convolution on the training reflectivity information image and each of the preset two-dimensional texture filters to obtain a plurality of training feature texture filter images; Each of the training feature texture filter images is subjected to local average filtering through a preset smoothing filter to obtain a plurality of training feature texture vector images.

4. The crystal flower detection method according to claim 2, wherein The second image reconstruction is performed based on the training light source-camera optical axis angle information and the multiple initial crystal flower images of the crystal flower of the one size category, and the training reflectivity information image is obtained, including: A second image reconstruction is performed on the multiple initial crystal flower images of the crystal flower of the one size category according to the training light source-camera optical axis angle information and the preset photometric stereo equation to obtain a training reflectivity information image.

5. The crystal flower detection method according to claim 1, wherein Calculating energy feature quantities corresponding to multiple gray-level co-occurrence matrices of the reflectivity information image to be measured based on multiple preset scanning directions, and determining the uniformity of crystal flower texture distribution according to the characteristic energy mean value obtained from all the energy feature quantities includes: Calculating multiple gray-level co-occurrence matrices of the reflectivity information image to be measured in different preset scanning directions; Determining the energy characteristic value of each gray level co-occurrence matrix according to each gray level co-occurrence matrix; Averaging the energy characteristic quantities to obtain a characteristic energy mean; If the characteristic energy mean is greater than or equal to the preset energy mean, the uniformity of the crystal flower texture distribution is determined to be uniform; If the characteristic energy mean is less than the preset energy mean, the uniformity of the crystal flower texture distribution is determined to be uneven; The preset scanning directions include a horizontal direction, a vertical direction, a right diagonal direction and a left diagonal direction.

6. The crystal flower detection method according to any one of claims 1 to 5, characterized in that A method for determining a preset two-dimensional texture filter includes: generating a plurality of one-dimensional texture filters based on preset texture features; Perform texture convolution on two different one-dimensional texture filters or two identical one-dimensional texture filters to obtain a preset two-dimensional texture filter; The one-dimensional texture filters include a one-dimensional grayscale texture filter, a one-dimensional edge texture filter, a one-dimensional ripple texture filter, a one-dimensional speckle texture filter, and a one-dimensional oscillation texture filter.

7. A crystal flower detection device, characterized in that: The device comprises: An image acquisition module is used to obtain multiple crystal flower images to be tested and detection light source-camera optical axis angle information of the coated metal plate to be tested, and perform a first image reconstruction based on the detection light source-camera optical axis angle information and the multiple crystal flower images to be tested to obtain a reflectivity information image to be tested, wherein the multiple crystal flower images to be tested are collected by a camera module when the metal plate to be tested is successively irradiated by light source modules with different preset detection directions; A texture filtering module is used to perform detection convolution on the reflectivity information image to be measured and a plurality of preset two-dimensional texture filters to obtain a plurality of target feature texture vector images; A crystal flower classification module is used to input all the target feature texture vector images into a crystal flower classification model to perform crystal flower size classification to obtain a target crystal flower size category; a uniformity determination module, configured to calculate energy feature quantities corresponding to multiple gray-level co-occurrence matrices of the reflectivity information image to be measured based on multiple preset scanning directions, and determine the uniformity of the crystal flower texture distribution according to the characteristic energy mean value obtained from all the energy feature quantities, and use the target crystal flower size category and the uniformity of the crystal flower texture distribution as the crystal flower detection result of the coated metal plate to be measured; The crystal flower classification model is obtained by training multiple initial crystal flower images and crystal flower size classification labels corresponding to each of the initial crystal flower images.

8. The crystal flower detection device according to claim 7, characterized in that The crystal flower detection device also includes an image capture module; The image capture module includes a coated metal plate to be tested, a camera module and at least three light source modules; The camera optical axis of the camera module is perpendicular to the surface to be tested of the coated metal plate to be tested; Each of the light source modules is installed around the camera module according to different preset detection light source directions, and is used to successively illuminate the coated metal plate to be tested; The camera module is used to capture images of each crystal flower to be tested that is successively illuminated by each of the light source modules on the coated metal plate to be tested.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the crystal flower detection method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the crystal flower detection method according to any one of claims 1 to 6.

Images