Zinc flower grading method and device based on machine vision, terminal and medium

Abstract

The application provides a zinc flower rating method and device based on machine vision, a terminal and a medium, including: segmenting multiple zinc flower images according to a preset pixel size and manually labeling zinc nucleation points to obtain corresponding unit images and labels; inputting the unit images and the labels into a neural network model for training and verification to output a density map of zinc nucleation points of each unit image, so as to obtain the number of zinc nucleation points of each unit image; fitting the density map of each unit image by using a clustering algorithm to obtain the position coordinates of the zinc nucleation points of each zinc flower in the unit image, so as to obtain the nearest neighbor distance of the zinc nucleation points and calculate the average value of the nearest neighbor distance of the zinc nucleation points corresponding to the unit image; taking the number of zinc nucleation points of each unit image and the average value of the nearest neighbor distance of the zinc nucleation points as a feature vector of zinc flower rating, inputting the feature vector into a classifier for training to obtain a zinc flower classification model, so as to realize automatic rating of zinc flowers.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and in particular to a machine vision-based zinc spangle rating method, apparatus, terminal and medium. Background Technology

[0002] Online inspection of galvanized surface quality, especially real-time monitoring of the uniformity of zinc spangle size on the galvanized sheet surface, is crucial to the quality of galvanized sheets. Currently, zinc spangle grading on the production line relies solely on visual inspection by on-site quality inspectors. The results depend on the inspectors' experience, are subjective, and prone to false positives and false negatives, resulting in low inspection efficiency. Therefore, enabling real-time detection of the uniformity and grade of zinc spangle size on galvanized sheets during operation is of great significance for optimizing galvanizing process parameters and reducing production costs.

[0003] In addition, there are some products on the market that use computer technology to detect defects on textured surfaces, but these mainly focus on identifying and classifying the texture of workpiece surface images and cannot solve the problem of automatic identification of zinc spangle levels. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a machine vision-based zinc spangle rating method, apparatus, terminal and medium to solve the technical problem of the inability to automatically perform zinc spangle rating.

[0005] To achieve the above and other related objectives, the first aspect of this application provides a zinc flower rating method based on machine vision, comprising: segmenting multiple zinc flower images according to a preset pixel size and manually annotating the zinc flower core points to obtain corresponding unit images and labels; inputting the unit images and their labels into a neural network model for training and validation to output a density map of the zinc flower core points of each unit image, thereby obtaining the number of core points in each unit image; using a clustering algorithm to fit the density map of each unit image to obtain the position coordinates of the core points of each zinc flower in the unit image, thereby obtaining the nearest neighbor distance of the core points and calculating the average value of the nearest neighbor distance of the core points corresponding to the unit image; using the number of core points and the average value of the nearest neighbor distance of the core points of each unit image as the feature vector for zinc flower rating, inputting it into a classifier for training to obtain a zinc flower classification model, which outputs the corresponding automatic rating result after inputting the zinc flower image to be rated into the zinc flower rating model; the rating model includes a convolutional neural network, a Gaussian mixture model, and a classification model.

[0006] In some embodiments of the first aspect of this application, before segmenting the zinc flower image, an image enhancement algorithm and an image smoothing algorithm are used to preprocess the zinc flower image to enhance the image contrast and eliminate image noise interference.

[0007] In some embodiments of the first aspect of this application, a clustering algorithm is used to fit the density map of each unit image to obtain the nucleation point coordinates of each zinc flower in the unit image and to calculate the nearest neighbor distance of the nucleation point. This includes: fitting the density map of each unit image using a clustering algorithm, extracting the center point position of the cluster in the fitting result, and obtaining the nucleation point coordinates of the zinc flower image accordingly; searching for the nearest neighboring zinc flower nucleation point based on the nucleation point coordinates of the zinc flower image, and obtaining the nearest neighbor distance of the nucleation point of the zinc flower image accordingly.

[0008] In some embodiments of the first aspect of this application, the step of inputting unit images and their labels into a neural network model for training and verification includes inputting a portion of the unit images and their labels as a training set into a convolutional neural network or an attention model for training; and using the remaining unit images and their labels as a test set to test and verify the trained neural network model.

[0009] In some embodiments of the first aspect of this application, the clustering algorithm includes any one of the following: Gaussian mixture model clustering algorithm, K-means clustering algorithm, DBSCAN clustering algorithm, OPTICS clustering algorithm, or BIRCH clustering algorithm.

[0010] In some embodiments of the first aspect of this application, the classification model includes any one of the following: decision tree classification model, support vector machine classification model, random forest classification model, or Bayesian classification model.

[0011] In some embodiments of the first aspect of this application, the process of forming manually annotated zinc flower core points includes: after arbitrarily annotating a zinc flower core point, marking its neighboring zinc flower core points based on the position of the zinc flower core point, until the annotation of all core points on the zinc flower image is completed.

[0012] To achieve the above and other related objectives, a second aspect of this application provides a zinc flower rating device based on machine vision, comprising: a segmentation and labeling module, used to segment multiple zinc flower images according to a preset pixel size and manually label the zinc flower core points to obtain corresponding unit images and labels; a model training module, used to input the unit images and their labels into a neural network model for training and verification, so as to output a density map of the zinc flower core points of each unit image, thereby obtaining the number of core points in each unit image; an image fitting module, used to use a clustering algorithm to fit the density map of each unit image to obtain the position coordinates of the core points of each zinc flower in the unit image, thereby obtaining the nearest neighbor distance of the core points and calculating the average value of the nearest neighbor distance of the core points corresponding to the unit image; and a zinc flower rating module, used to use the number of core points and the average value of the nearest neighbor distance of the core points of each unit image as the feature vector for zinc flower rating, inputting it into a classifier for training to obtain a zinc flower classification model, so as to output the corresponding automatic rating result after inputting the zinc flower image to be rated into the zinc flower rating model; the rating model includes a convolutional neural network, a Gaussian mixture model, and a classification model.

[0013] To achieve the above and other related objectives, a third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the machine vision-based zinc spangle rating method.

[0014] To achieve the above and other related objectives, a fourth aspect of this application provides an electronic terminal, comprising: a processor and a memory; the memory for storing a computer program, and the processor for executing the computer program stored in the memory, so that the terminal performs the machine vision-based zinc spangle rating method.

[0015] As described above, the machine vision-based zinc spangle rating method, apparatus, terminal, and medium of this application have the following beneficial effects:

[0016] (1) The zinc flower rating method proposed in this invention not only automatically and correctly judges the grade of zinc flower plate, but also describes the essential characteristics of zinc flower, the number of zinc flower core points and the average nearest neighbor distance of each zinc flower core point.

[0017] (2) This invention enables in-service zinc spangle quality inspection without the need for manual inspection. It utilizes machine vision and artificial intelligence technologies to automatically grade zinc spangle, which not only avoids false detection caused by manual inspection, but also greatly improves the inspection efficiency and traceability of zinc spangle grading. Attached Figure Description

[0018] Figure 1 The diagram shown is a flowchart of a machine vision-based zinc spangle rating method according to an embodiment of this application.

[0019] Figure 2A The image shown is a schematic diagram of a zinc flower from one embodiment of this application.

[0020] Figure 2B The diagram shown is a density map of manually labeled results in one embodiment of this application.

[0021] Figure 2C The diagram shown is a density map of the recognition results of a neural network model in one embodiment of this application.

[0022] Figure 3 The diagram shows an algorithm flow diagram based on a convolutional neural network and decision tree classification model in one embodiment of this application.

[0023] Figure 4 The diagram shown is a structural schematic of an electronic terminal according to an embodiment of this application.

[0024] Figure 5 The diagram shown is a schematic representation of a machine vision-based zinc flower rating device according to an embodiment of this application. Detailed Implementation

[0025] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0026] As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms “comprising,” “including,” indicate the presence of the stated feature, operation, element, component, item, kind, and / or group, but do not preclude the presence, occurrence, or addition of one or more other features, operations, elements, components, items, kinds, and / or groups. The terms “or” and “and / or” as used herein are interpreted as inclusive, or mean any one or any combination thereof. Thus, “A, B, or C” or “A, B, and / or C” means “any one of: A; B; C; A and B; A and C; B and C; A, B, and C.” Exceptions to this definition occur only when combinations of elements, functions, or operations are inherently mutually exclusive in some manner.

[0027] To address the problems mentioned above in the background technology, this invention provides a zinc spangle rating method, device, terminal, and medium based on machine vision. The aim is to use machine vision algorithms to identify and rate zinc spangles on galvanized sheets. Specifically, it includes image preprocessing, definition and annotation of essential features of zinc spangles, identification and extraction of essential features of zinc spangles based on neural networks and clustering algorithms, and judgment of zinc spangle level using a classification model.

[0028] Meanwhile, to make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the invention.

[0029] Before providing a further detailed description of the present invention, the nouns and terms used in the embodiments of the present invention are explained, and the nouns and terms used in the embodiments of the present invention are subject to the following interpretations:

[0030] <1> Machine vision: A rapidly developing branch of artificial intelligence, it involves using machines to replace human eyes for measurement and judgment. A machine vision system uses machine vision products (i.e., CMOS and CCD image acquisition devices) to convert the captured target into image signals, which are then transmitted to a dedicated image processing system. This system obtains the target's shape information and, based on pixel distribution, brightness, color, and other information, converts it into digital signals. The image processing system performs various calculations on these signals to extract the target's features, and then controls the on-site equipment based on the judgment results.

[0031] <2> Zinc flowers: When the hot-dip galvanized coating on the surface of the steel sheet solidifies, crystalline patterns of the coating metal appear. These crystalline patterns are particularly noticeable on galvanized sheets, presenting a beautiful crystalline pattern appearance, and thus becoming an important feature of the appearance of hot-dip galvanized sheets. These crystalline patterns are usually called zinc flowers.

[0032] This invention provides a machine vision-based zinc spangle rating method, a system for implementing the machine vision-based zinc spangle rating method, and a storage medium storing an executable program for implementing the machine vision-based zinc spangle rating method. Regarding the implementation of the machine vision-based zinc spangle rating method, this invention will describe exemplary implementation scenarios of machine vision-based zinc spangle rating.

[0033] like Figure 1 The diagram illustrates a flowchart of a machine vision-based zinc spangle rating method according to an embodiment of the present invention. The machine vision-based zinc spangle rating method in this embodiment mainly includes the following steps:

[0034] Step S11: Divide multiple zinc flower images into preset pixel sizes and manually label the core points of the zinc flower shapes to obtain the corresponding unit images and labels.

[0035] In this embodiment, before segmenting the zinc flower image, image enhancement and image smoothing algorithms are used to preprocess the zinc flower image to enhance image contrast and eliminate noise interference. The image enhancement algorithms include, but are not limited to, histogram equalization-based image enhancement algorithms, object log transform-based image enhancement algorithms, and Laplacian operator-based image enhancement algorithms; the image smoothing algorithms include, but are not limited to, Gaussian filtering algorithms, median filtering algorithms, and mean filtering algorithms.

[0036] Specifically, histogram equalization-based image enhancement algorithms are methods in image processing that use image histograms to adjust contrast, increasing the overall contrast of an image. Object log transform-based image enhancement algorithms expand the low-grayscale portions of an image, revealing more detail in these areas, while compressing the high-grayscale portions to reduce detail, thus emphasizing the grayscale aspects of the image. Laplacian-based image enhancement algorithms utilize the Laplacian operator to sharpen the image and improve contrast, essentially using the second derivative of the image. Gaussian filtering is a linear smoothing filter used to eliminate Gaussian noise. It works by weighted averaging across the entire image; the value of each pixel is obtained by averaging its own value and the values ​​of its neighboring pixels. Median filtering is a non-linear smoothing technique that sets the grayscale value of each pixel to the median of all pixels within its neighborhood window. Mean filtering involves creating a template for the target pixel in the image, including its neighboring pixels, and then replacing the original pixel value with the average value of all pixels in the template.

[0037] In some examples, zinc flower images can be acquired using industrial cameras. Industrial cameras are video image acquisition devices applicable to industrial environments. They can directly store images on a hard drive. Compared to ordinary cameras, industrial cameras offer high comparability in resolution, frame rate, light requirements, and exposure methods. Their main component is a CCD image sensor. Major types of industrial cameras include area-array CCD industrial cameras, line-array CCD industrial cameras, three-line sensor CCD industrial cameras, interleaved transmission CCD industrial cameras, and full-frame CCD industrial cameras.

[0038] In this embodiment, the process of dividing multiple zinc flower images according to a preset pixel size and manually annotating the zinc flower core points yields corresponding unit images and labels. For example, the preprocessed zinc flower images can be divided into 350*350 pixel sizes, and the zinc flower core points on the images can be manually annotated to obtain corresponding 350*350 pixel size unit images and their labels.

[0039] In some examples, based on the growth mechanism of zinc flowers, the essential characteristics of zinc flowers are defined and labeled. The size of zinc flowers is affected by the distance between the nucleation points (crystallization points of zinc flower nucleation). The primary dendrite length of zinc flowers reflects the distance between adjacent nucleation points and can accurately quantify the size of zinc flowers. Therefore, this invention defines the primary dendrite length of zinc flowers as the essential characteristic for zinc flower identification. It should be understood that the primary dendrite length of zinc flowers refers to the dendritic crystal growth of grains caused by the appearance of a temperature gradient in front of the solidification interface during the solidification process. The earliest batch of grains that grow is called primary dendrites. The key to zinc flower rating in this application lies in the identification of the essential characteristics of zinc flowers, namely, the nucleation points and the primary dendrite length. Since primary dendrites are difficult to identify macroscopically, the average nearest neighbor distance of the nucleation points is used instead.

[0040] In some examples, the zinc flower nuclei need to be manually labeled before automatic identification. Therefore, based on the growth pattern of zinc flowers, there will be several adjacent zinc flowers around a zinc flower. After labeling a nuclei, the neighboring zinc flower nuclei are labeled based on that nuclei, until all the nuclei on the map are labeled.

[0041] Step S12: Input the unit image and its label into the neural network model for training and validation, so as to output the density map of the zinc flower core points of each unit image, and obtain the number of core points of each unit image.

[0042] Specifically, all unit images form an image dataset, which is then divided into a training set and a test set according to a certain ratio. The unit images and their corresponding labels from the training set are input into the neural network model for training. After training, the unit images and their corresponding labels from the test set are used to test and validate the model until a satisfactory neural network model is obtained. In this embodiment, the neural network model can use a convolutional neural network or an attention model, etc. A convolutional neural network is a feedforward neural network that includes convolutional computation and has a deep structure; it is one of the representative algorithms of deep learning. Specific convolutional neural networks that can be used in this embodiment include, for example, LeNet-5, VGGNet, GoogleNet, ResNet, DenseNet, and MobileNet. An attention model is a model that simulates human brain attention; that is, when a person's eyes observe an image, the human brain's attention to the entire image is not balanced but rather differentiated by certain weights.

[0043] Different levels of zinc flower images, along with manually labeled data, are input into a neural network model (convolutional neural network or attention model, etc.) to identify the zinc flower nuclei in the images and generate a density map, thereby obtaining the number of nuclei. Figures 2A-2C As shown, Figure 2A It is a zinc flower image. Figure 2B This is a density map of manually labeled results. Figure 2C It is a density map of the recognition results of the neural network model.

[0044] Step S13: Use a clustering algorithm to fit the density map of each unit image to obtain the nucleation point coordinates of each zinc flower in the unit image, thereby obtaining the nearest neighbor distance of the nucleation point and calculating the average value of the nearest neighbor distance of the nucleation point corresponding to the unit image.

[0045] Specifically, the distance between the zinc flower core points refers to the distance between a zinc flower core point and its neighboring zinc flower core points; as the name suggests, the nearest neighbor distance of each zinc flower core point refers to the distance between each zinc flower and its nearest neighboring zinc flower core point; and the average value of the nearest neighbor distance of the core points refers to, for all zinc flowers in a unit image, first calculating the nearest neighbor distance of the core points corresponding to each zinc flower, and then calculating the average value of the nearest neighbor distance of the core points of all zinc flowers.

[0046] In this embodiment, a clustering algorithm is used to fit the density map of each unit image to obtain the nearest neighbor distance of the nucleation point of each zinc flower in the unit image. Specifically, this includes: fitting the density map of each unit image using a clustering algorithm, extracting the center point position of the cluster in the fitting result, and obtaining the nucleation point coordinates of the zinc flower image; searching for the nearest neighboring zinc flower nucleation point based on the nucleation point coordinates of the zinc flower image, and obtaining the nearest neighbor distance of the nucleation point of the zinc flower image.

[0047] In this embodiment, clustering algorithms such as Gaussian mixture model, K-means clustering, DBSCAN clustering, OPTICS clustering, and BIRCH clustering can be used to fit the generated density map, extract the center point position of the cluster in the fitting result, obtain the nucleation point coordinates of the zinc flower image, and calculate the nearest neighbor distance of the nucleation point to quantify the length of the primary dendrite of the zinc flower.

[0048] Step S14: The number of nucleus points and the average nearest neighbor distance of the nucleus points in each unit image are used as the feature vector for zinc flower rating. The vector is input into the classifier for training to obtain the zinc flower classification model. The model is used to output the corresponding automatic rating result after the zinc flower image to be rated is input into the zinc flower rating model. The rating model includes a convolutional neural network, a Gaussian mixture model and a classification model.

[0049] In this embodiment, when rating zinc flower, the number of nucleation points and the average of the nearest neighbor distances between nucleation points are used as the feature vectors for rating zinc flower. As mentioned earlier, the zinc flower image to be rated is first divided into several unit images, and the number of nucleation points and the nearest neighbor distances between nucleation points are obtained. For each unit image, the average of multiple nearest neighbors is calculated. The number of nucleation points and the average of the nearest neighbor distances between nucleation points are used as two input feature vectors to train a classification model (decision tree classification model, support vector machine classification model, random forest classification model, Bayesian classification model, etc.). Then, the rating result of each unit image is given, thereby statistically obtaining the level of the zinc flower image to be rated, and finally realizing the automatic identification of the zinc flower level.

[0050] Now Figure 3 The flowchart illustrating the algorithm based on convolutional neural networks and decision tree classification models is provided to help those skilled in the art better understand the implementation process of the machine vision-based zinc flower rating method in this invention:

[0051] Step 1: Use an industrial area array camera to photograph the zinc flower plates of grades 2, 3, and 4 respectively, to obtain original zinc flower images with a pixel size of 2350*2000.

[0052] Step 2: After histogram equalization and mean filtering, the original zinc flower image is divided into 350*350 pixel unit images, and the zinc flower core points on the unit images are manually labeled.

[0053] Step 3: Divide the unit images and their labels into training and validation sets in an 8:2 ratio. Input the training set images and their labels into the MCNN (Multi-Column Convolutional Neural Network) convolutional neural network, and output the density map of zinc flower kernel points for each unit image to obtain the number of kernel points.

[0054] Step 4: Fit the output density map using a Gaussian mixture model to obtain the two-dimensional coordinates of the nucleation points, calculate the nearest neighbor distance of the nucleation points representing the first dendrite length, and take the average of the nearest neighbor distances between the nucleation points in each unit image.

[0055] Step 5: Input the number of kernel points and the average nearest neighbor distance of each unit image as two feature vectors into the decision tree classification model, and output the automatic zinc flower classification result of the unit image. The decision tree classification model has a rating accuracy of up to 93.6% on the validation set of unit images.

[0056] Step 6: Based on the rating results of the unit images, the level of the zinc flower image (2350*2000 pixels) to be rated is finally obtained, and the rating accuracy reaches 100%.

[0057] The machine vision-based zinc spangle rating method provided in this invention can be implemented on the terminal side or the server side. For the hardware structure of the machine vision-based zinc spangle rating terminal, please refer to [link to relevant documentation]. Figure 4 This is a schematic diagram of an optional hardware structure of a zinc spangle grading terminal 400 based on machine vision provided in an embodiment of the present invention. The terminal 400 can be a mobile phone, computer device, tablet device, personal digital processing device, factory back-end processing device, etc. The machine vision-based zinc spangle grading terminal 400 includes: at least one processor 401, a memory 402, at least one network interface 404, and a user interface 406. The various components in the device are coupled together through a bus system 405. It is understood that the bus system 405 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 405 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 4 The general will label all buses as bus systems.

[0058] The user interface 406 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0059] It is understood that memory 402 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable categories of memory.

[0060] In this embodiment of the invention, the memory 402 is used to store various categories of data to support the operation of the machine vision-based zinc spangle rating terminal 400. Examples of this data include: any executable program for operation on the machine vision-based zinc spangle rating terminal 400, such as operating system 4021 and application program 4022; operating system 4021 includes various system programs, such as framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks. Application program 4022 may include various applications, such as media player, browser, etc., for implementing various application services. The implementation of the machine vision-based zinc spangle rating method provided in this embodiment of the invention can be included in application program 4022.

[0061] The methods disclosed in the above embodiments of the present invention can be applied to processor 401, or implemented by processor 401. Processor 401 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 401 or by instructions in the form of software. The processor 401 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 401 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. General-purpose processor 401 may be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in a memory. The processor reads the information in the memory and combines it with its hardware to complete the steps of the aforementioned method.

[0062] In an exemplary embodiment, the machine vision-based zinc flower rating terminal 400 may be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs) to perform the aforementioned method.

[0063] like Figure 5The diagram illustrates a structural schematic of a zinc spangle rating device based on machine vision, according to an embodiment of the present invention. In this embodiment, the machine vision-based zinc spangle rating device 500 includes a segmentation and annotation module 501, a model training module 502, an image fitting module 503, and a zinc spangle rating module 504.

[0064] The segmentation and labeling module 501 is used to segment multiple zinc flower images according to a preset pixel size and manually label the core points of the zinc flower shape to obtain the corresponding unit images and labels.

[0065] The model training module 502 is used to input the unit image and its label into the neural network model for training and verification, so as to output the density map of the zinc flower core points of each unit image, thereby obtaining the number of core points of each unit image.

[0066] The image fitting module 503 is used to fit the density map of each unit image using a clustering algorithm to obtain the nucleation point coordinates of each zinc flower in the unit image, thereby obtaining the nearest neighbor distance of the nucleation point and calculating the average value of the nearest neighbor distance of the nucleation point corresponding to the unit image.

[0067] The zinc flower rating module 504 is used to take the number of nucleus points and the average of the nearest neighbor distance of the nucleus points in each unit image as the feature vector for zinc flower rating, input it into the classifier for training to obtain the zinc flower classification model, and output the corresponding automatic rating result after inputting the zinc flower image to be rated into the zinc flower rating model; the rating model includes a convolutional neural network, a Gaussian mixture model and a classification model.

[0068] In some examples, the segmentation and annotation module 501 preprocesses the zinc flower image using image enhancement and image smoothing algorithms before segmenting the zinc flower image, in order to enhance the image contrast and eliminate image noise interference.

[0069] In some examples, the image fitting module 503 uses a clustering algorithm to fit the density map of each unit image to obtain the nucleation point coordinates of each zinc flower in the unit image, and obtains the nearest neighbor distance of the nucleation point accordingly. This includes: using a clustering algorithm to fit the density map of each unit image, extracting the center point position of the cluster in the fitting result, and obtaining the nucleation point coordinates of the zinc flower image accordingly; searching for the nearest neighboring zinc flower nucleation point based on the nucleation point coordinates of the zinc flower image, and obtaining the nearest neighbor distance of the nucleation point of the zinc flower image accordingly.

[0070] In some examples, the model training module 502 inputs unit images and their labels into the neural network model for training and validation, including inputting a portion of the unit images and their labels as a training set into the convolutional neural network or attention model for training; and using the remaining unit images and their labels as a test set to test and validate the trained neural network model.

[0071] In some examples, the clustering algorithm includes any of the following: Gaussian mixture model clustering algorithm, K-means clustering algorithm, DBSCAN clustering algorithm, OPTICS clustering algorithm, or BIRCH clustering algorithm.

[0072] In some examples, the classification model includes any of the following: decision tree classification model, support vector machine classification model, random forest classification model, or Bayesian classification model.

[0073] It should be noted that the machine vision-based zinc spangle rating device provided in the above embodiments is only illustrated by the division of the above-described program modules when performing machine vision-based zinc spangle rating. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the machine vision-based zinc spangle rating device and the machine vision-based zinc spangle rating method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0074] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented using computer program-related hardware. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0075] In the embodiments provided in this application, the computer-readable and writable storage medium may include read-only memory, random access memory, EEPROM, CD-ROM or other optical disc storage devices, disk storage devices or other magnetic storage devices, flash memory, USB flash drive, portable hard drive, or any other medium capable of storing desired program code in the form of instructions or data structures and accessible by a computer. Additionally, any connection may be appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of the medium. However, it should be understood that computer-readable and writable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but are intended for non-transient, tangible storage media. The disks and optical discs used in the application include compact discs (CDs), laser discs, optical discs, digital multifunction discs (DVDs), floppy disks, and Blu-ray discs, where disks typically copy data magnetically, while optical discs use lasers to copy data optically.

[0076] In summary, this application provides a zinc spangle grading method, apparatus, terminal, and medium based on machine vision. The zinc spangle grading method proposed in this invention not only automatically and accurately determines the grade of zinc spangle sheets but also characterizes the essential features of zinc spangles, including the number of zinc spangle nuclei and the nearest neighbor nuclei for each nucleus. This invention enables in-service zinc spangle quality inspection without manual inspection, utilizing machine vision and artificial intelligence technologies to automatically grade zinc spangles. This not only avoids false detections caused by manual inspection but also significantly improves the efficiency and traceability of zinc spangle grading. Therefore, this application effectively overcomes the various shortcomings of existing technologies and has high industrial application value.

[0077] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A machine vision based zinc flower grading method, characterized by, The method comprises the following steps: segmenting multiple zinc flower images according to a preset pixel size and manually labeling zinc flower nucleation points to obtain corresponding unit images and labels; training and verifying the unit images and the labels in a neural network model to output a density map of zinc flower nucleation points of each unit image, and to obtain the number of zinc flower nucleation points of each unit image; using a clustering algorithm to fit the density map of each unit image to obtain the position coordinates of zinc flower nucleation points in the unit image, to obtain the nearest neighbor distance of the zinc flower nucleation points, and to calculate the average value of the nearest neighbor distance of the zinc flower nucleation points corresponding to the unit image; using the number of zinc flower nucleation points of each unit image and the average value of the nearest neighbor distance of the zinc flower nucleation points as a feature vector of zinc flower rating, inputting the feature vector into a classifier for training to obtain a zinc flower classification model, and inputting a zinc flower image to be rated into the zinc flower rating model to output an automatic rating result corresponding to the zinc flower image; the rating model comprises a convolutional neural network, a Gaussian mixture model, and a classification model.

2. The machine vision-based zinc flower grading method according to claim 1, characterized in that: Before segmenting the zinc flower image, an image enhancement algorithm and an image smoothing algorithm are used to pre-process the zinc flower image to enhance the contrast of the image and eliminate noise interference.

3. The machine vision-based zinc flower grading method according to claim 1, wherein, The clustering algorithm is used to fit the density map of each unit image to obtain the position coordinates of zinc flower nucleation points in the unit image and to calculate the nearest neighbor distance of the zinc flower nucleation points, which comprises the following steps: using a clustering algorithm to fit the density map of each unit image, extracting the center point position of the cluster in the fitting result, and obtaining the position coordinates of zinc flower nucleation points of the zinc flower image; searching for the nearest neighbor zinc flower nucleation point closest to the zinc flower nucleation point of the zinc flower image according to the position coordinates of the zinc flower nucleation point of the zinc flower image, and obtaining the nearest neighbor distance of the zinc flower nucleation point of the zinc flower image.

4. The machine vision-based zinc flower grading method according to claim 1, wherein, The training and verification of the unit images and the labels in the neural network model comprises the following steps: inputting part of the unit images and the labels as a training set into a convolutional neural network or an attention model for training; and inputting the remaining unit images and the labels as a test set to test and verify the trained neural network model.

5. The machine vision-based zinc flower grading method according to claim 1, wherein, The clustering algorithm comprises any one of the following: a Gaussian mixture model clustering algorithm, a K-means clustering algorithm, a DBSCAN clustering algorithm, an OPTICS clustering algorithm, or a BIRCH clustering algorithm.

6. The machine vision-based zinc flower grading method according to claim 1, wherein, The classification model comprises any one of the following: a decision tree classification model, a support vector machine classification model, a random forest classification model, or a Bayesian classification model.

7. The machine vision based zinc flower grading method as claimed in claim 1 wherein, The manual labeling process of zinc flower nucleation points comprises the following steps: after labeling an arbitrary zinc flower nucleation point, labeling the adjacent zinc flower nucleation points based on the position of the zinc flower nucleation point, and labeling all zinc flower nucleation points on the zinc flower image.

8. A machine vision based zinc flower grading device, characterized by, The method comprises the following steps: a segmentation and labeling module for segmenting multiple zinc flower images according to a preset pixel size and manually labeling zinc flower nucleation points to obtain corresponding unit images and labels; a model training module for training and verifying the unit images and the labels in a neural network model to output a density map of zinc flower nucleation points of each unit image, and to obtain the number of zinc flower nucleation points of each unit image; an image fitting module configured to fit a density map of each of the unit images using a clustering algorithm to obtain a position coordinate of a nucleation point of each of the zinc flowers in the unit image, based on which a nearest neighbor distance of the nucleation point is obtained and an average value of the nearest neighbor distances of the nucleation points corresponding to the unit image is calculated; a zinc flower classification module configured to input the number of the nucleation points of each of the unit images and the average value of the nearest neighbor distances of the nucleation points as a feature vector of a zinc flower rating into a classifier to obtain a zinc flower classification model after training, and to input a zinc flower image to be rated into the zinc flower rating model to output a corresponding automatic rating result.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the machine vision-based zinc flower rating method of any one of claims 1 to 7.

10. An electronic terminal, characterized in that comprising: a processor and a memory; the memory is configured to store a computer program; the processor is configured to execute the computer program stored in the memory, so that the terminal executes the machine vision-based zinc flower rating method of any one of claims 1 to 7.

Images