A chip pin rust spot identification method, device and storage medium

Abstract

The present disclosure provides a method, device, and storage medium for identifying rust spots on chip pins, including: using machine vision to photograph chip pins from multiple angles to obtain chip pin images; inputting the grayscale values ​​of the chip pin image pixels into a chip pin rust spot identification and classification model to determine the category to which the chip pin image belongs; wherein the chip pin rust spot identification and classification model is constructed based on a convolutional neural network model and a hidden Markov model; and determining whether the chip pin image has rust spots based on the category to which the chip pin image belongs. Compared with the prior art, the chip pin rust spot identification and classification model of the present disclosure is constructed based on a convolutional neural network model and a hidden Markov model, has a high degree of automation, can complete the detection of a large number of chip pins in a short period of time, is suitable for rapid quality detection in large-scale production, and significantly improves the accuracy of chip pin rust spot judgment.

Description

Technical Field

[0001] The present disclosure relates to the field of semiconductor technology, and in particular to a method, device, and storage medium for identifying rust spots on chip pins. Background Art

[0002] Bulk purchasing is a common method for reducing chip procurement costs. Vacuum-dried and airtight, chips typically last a long time. However, improper storage, particularly when the packaging is damaged, can lead to rust on the chip pins. In printed circuit board assembly, due to the chip bonding and soldering process, manually sorting chips from damaged packaging individually is insufficient to meet production line requirements. Therefore, machine vision is a necessary screening method for identifying rust on chip pins. Machine vision technology uses optical imaging equipment to capture images of chip pins and analyzes and processes them using computer algorithms to identify rust spots. However, traditional machine vision methods have significant limitations in identifying rust on chip pins, resulting in low accuracy. This is primarily due to the diverse shapes and sizes of rust spots on chip pins, as well as the complex variations in their image features under varying lighting conditions. Traditional machine vision algorithms often struggle to fully and accurately extract the characteristic information of rust spots, resulting in frequent misidentifications and missed detections. Therefore, there is an urgent need for a method to improve the accuracy of identifying rust spots on chip pins. Summary of the Invention

[0003] The embodiments of the present disclosure provide a method, device, and storage medium for identifying rust spots on chip pins, so as to solve the problem of low accuracy of existing machine vision algorithms in identifying rust spots on chip pins.

[0004] In view of the above problems, in a first aspect, an embodiment of the present disclosure provides a method for identifying rust spots on chip pins, comprising:

[0005] Use machine vision to shoot chip pins at multiple angles to obtain chip pin images;

[0006] Inputting the grayscale value of the chip pin image pixel into a chip pin rust spot recognition and classification model to determine the category to which the chip pin image belongs; wherein the chip pin rust spot recognition and classification model is constructed based on a convolutional neural network model and a hidden Markov model;

[0007] Determine whether there is rust on the chip pin image according to the category to which the chip pin image belongs.

[0008] In conjunction with the first aspect, in one possible implementation, inputting the grayscale values ​​of the chip pin image pixels into a chip pin rust spot recognition and classification model to determine the category to which the chip pin image belongs includes:

[0009] Inputting the grayscale values ​​of the chip pin image pixels into a convolutional neural network model for feature extraction to obtain a first number of two-dimensional rust spot feature matrices;

[0010] Using Peano scanning to convert the first number of two-dimensional rust spot feature matrices into a first number of hidden Markov chains;

[0011] The first number of hidden Markov chains are input into a hidden Markov model to determine the category to which the chip pin image belongs.

[0012] In combination with the first aspect, in one possible implementation, inputting the grayscale values ​​of the chip pin image pixels into a convolutional neural network model for feature extraction to obtain a first number of two-dimensional rust feature matrices includes:

[0013] Inputting the grayscale values ​​of the chip pin image pixels into the input layer of the convolutional neural network model;

[0014] Performing feature extraction on the grayscale values ​​of the chip pin image pixels through a preset number of convolutional layers, activation functions, and pooling layers in a convolutional neural network model, and outputting a first number of two-dimensional rust spot feature matrices through an output layer;

[0015] The convolutional neural network model uses the ReLU function as the activation function and the Softmax function as the output layer.

[0016] In conjunction with the first aspect, in a possible implementation, inputting the first number of hidden Markov chains into a hidden Markov model to determine the category to which the chip pin image belongs includes:

[0017] Determining the first number of hidden Markov chains as observable quantities of the first number of rust spot features of the hidden Markov model;

[0018] Determining, based on the observable quantities of the first number of rust spot features, a conditional probability product of a hidden state transition probability matrix under the observable quantities of the first number of rust spot features;

[0019] An expectation-maximization algorithm is used to iteratively determine the parameters of the hidden Markov model until an iteration stopping condition is met. During each iteration, a Bayesian marginal posterior model is used to determine the probability that the hidden state is the category to which the chip pin image belongs, based on the conditional probability product of the hidden state transition probability matrix under the observable quantities of the first number of rust spot features and the parameters of the current generation of the hidden Markov model.

[0020] In combination with the first aspect, in one possible implementation, the parameters of the hidden Markov model include: a state probability vector, a state transition probability matrix, and an observation probability matrix;

[0021] The method further comprises determining the probability that the hidden state is the category to which the chip pin image belongs using a Bayesian marginal posterior model based on the conditional probability product of the hidden state transition probability matrix under the observable quantities of the first number of rust spot features and the parameters of the current generation hidden Markov model, including:

[0022] Determine the forward probability and backward probability of a Bayesian marginal posterior model based on the conditional probability product of the hidden state transition probability matrix under the observable quantities of the first number of rust spot features, the state probability vector in the parameters of the current generation of the hidden Markov model, the state transition probability matrix, and the observation probability matrix;

[0023] According to the product of the forward probability and the backward probability, the probability that the hidden state is the category to which the chip pin image belongs is determined.

[0024] In conjunction with the first aspect, in one possible implementation, the iterative determination of the parameters of the hidden Markov model using the expectation maximization algorithm until an iteration stop condition is satisfied includes:

[0025] Iteratively determining an expected covariance value of the observable quantities of the first number of rust spot features and parameters of a current generation hidden Markov model using an expectation-maximization algorithm based on the observable quantities and hidden variables of the first number of rust spot features, until the iteration is terminated when the covariance value of the observable quantities of the first number of rust spot features is less than or equal to a first preset value;

[0026] The hidden variables include: a state transition probability matrix and covariance values ​​of observable quantities of the expected first number of rust spot features.

[0027] In conjunction with the first aspect, in one possible implementation, determining whether the chip pin image has rust spots according to the category to which the chip pin image belongs includes:

[0028] Determining a linear weighted value of the observation probability matrix in the parameters of the corresponding hidden Markov model according to the maximum value of the probability that the hidden state is the category to which the chip pin image belongs;

[0029] When the linear weighted value is greater than a second preset value, determining whether the chip pin image has rust spots according to the category of the chip pin image corresponding to the maximum value of the probability that the hidden state is the category of the chip pin image;

[0030] The categories to which the chip pin image belongs include: the chip pin image with rust spots, the chip pin image without rust spots; and / or the level of the chip pin image with rust spots.

[0031] In conjunction with the first aspect, in one possible implementation, the chip pin rust spot recognition and classification model is trained in the following manner:

[0032] Screening the chip pin image to determine training data for the chip pin rust spot recognition and classification model;

[0033] Inputting the grayscale values ​​of the pixels of the training data into the chip pin rust spot recognition and classification model, and determining the recall rate, precision, F1 score and accuracy of the chip pin rust spot recognition and classification model;

[0034] When the recall rate is greater than or equal to the third preset value, the precision is greater than or equal to the fourth preset value, the F1 score is greater than or equal to the fifth preset value, and the accuracy is greater than or equal to the sixth preset value, the training of the chip pin rust spot recognition and classification model is completed.

[0035] In a second aspect, a device for identifying rust spots on chip pins is provided, comprising:

[0036] An image acquisition module is used to shoot chip pins from multiple angles using machine vision to obtain chip pin images;

[0037] The chip pin rust spot recognition module is used to input the grayscale value of the chip pin image pixel into the chip pin rust spot recognition and classification model to determine the category to which the chip pin image belongs; wherein the chip pin rust spot recognition and classification model is constructed based on a convolutional neural network model and a hidden Markov model; based on the category to which the chip pin image belongs, it is determined whether the chip pin image has rust spots.

[0038] In a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for identifying chip pin rust as described in the first aspect or any possible embodiment in combination with the first aspect are executed.

[0039] The beneficial effects of the embodiments of the present disclosure include:

[0040] The present disclosure provides a method, device and storage medium for identifying rust spots on chip pins, including: using machine vision to shoot chip pins at multiple angles to obtain chip pin images; inputting the grayscale values ​​of the chip pin image pixels into a chip pin rust spot identification and classification model to determine the category to which the chip pin image belongs; wherein the chip pin rust spot identification and classification model is constructed based on a convolutional neural network model and a hidden Markov model; and determining whether there are rust spots on the chip pin image according to the category to which the chip pin image belongs. The method for identifying rust spots on chip pins provided by the embodiment of the present disclosure uses machine vision technology to image chip pins with high resolution and high precision. Compared with the existing technology, the chip pin rust spot identification and classification model is constructed based on a convolutional neural network model and a hidden Markov model, has a high degree of automation, can complete the detection of a large number of chip pins in a short time, greatly improves the detection efficiency, is suitable for rapid quality detection in large-scale production, and has significantly improved the accuracy of chip pin rust spot judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A flowchart of a method for identifying rust spots on chip pins provided in an embodiment of the present disclosure;

[0042] Figure 2 A schematic diagram of a comparison of recall rates provided in an embodiment of the present disclosure;

[0043] Figure 3 A schematic diagram of accuracy comparison provided by an embodiment of the present disclosure;

[0044] Figure 4 A schematic diagram of the comparison of F1 scores provided in an embodiment of the present disclosure;

[0045] Figure 5 A schematic diagram of accuracy comparison provided by an embodiment of the present disclosure;

[0046] Figure 6 This is a structural diagram of the device for identifying rust spots on chip pins provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0047] The present disclosure provides a method, device, and storage medium for identifying chip pin rust. Preferred embodiments of the present disclosure are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are intended only to illustrate and explain the present disclosure and are not intended to limit the present disclosure. Furthermore, the embodiments and features within the embodiments of the present disclosure may be combined with one another unless there is a conflict.

[0048] The present disclosure provides a method for identifying rust spots on chip pins. Figure 1 Shown, including:

[0049] S101, using machine vision to photograph chip pins from multiple angles to obtain chip pin images;

[0050] S102, inputting the grayscale value of the chip pin image pixel into a chip pin rust spot recognition and classification model to determine the category to which the chip pin image belongs; wherein the chip pin rust spot recognition and classification model is constructed based on a convolutional neural network model and a hidden Markov model;

[0051] S103 : Determine whether there is rust on the chip pin image according to the category to which the chip pin image belongs.

[0052] The disclosed embodiments are applied to the field of semiconductor technology, particularly in printed circuit board assembly (PCBA) processes, to identify and classify rust spots on chip pins. In the modern electronics industry, chips are core components, and their quality and performance directly impact the quality of electronic products. Bulk purchasing is widely adopted by many companies as an effective means of reducing chip procurement costs. In a vacuum-dried, sealed environment, chips can be stored for long periods of time with stable performance. However, improper storage, particularly damage to the chip packaging, can easily lead to rust spots on chip pins during long-term storage. Chip pins are the key connection between the chip and the external circuit. Due to the chip bonding and soldering process in PCBA, manually sorting chips from damaged packaging individually is inefficient and difficult to meet the demands of large-scale, high-speed production. Therefore, the introduction of machine vision technology to identify rust spots on chip pins has become a necessary screening measure to improve production efficiency and product quality. Machine vision technology uses optical imaging equipment to capture images of chip pins and uses machine vision algorithms to analyze and process the images to identify rust spots. However, traditional machine vision methods have significant limitations in identifying rust spots on chip pins, resulting in low accuracy. This is primarily due to the fact that rust spots on chip pins vary in shape and size, and their image features vary complexly under different lighting conditions. Traditional machine vision algorithms often struggle to fully and accurately extract the characteristic information of rust spots, resulting in frequent misidentifications and missed detections.

[0053] In the disclosed embodiments, machine vision is used to capture chip pins from multiple angles, producing clear images of the chip pins. Machine vision utilizes a high-resolution industrial camera, a multi-axis motion control system, and an optical lighting system to capture chip pin images from various angles. For example, machine vision uses a rotating platform or robotic arm to adjust the chip's position, for example, to 0°, 45°, and 90°, eliminating blind spots associated with a single viewing angle and enabling multi-angle imaging of the chip pins. For reflective surfaces of the chip, multiple exposures are used to synthesize a clear image, enhancing the contrast between the chip pins and the background. For chip pin images, a pixel is the smallest unit of an image, representing a point in the image. In chip pin images, each pixel has one or more numerical values ​​describing its color and brightness. Chip pin images can include both grayscale and color images. In grayscale images, each pixel has only one numerical value, the grayscale value. The grayscale value represents the brightness of the pixel, typically expressed as an integer between 0 and 255. In a grayscale image, 0 represents pure black, 255 represents pure white, and values ​​in between represent varying shades of gray. The range of grayscale values ​​depends on the image's bit depth. For example, the grayscale value range of an 8-bit grayscale image is 0 to 255, while the grayscale value range of a 16-bit grayscale image is 0 to 65535. The higher the grayscale value, the brighter the pixel; the lower the grayscale value, the darker the pixel. Grayscale images are single-channel images, and each pixel has only one grayscale value. For color images, such as RGB images, each pixel has three channels, namely red, green, and blue channels, and each channel has a corresponding grayscale value. Color images can also be converted to grayscale images, for example, by using a formula to merge the grayscale values ​​of the three channels into one grayscale value Gray. The formula is as follows:

[0054] Gray=0.299×R+0.587×G+0.114×B

[0055] Here, R, G, and B represent the grayscale values ​​of the red, green, and blue channels, respectively. Grayscale values ​​can be used to identify and process different regions in an image. In chip pin detection, grayscale values ​​can be used to extract features such as the pin's shape and texture. For example, by calculating changes in grayscale values, the pin's edge can be detected, thereby determining its position and shape. For detecting rust on chip pins, changes in grayscale values ​​can reflect the rust condition on the pin's surface. The grayscale values ​​of rusted areas typically differ from those of normal pin areas. Analyzing the distribution of grayscale values ​​can be used to identify rust on chip pins. Before inputting the grayscale values ​​of the chip pin image pixels into the chip pin rust detection and classification model, the chip pin image can be preprocessed using grayscaling, denoising, and contrast enhancement. Grayscaling converts a color image into a grayscale image. Denoising reduces noise interference in the image, and contrast enhancement makes the grayscale difference between the pin area and the background more pronounced.

[0056] Furthermore, the grayscale values ​​of the chip pin image pixels are input into the chip pin rust spot recognition and classification model. This model is constructed based on a convolutional neural network model and a hidden Markov model. Convolutional neural networks (CNNs) have powerful image feature extraction capabilities and can automatically learn complex features in images. Hidden Markov models (HMMs) can model temporal or spatial correlations in image sequences. Combining these two models can more accurately identify rust spots in chip pin images. After processing by the chip pin rust spot recognition and classification model, the chip pin images can be classified into different categories, and the category to which the chip pin images belong is determined. The categories to which the pin images belong can include: presence of rust spots on the chip pin images; absence of rust spots on the chip pin images; or the level of rust presence in the chip pin images, such as severe rust spots, mild rust spots, or absence of rust spots. Based on the category to which the chip pin images belong, the presence of rust spots on the chip pins can be determined.

[0057] In the embodiments of the present application, machine vision technology can be used to achieve high-resolution and high-precision imaging of chip pins. On this basis, the chip pin rust spot recognition and classification model constructed integrates the convolutional neural network model and the hidden Markov model, and has a high degree of automation. The chip pin rust spot recognition and classification model can complete the inspection of large quantities of chip pins in a very short time, significantly improving the inspection efficiency, and is particularly suitable for the rapid quality inspection needs in large-scale production scenarios. In addition, the chip pin rust spot recognition and classification model has achieved a significant improvement in the accuracy of chip pin rust spot judgment, providing a strong guarantee for chip quality control.

[0058] In another embodiment of the present disclosure, in the above step S102, the grayscale value of the chip pin image pixel is input into the chip pin rust spot recognition and classification model to determine the category to which the chip pin image belongs, including the following steps:

[0059] Step 1: Input the grayscale values ​​of the chip pin image pixels into a convolutional neural network model for feature extraction to obtain a first number of two-dimensional rust spot feature matrices;

[0060] Step 2: using Peano scanning to convert the first number of two-dimensional rust spot feature matrices into the first number of hidden Markov chains;

[0061] Step 3: Input the first number of hidden Markov chains into the hidden Markov model to determine the category to which the chip pin image belongs.

[0062] In the embodiment of the present disclosure, the grayscale value of the chip pin image is input into the convolutional neural network model to extract the two-dimensional rust feature matrix, which is converted into a hidden Markov chain through Peano scanning and then input into the hidden Markov model for identification and classification. For the above step 1, the grayscale value of the pixel of the chip pin image captured by machine vision is first input into the convolutional neural network model. For example, the convolutional neural network model can construct the corresponding number of input layer nodes based on the number of pixel grayscale values. The input layer sends the grayscale value to the convolution layer, which extracts the grayscale features to obtain a feature matrix. After activation by the activation function, the feature matrix is ​​sent to the pooling layer. The pooling layer condenses the feature matrix and generates a first number of two-dimensional rust feature matrices according to the preset number of features. The output layer outputs the matrix. For the above step 2, Peano scanning is used to convert the first number of two-dimensional rust feature matrices into the first number of hidden Markov chains. Peano scanning traverses each point in the two-dimensional space in an orderly manner, and can arrange the elements in the two-dimensional feature matrix into a one-dimensional sequence in a certain order, thereby retaining the spatial structure information in the matrix. For example, a two-dimensional rust feature matrix M has a size of 3×3.

[0063]

[0064] Peano scanning extracts elements from the matrix M sequentially along the path of the Peano curve, forming a one-dimensional sequence S that preserves the spatial relationships between the elements in the matrix. For example, if the Peano scanning path is: Path = [1, 2, 3, 6, 5, 4, 7, 8, 9], the resulting one-dimensional sequence S is S = [1, 2, 3, 6, 5, 4, 7, 8, 9]. This resulting one-dimensional sequence S is used as a hidden Markov chain. Each two-dimensional rust feature matrix corresponds to a hidden Markov chain. The number of hidden Markov chains is the same as the number of two-dimensional rust feature matrices output by the convolutional neural network model, both of which are the first number. For example, if the convolutional neural network model outputs N two-dimensional rust feature matrices, where N represents the first number, then Peano scanning is performed on these N two-dimensional rust feature matrices to obtain N hidden Markov chains. Regarding step 3 above, all hidden Markov chains obtained by Peano scanning have the following characteristics: each hidden Markov chain can be used as an observable of the rust feature, that is, an observable of the hidden Markov model. The first number of hidden Markov chains are input into a hidden Markov model to determine the category of the chip pin image. The convolutional neural network model can automatically learn the complex features in the chip pin image. The hidden Markov model can model the feature sequence and consider the temporal and spatial correlations between features, thereby accurately classifying the chip pin image and determining whether the chip pin has rust spots.

[0065] In another embodiment of the present disclosure, in the above step 1, the grayscale values ​​of the chip pin image pixels are input into the convolutional neural network model for feature extraction to obtain a first number of two-dimensional rust feature matrices, including the following steps:

[0066] Step 1: Input the grayscale value of the chip pin image pixel into the input layer of the convolutional neural network model;

[0067] Step 2: extracting features from the grayscale values ​​of the chip pin image pixels through a preset number of convolutional layers, activation functions, and pooling layers in the convolutional neural network model, and outputting a first number of two-dimensional rust spot feature matrices through an output layer;

[0068] Among them, the convolutional neural network model uses the ReLU function as the activation function and the Softmax function as the output layer.

[0069] In the embodiment of the present disclosure, a convolutional neural network model is used to extract features from chip pin images. For the above step one, the grayscale values ​​of the chip pin image pixels are input into the input layer of the convolutional neural network model. The input layer is the first layer of the convolutional neural network model. It is used to receive the grayscale values ​​of the chip pin image pixels and pass them to the convolution layer. The number of input layer nodes is related to the number of grayscale values ​​of the chip pin image pixels. Exemplarily, the number of input layer nodes = image width × height × number of channels. For grayscale images, the number of channels is 1, and for color images, such as RGB images, the number of channels is 3. For example, for a grayscale image with a resolution of 640×480 pixels, the number of input layer nodes = 640×480×1 = 307200. For a color image with a resolution of 640×480 pixels, the number of input layer nodes = 640×480×3 = 921600. For the above step two, the convolution layer is used to find the pin rust feature data from the grayscale values ​​received by the input layer. The ReLU function is used as the activation function to convert the linear convolution operation into a nonlinear transformation, which can fit the complex features of the pin rust, set the low response values ​​in the normal area of ​​the pin to zero, highlight the high response features related to the rust, and retain the intensity information of the rust features. The extracted feature data is sent to the pooling layer, which is used to reduce the size of the rust feature matrix while retaining the most critical features. The output layer is used to output the feature matrix of the pooling layer to obtain a first number of two-dimensional rust feature matrices. The Softmax function is used as the output layer, and the probability distribution can be included in the output information, making the output of the convolutional neural network model more intuitive and easy to explain. Each two-dimensional rust feature matrix can represent a feature of the chip pin image, such as a texture feature, totaling a first number of features. Through the above steps, the convolutional neural network model can automatically learn the features in the chip pin image, and use these features to accurately determine whether the pin has rust.

[0070] In another embodiment of the present disclosure, in step 3 above, inputting the first number of hidden Markov chains into the hidden Markov model to determine the category to which the chip pin image belongs includes the following steps:

[0071] Step 1: determining a first number of hidden Markov chains as observable quantities of a first number of rust spot features of the hidden Markov model;

[0072] Step 2: determining the conditional probability product of the hidden state transition probability matrix under the observable quantities of the first number of rust spot features based on the observable quantities of the first number of rust spot features;

[0073] Step 3. Use the expectation-maximization algorithm to iteratively determine the parameters of the hidden Markov model until the iteration stopping condition is met; in each iteration, based on the conditional probability product of the hidden state transition probability matrix under the observable quantities of the first number of rust spot features and the parameters of the current generation hidden Markov model, use the Bayesian marginal posterior model to determine the probability that the hidden state is the category to which the chip pin image belongs.

[0074] In the embodiment of the present disclosure, a hidden Markov model is used to process the first number of hidden Markov chains to determine the category to which the chip pin image belongs. The hidden Markov model is a statistical model. The process of determining the category to which the chip pin image belongs can be regarded as the process of solving the hidden state based on the observable quantity of the hidden Markov model. The hidden Markov model contains two main parts: a hidden state sequence, which represents the internal state changes, and these states are unobservable. Observable quantity sequence: observable data generated by the hidden state. In the classification of chip pin images, the hidden state can be used to represent the category to which the chip pin image belongs, such as normal or rust, and the observable quantity sequence can be derived from the two-dimensional rust feature matrix extracted from the chip pin image. For the above step one, for the hidden Markov model, the first number of hidden Markov chains are known quantities, which can be objective observable quantities. The first number of hidden Markov chains are determined as the observable quantity of the first number of rust features of the hidden Markov model, denoted as Y, then Y i Represents the i-th observable. The hidden Markov chain is the state variable that characterizes the rust feature or the descriptive value of each rust feature. For the above step 2, X is the hidden state transition probability matrix of the hidden Markov model, X i Then the hidden state transition probability is , where i = 1, 2…M, M is the hidden state transition probability X in the hidden Markov model corresponding to each rust feature matrix i The number of hidden state transition probability X i The probability distribution P(X) can be expressed as:

[0075]

[0076] Hidden state transition probability X i Satisfies Gaussian distribution, then the hidden state transition probability X i In the observable quantity Y i The conditional probability under can be expressed as:

[0077]

[0078] in, Represents the conditional probability of the hidden state transition probability matrix X under the condition of known observable value Y. represents the conditional probability product of the observable variable Y in the hidden Markov model and the hidden state transition probability matrix X, that is, the conditional probability product of the hidden state transition probability matrix under the observable variables of the first number of rust spot features. Regarding step three above, the expectation-maximization algorithm is an iterative optimization algorithm commonly used for maximum likelihood estimation of the parameters of probabilistic models with hidden variables. The EM algorithm progressively optimizes the parameters of the hidden Markov model by alternating between two steps: the E-step (expectation step) and the M-step (maximization step) until the iterative stopping condition is met. In each iteration, the probability of the hidden state belonging to the chip pin image's category is determined using a Bayesian marginal posterior model based on the conditional probability product of the hidden state transition probability matrix under the observable variables of the first number of rust spot features and the parameters of the current generation of the hidden Markov model. The Bayesian marginal posterior model (MPM) makes the classification method unsupervised by maximizing the marginal probabilities of the posterior distribution to determine the most likely parameter values. The hidden Markov model that combines the expectation-maximization algorithm with the Bayesian marginal posterior model can be used to accurately determine the category of the chip pin image.

[0079] In yet another embodiment of the present disclosure, the parameters of the hidden Markov model include: a state probability vector, a state transition probability matrix, and an observation probability matrix;

[0080] In step 3 above, the probability that the hidden state belongs to the category of the chip pin image is determined using a Bayesian marginal posterior model based on the conditional probability product of the hidden state transition probability matrix under the observable values ​​of the first number of rust spot features and the parameters of the current generation hidden Markov model, including:

[0081] Step (1) determining the forward probability and backward probability of the Bayesian marginal posterior model based on the conditional probability product of the hidden state transition probability matrix under the observable quantities of the first number of rust spot features, the state probability vector in the parameters of the current generation of the hidden Markov model, the state transition probability matrix and the observation probability matrix;

[0082] Step (2): Determine the probability that the hidden state is the category to which the chip pin image belongs based on the product of the forward probability and the backward probability.

[0083] In the disclosed embodiment, the Bayesian marginal posterior model (MPM) can make the classification method an unsupervised classification method, and estimate the value of the hidden state transition probability matrix X in the hidden Markov model. is the estimated value set of the hidden state transition probability matrix X, . for For the first number N of hidden Markov chains, , all satisfy:

[0084]

[0085] in, is the set to be classified, dimensionless; is the forward probability; For the above step (1), the forward and backward probabilities of the Bayesian marginal posterior model are determined based on the conditional probability product of the hidden state transition probability matrix under the observable quantity of the first number of rust spot features, the state probability vector in the parameters of the current generation of the hidden Markov model, the state transition probability matrix and the observation probability matrix. Forward probability , backward probability . Forward probability and backward probability The recursion is performed according to the following formula:

[0086]

[0087]

[0088]

[0089]

[0090] in, and is the forward probability The recursive formula of . Represents the initial estimate of the hidden state transition probability and the initial value of observable quantity The joint probability of . is the forward probability The recursive formula of . is the final value of the backward probability. It is the recursive formula for forward extrapolation of backward probability.

[0091] Regarding step (2) above, , which can be expressed as the estimated hidden state transition probability is the forward probability and backward probability The product falls into The maximum value is achieved when the class. Class can represent the probability of the category to which the chip pin image belongs. The state probability vector is expressed as , that is, the initial state probability and state transition probability matrix can be expressed as , the observation probability matrix can be expressed as For any one-dimensional hidden Markov chain expanded by a two-dimensional rust feature matrix, the observable quantity Y can be written as: .

[0092]

[0093] Among them, for the i-th hidden Markov chain, is the forward probability, t is the tth step of the iteration; is the t+1th iteration of the backward probability; is the initial state transition probability value of the hidden Markov chain, = , that is, the state transition probability matrix; is the iterative observation probability value, that is, the observation probability matrix. Under these conditions, the hidden state transition probability can be observed The probability value of the class achieving the maximum value can be calculated using the following formula:

[0094] =

[0095]

[0096] It represents the iterative state probability value of the i-th hidden Markov chain; the iterative calculation formula of the iterative state transition probability is expressed as follows:

[0097]

[0098] According to the product of forward probability and backward probability, the hidden state transition probability estimate can be Classify it into the corresponding category with the maximum probability, thus obtaining the maximum probability that the hidden state is the category to which the chip pin image belongs.

[0099] In another embodiment of the present disclosure, in step 3 above, the parameters of the hidden Markov model are iteratively determined using the expectation maximization algorithm until an iteration stop condition is satisfied, including:

[0100] Iteratively determining an expected covariance value of the observable quantities of the first number of rust spot features and parameters of a current generation hidden Markov model using an expectation-maximization algorithm based on observable quantities and hidden variables of the first number of rust spot features, until the iteration is stopped when the expected covariance value of the observable quantities of the first number of rust spot features is less than or equal to a first preset value;

[0101] The hidden variables include: a state transition probability matrix and covariance values ​​of observable quantities of the expected first number of rust spot features.

[0102] In the embodiment of the present disclosure, the iteration is stopped when the covariance value of the observable quantity of the desired first number of rust spot features is less than or equal to the first preset value. The hidden variables include: the state transition probability matrix and the covariance value of the observable quantity of the desired first number of rust spot features. The state transition probability matrix can be expressed as The formula for calculating the covariance value of the observable quantity of the expected first number of rust spot characteristics is:

[0103]

[0104] in The mean of the observable quantity of the first number of expected rust spot characteristics is expressed as follows:

[0105]

[0106] Based on the observables and hidden variables of the first number of rust spot features, the expected covariance value of the observables of the first number of rust spot features and the parameters of the current generation hidden Markov model are iteratively determined using the expectation maximization algorithm. The parameters of the current generation hidden Markov model include the state probability vector represented as , that is, the initial state probability and state transition probability matrix can be expressed as , the observation probability matrix can be expressed as When the covariance value of the observable quantity of each desired rust spot feature is less than or equal to a first preset value, the number of iterations for the observable quantity of each rust spot feature may be different. When the covariance values ​​of the observable quantities of the desired first number of rust spot features are less than or equal to the first preset value, the iteration is stopped.

[0107] In another embodiment of the present disclosure, in the above step S103, determining whether the chip pin image has rust spots according to the category to which the chip pin image belongs includes:

[0108] Step 1: Determine the linear weighted value of the observation probability matrix in the parameters of the corresponding hidden Markov model according to the maximum value of the probability that the hidden state is the category to which the chip pin image belongs;

[0109] Step 2: When the linear weighted value is greater than a second preset value, determine whether the chip pin image has rust spots based on the category of the chip pin image corresponding to the maximum probability that the hidden state is the category of the chip pin image;

[0110] The categories to which the chip pin image belongs include: the chip pin image having rust spots, the chip pin image not having rust spots; and / or the level of the chip pin image having rust spots.

[0111] In the embodiment of the present disclosure, the observation probability matrix in the parameters of the corresponding hidden Markov model is subjected to linear weighted calculation. When the obtained linear weighted value is greater than the second preset value, it is considered that the maximum value of the probability that the hidden state determined by the hidden Markov model is the category to which the chip pin image belongs is a valid value. With respect to the above step 1, during the iteration process, the corresponding observation probability matrix can be obtained by the maximum value of the probability that the hidden state is the category to which the chip pin image belongs. Since the two-dimensional rust feature matrix corresponds to the observation probability value obtained by the hidden Markov chain, that is, the number of iterations of the expected observable quantity of each rust feature may be different, the number of iterations corresponding to the maximum value of the probability that the hidden state is the category to which the chip pin image belongs may also be different, and the preset The observed probability value obtained by the hidden Markov chain corresponding to the first two-dimensional rust feature matrix The weight of for After t1 iterations, the final calculated observation probability value; The observed probability value obtained by the hidden Markov chain corresponding to the second two-dimensional rust feature matrix The weight of for After t2 iterations, the calculated observation probability value; similarly, The observed probability value obtained by the hidden Markov chain corresponding to the Nth two-dimensional rust feature matrix The weight of , the first quantity can be represented by N; where,

[0112]

[0113] The linear weighted values ​​of the observation probability matrix in the corresponding hidden Markov model parameters are: With respect to step 2 above, when the linear weighted value is greater than the second preset value, for example, Q>0.5, determine whether the chip pin image has rust spots based on the chip pin image category corresponding to the maximum probability of the hidden state being the category to which the chip pin image belongs. The category to which the pin image belongs may include: the chip pin image has rust spots, the chip pin image does not have rust spots; or the level of rust spots on the chip pin image, such as severe rust spots, mild rust spots, no rust spots, etc.

[0114] In another embodiment of the present disclosure, the chip pin rust spot recognition and classification model is trained in the following manner:

[0115] Step 1: Screen the chip pin images to determine the training data for the chip pin rust spot recognition and classification model;

[0116] Step 2: Input the grayscale values ​​of the pixels of the training data into the chip pin rust spot recognition and classification model to determine the recall rate, precision, F1 score and accuracy of the chip pin rust spot recognition and classification model;

[0117] Step 3: When the recall rate is greater than or equal to the third preset value, the precision is greater than or equal to the fourth preset value, the F1 score is greater than or equal to the fifth preset value, and the accuracy is greater than or equal to the sixth preset value, the training of the chip pin rust spot recognition and classification model is completed.

[0118] In the embodiment of the present disclosure, when the recall rate, precision and F1 score of the chip pin rust spot recognition and classification model meet the requirements, the training of the chip pin rust spot recognition and classification model is completed. For the above step 1, the chip pin images are screened, and the screened images may have typical rust features, and the screened chip pin images are determined as the training data of the chip pin rust spot recognition and classification model. For the above step 2, the grayscale values ​​of the pixels of the training data are input into the chip pin rust spot recognition and classification model to determine the recall rate, precision, F1 score and accuracy of the chip pin rust spot recognition and classification model. For the above step 3, when the recall rate is greater than or equal to the third preset value, the precision is greater than or equal to the fourth preset value, the F1 score is greater than or equal to the fifth preset value and the accuracy is greater than or equal to the sixth preset value, the training of the chip pin rust spot recognition and classification model is completed. A comparison model is introduced, and the comparison model includes a convolutional neural network model and a convolutional neural network-support vector machine model. The calculation formula for the recall rate is: true positive examples / (true positive examples + false negative examples). As Figure 2 As shown in the figure, the chip pin rust spot recognition and classification model is much better than the simple convolutional neural network classification model and the convolutional neural network-support vector machine classification model in terms of recall rate. As the number of samples increases, the recall rate growth trend of the chip pin rust spot recognition and classification model is not as good as that of the single convolutional neural network, but the recall rate data performance is better under the small sample state. The convolutional neural network-support vector machine classification model shows a relatively stable recall rate growth value as the number of training samples increases, and the recall rate under the small sample state is also at a medium level among the three classification models. From the perspective of recall rate, the convolutional neural network performs better than the convolutional neural network-support vector machine classification model when the number of samples exceeds 500. The accuracy calculation formula is: true positive examples / (true positive examples + false positive examples). As shown in the figure, the chip pin rust spot recognition and classification model has a relatively stable recall rate growth value as the number of training samples increases, and the recall rate under the small sample state is also at a medium level among the three classification models. From the perspective of recall rate, the convolutional neural network performs better than the convolutional neural network-support vector machine classification model when the number of samples exceeds 500. The accuracy calculation formula is: true positive examples / (true positive examples + false positive examples). Figure 3As shown in the figure, the performance of the three classification models is relatively consistent with expectations. As the number of samples increases, the accuracy of each model increases. Among them, the chip pin rust recognition classification model is more accurate than the simple convolutional neural network model and the convolutional neural network-support vector machine model. The F1 score calculation formula is: 2 * precision * recall / (precision + recall). Figure 4 As shown in the figure, the F1 score increases with the number of samples. The chip pin rust spot recognition and classification model outperforms the simple convolutional neural network model and the convolutional neural network-support vector machine model in terms of F1 score. As the number of samples increases, the F1 value increase rate of the convolutional neural network model relative to the convolutional neural network-support vector machine model becomes larger. The accuracy calculation formula is: (true positive examples + true negative examples) / (true positive examples + false positive examples + true negative examples + false negative examples). Figure 5 As shown, the three classification models performed relatively well as expected, with the chip pin rust spot recognition classification model achieving the highest accuracy. The accuracy of all three classification models increased with increasing sample size. The convolutional neural network-support vector machine model outperformed the simple convolutional neural network model in terms of accuracy. Training the chip pin rust spot recognition classification model, when the recall rate is greater than or equal to the third preset value, the precision is greater than or equal to the fourth preset value, the F1 score is greater than or equal to the fifth preset value, and the accuracy is greater than or equal to the sixth preset value, can improve the accuracy of chip pin rust spot recognition.

[0119] Based on the same disclosed concept, the embodiments of the present disclosure also provide a device for identifying rust spots on chip pins. Since the principles of the problems solved by these devices are similar to the aforementioned method for determining the inclination angle of the photovoltaic array, the implementation of the device can refer to the implementation of the aforementioned method, and the repeated parts will not be repeated.

[0120] The present disclosure provides a device for identifying rust spots on chip pins. Figure 6 Shown, including:

[0121] The image acquisition module 601 is used to use machine vision to shoot chip pins from multiple angles to obtain chip pin images;

[0122] The chip pin rust spot recognition module 602 is used to input the grayscale value of the chip pin image pixel into the chip pin rust spot recognition and classification model to determine the category to which the chip pin image belongs; wherein the chip pin rust spot recognition and classification model is constructed based on a convolutional neural network model and a hidden Markov model; and determine whether the chip pin image has rust spots based on the category to which the chip pin image belongs.

[0123] In another embodiment of the present disclosure, the chip pin rust spot recognition module 602 is configured to input the grayscale values ​​of the chip pin image pixels into a convolutional neural network model for feature extraction to obtain a first number of two-dimensional rust spot feature matrices;

[0124] Using Peano scanning to convert the first number of two-dimensional rust spot feature matrices into a first number of hidden Markov chains;

[0125] The first number of hidden Markov chains are input into a hidden Markov model to determine the category to which the chip pin image belongs.

[0126] In another embodiment of the present disclosure, the chip pin rust spot recognition module 602 is configured to input the grayscale value of the chip pin image pixel into the input layer of the convolutional neural network model;

[0127] Performing feature extraction on the grayscale values ​​of the chip pin image pixels through a preset number of convolutional layers, activation functions, and pooling layers in a convolutional neural network model, and outputting a first number of two-dimensional rust spot feature matrices through an output layer;

[0128] The convolutional neural network model uses the ReLU function as the activation function and the Softmax function as the output layer.

[0129] In another embodiment of the present disclosure, the chip pin rust spot identification module 602 is configured to determine the first number of hidden Markov chains as observable quantities of the first number of rust spot features of the hidden Markov model;

[0130] Determining, based on the observable quantities of the first number of rust spot features, a conditional probability product of a hidden state transition probability matrix under the observable quantities of the first number of rust spot features;

[0131] An expectation-maximization algorithm is used to iteratively determine the parameters of the hidden Markov model until an iteration stopping condition is met. During each iteration, a Bayesian marginal posterior model is used to determine the probability that the hidden state is the category to which the chip pin image belongs, based on the conditional probability product of the hidden state transition probability matrix under the observable quantities of the first number of rust spot features and the parameters of the current generation of the hidden Markov model.

[0132] In yet another embodiment of the present disclosure, the parameters of the hidden Markov model include: a state probability vector, a state transition probability matrix, and an observation probability matrix;

[0133] The chip pin rust spot identification module 602 is configured to determine the forward probability and backward probability of a Bayesian marginal posterior model based on the conditional probability product of the hidden state transition probability matrix under the observable quantities of the first number of rust spot features, the state probability vector in the parameters of the current generation hidden Markov model, the state transition probability matrix, and the observation probability matrix;

[0134] According to the product of the forward probability and the backward probability, the probability that the hidden state is the category to which the chip pin image belongs is determined.

[0135] In another embodiment of the present disclosure, the chip pin rust spot identification module 602 is configured to iteratively determine a covariance value of the observable quantities of the first number of rust spot features and parameters of a current generation hidden Markov model using an expectation-maximization algorithm based on the observable quantities and hidden variables of the first number of rust spot features, and stop the iteration when the covariance value of the observable quantities of the first number of rust spot features is less than or equal to a first preset value;

[0136] The hidden variables include: a state transition probability matrix and covariance values ​​of observable quantities of the expected first number of rust spot features.

[0137] In another embodiment of the present disclosure, the chip pin rust spot recognition module 602 is configured to determine a linear weighted value of the observation probability matrix in the corresponding hidden Markov model parameters according to the maximum value of the probability that the hidden state is the category to which the chip pin image belongs;

[0138] When the linear weighted value is greater than a second preset value, determining whether the chip pin image has rust spots according to the category of the chip pin image corresponding to the maximum value of the probability that the hidden state is the category of the chip pin image;

[0139] The categories to which the chip pin image belongs include: the chip pin image with rust spots, the chip pin image without rust spots; and / or the level of the chip pin image with rust spots.

[0140] In another embodiment of the present disclosure, the chip pin rust spot recognition module 602 is configured to train the chip pin rust spot recognition and classification model in the following manner:

[0141] Screening the chip pin image to determine training data for the chip pin rust spot recognition and classification model;

[0142] Inputting the grayscale values ​​of the pixels of the training data into the chip pin rust spot recognition and classification model, and determining the recall rate, precision, F1 score and accuracy of the chip pin rust spot recognition and classification model;

[0143] When the recall rate is greater than or equal to the third preset value, the precision is greater than or equal to the fourth preset value, the F1 score is greater than or equal to the fifth preset value, and the accuracy is greater than or equal to the sixth preset value, the training of the chip pin rust spot recognition and classification model is completed.

[0144] Based on the same disclosed concept, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for identifying chip pin rust as described in any of the above embodiments are executed.

[0145] Through the above description of the embodiments, those skilled in the art will clearly understand that the embodiments of the present disclosure can be implemented through hardware or through software plus the necessary general-purpose hardware platform. Based on this understanding, the technical solutions of the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or mobile hard drive) and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in the various embodiments of the present disclosure.

[0146] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes in the accompanying drawings are not necessarily required for implementing the present disclosure.

[0147] Those skilled in the art will appreciate that the modules in the devices of the embodiments may be distributed in the devices of the embodiments as described in the embodiments, or may be located in one or more devices different from the embodiments with corresponding changes. The modules of the above embodiments may be combined into one module or further split into multiple submodules.

[0148] The serial numbers of the above-mentioned embodiments of the present disclosure are for description only and do not represent the advantages or disadvantages of the embodiments.

[0149] Obviously, those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. A method for identifying rust spots on chip pins, characterized in that: include: Use machine vision to shoot the pins of the stored chip at multiple angles to obtain chip pin images; Inputting the grayscale value of the chip pin image pixel into a chip pin rust spot recognition and classification model to determine the category to which the chip pin image belongs; wherein the chip pin rust spot recognition and classification model is constructed based on a convolutional neural network model and a hidden Markov model; Inputting the grayscale value of the chip pin image pixel into the chip pin rust spot recognition and classification model to determine the category to which the chip pin image belongs includes: Inputting the grayscale values ​​of the chip pin image pixels into a convolutional neural network model for feature extraction to obtain a first number of two-dimensional rust spot feature matrices; Using Peano scanning to convert the first number of two-dimensional rust spot feature matrices into a first number of hidden Markov chains; Inputting the first number of hidden Markov chains into a hidden Markov model to determine the category to which the chip pin image belongs; Inputting the first number of hidden Markov chains into a hidden Markov model to determine the category to which the chip pin image belongs includes: Determining the first number of hidden Markov chains as observable quantities of the first number of rust spot features of the hidden Markov model; Determining, based on the observable quantities of the first number of rust spot features, a conditional probability product of a hidden state transition probability matrix under the observable quantities of the first number of rust spot features; An expectation-maximization algorithm is used to iteratively determine parameters of the hidden Markov model until an iteration stopping condition is satisfied; during each iteration, a Bayesian marginal posterior model is used to determine the probability that the hidden state belongs to the category of the chip pin image based on the conditional probability product of the hidden state transition probability matrix under the observable quantities of the first number of rust spot features and the parameters of the current generation of the hidden Markov model; The parameters of the hidden Markov model include: state probability vector, state transition probability matrix and observation probability matrix; The method further comprises determining the probability that the hidden state is the category to which the chip pin image belongs using a Bayesian marginal posterior model based on the conditional probability product of the hidden state transition probability matrix under the observable quantities of the first number of rust spot features and the parameters of the current generation hidden Markov model, including: Determine the forward probability and backward probability of a Bayesian marginal posterior model based on the conditional probability product of the hidden state transition probability matrix under the observable quantities of the first number of rust spot features, the state probability vector in the parameters of the current generation of the hidden Markov model, the state transition probability matrix, and the observation probability matrix; Determining the probability that the hidden state is the category to which the chip pin image belongs based on the product of the forward probability and the backward probability; Determining whether the chip pin image has rust spots according to the category to which the chip pin image belongs includes: Determining a linear weighted value of the observation probability matrix in the parameters of the corresponding hidden Markov model according to the maximum value of the probability that the hidden state is the category to which the chip pin image belongs; When the linear weighted value is greater than a second preset value, it is determined whether the chip pin image has rust spots according to the category of the chip pin image corresponding to the maximum probability that the hidden state is the category of the chip pin image.

2. The method according to claim 1, wherein The grayscale values ​​of the chip pin image pixels are input into a convolutional neural network model for feature extraction to obtain a first number of two-dimensional rust feature matrices, including: Inputting the grayscale values ​​of the chip pin image pixels into the input layer of the convolutional neural network model; Performing feature extraction on the grayscale values ​​of the chip pin image pixels through a preset number of convolutional layers, activation functions, and pooling layers in a convolutional neural network model, and outputting a first number of two-dimensional rust spot feature matrices through an output layer; The convolutional neural network model uses the ReLU function as the activation function and the Softmax function as the output layer.

3. The method according to claim 1, wherein The method of iteratively determining the parameters of the hidden Markov model using the expectation maximization algorithm until an iteration stopping condition is satisfied includes: Iteratively determining an expected covariance value of the observable quantities of the first number of rust spot features and parameters of a current generation hidden Markov model using an expectation-maximization algorithm based on the observable quantities and hidden variables of the first number of rust spot features, until the iteration is terminated when the covariance value of the observable quantities of the first number of rust spot features is less than or equal to a first preset value; The hidden variables include: a state transition probability matrix and covariance values ​​of observable quantities of the expected first number of rust spot features.

4. The method according to claim 1, wherein The categories to which the chip pin image belongs include: the chip pin image with rust spots, the chip pin image without rust spots; and / or the level of the chip pin image with rust spots.

5. The method according to claim 1, wherein The chip pin rust spot recognition and classification model is trained in the following manner: Screening the chip pin image to determine training data for the chip pin rust spot recognition and classification model; Inputting the grayscale values ​​of the pixels of the training data into the chip pin rust spot recognition and classification model, and determining the recall rate, precision, F1 score and accuracy of the chip pin rust spot recognition and classification model; When the recall rate is greater than or equal to the third preset value, the precision is greater than or equal to the fourth preset value, the F1 score is greater than or equal to the fifth preset value, and the accuracy is greater than or equal to the sixth preset value, the training of the chip pin rust spot recognition and classification model is completed.

6. A device for identifying rust spots on chip pins, characterized in that: include: An image acquisition module is used to use machine vision to shoot the pins of the stored chip from multiple angles to obtain chip pin images; A chip pin rust spot recognition module is used to input the grayscale value of the chip pin image pixel into a chip pin rust spot recognition and classification model to determine the category to which the chip pin image belongs; wherein the chip pin rust spot recognition and classification model is constructed based on a convolutional neural network model and a hidden Markov model; Inputting the grayscale values ​​of the chip pin image pixels into a convolutional neural network model for feature extraction to obtain a first number of two-dimensional rust spot feature matrices; Using Peano scanning to convert the first number of two-dimensional rust spot feature matrices into a first number of hidden Markov chains; Inputting the first number of hidden Markov chains into a hidden Markov model to determine the category to which the chip pin image belongs; Determining the first number of hidden Markov chains as observable quantities of the first number of rust spot features of the hidden Markov model; Determining, based on the observable quantities of the first number of rust spot features, a conditional probability product of a hidden state transition probability matrix under the observable quantities of the first number of rust spot features; An expectation-maximization algorithm is used to iteratively determine parameters of the hidden Markov model until an iteration stopping condition is satisfied; during each iteration, a Bayesian marginal posterior model is used to determine the probability that the hidden state belongs to the category of the chip pin image based on the conditional probability product of the hidden state transition probability matrix under the observable quantities of the first number of rust spot features and the parameters of the current generation of the hidden Markov model; The parameters of the hidden Markov model include: state probability vector, state transition probability matrix and observation probability matrix; Determine the forward probability and backward probability of a Bayesian marginal posterior model based on the conditional probability product of the hidden state transition probability matrix under the observable quantities of the first number of rust spot features, the state probability vector in the parameters of the current generation of the hidden Markov model, the state transition probability matrix, and the observation probability matrix; Determining the probability that the hidden state is the category to which the chip pin image belongs based on the product of the forward probability and the backward probability; determining whether the chip pin image has rust spots according to the category to which the chip pin image belongs; Determining a linear weighted value of the observation probability matrix in the parameters of the corresponding hidden Markov model according to the maximum value of the probability that the hidden state is the category to which the chip pin image belongs; When the linear weighted value is greater than a second preset value, it is determined whether the chip pin image has rust spots according to the category of the chip pin image corresponding to the maximum probability that the hidden state is the category of the chip pin image.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the method for identifying chip pin rust spots according to any one of claims 1 to 5.

Images